vvvv progress

I have been very productive today and organized my datasets into text-files to be read into vvvv. I decided that for now I will be working with two specific datasets: Arrivals and Departures by Country, but narrowed down to regions such as Oceania, the Americas, etc.

I have written a patch which reads these text files into vvvv and separates the data into slices.


So one challenge down, a zillion to go!

The next thing I decided I really needed to spend time on was to play around with the system to see what sorts of visualisations I could create with it, and try and broaden my knowledge of patching in general. I experimented with a few things - had a fit trying to draw bezier curves (evil). I particularly liked the rope patch for DX9.





Another VERY interesting thing i found was the "Camera (SoftImage)" patch, which essentially links keyboard input and mouse clicks to control zoom, pan, rotation etc of the camera! What is great about this is that all patches can be edited... thus I am considering using this existing patch, and editing it to read off the arduino inputs rather than the keyboard/mouse inputs. If this works, it could be a giant leap in my project.... :)

So that's all for now. Hopefully next post I will have an update on the camera patch.

Datasets

After much consideration and looking at several different datasets, I have finally decided upon a dataset for the project.

I was thinking about the DJ metaphor quite a little when considering which dataset would be appropriate. DJ'ing involves mixing music, but the underlying factor which must be "correlated" is the rhythm of the music. Rhythm and movement play an important part in the overall DJ setup.

After scouring through the Australian Bureau of Statistics for many hours, I found a particularly interesting dataset that tells the story of the rhythm and movement of people.

The dataset is the "Overseas Arrivals and Departures" within Australia. It is a very complete set of data based on the arrival and departure cards one must fill out when entering or leaving the country.

Some initial thoughts on the use of this dataset:
- considering having the two different "vinyls" being mixed, the arrivals and the departures. (i.e. imagine one vinyl was full of arrival information and one vinyl full of departure information).
- the dataset is very huge - i will have to narrow it down. I was considering using the country specific data - i.e. WHERE people are going to, and WHERE people are coming from.
- Also some thoughts on overlaying this data with data on the general distance from Australia to the countries. This could add more dimension to the data, so you can also see the distances being travelled and by how many people over time.

Visualisation Thoughts
- The user will need some sort of cue as to where within the tracks they are. Normally a DJ will have the visual cue of the record needle, as well as the audio cue of the output music.
- I want the visualisation to be quite minimalistic, perhaps just using lines and the different attributes of a line such as stroke, colour, thickness, direction etc.
- I was thinking what could be interesting would be to map the total amount of people into line thicknesses (such as in Minard's depiction of the fate of Napoleon's army)
- Country locations could also perhaps be used for mapping (though perhaps too literal if it were lines drawn from an imaginary globe...) Maybe if this was abstracted... if australia was the central point of the visualisation, and lines were drawn in the general direction of the country relative to australia?
- Will time be mapped? or will I just let the visualisation change as the user "scratches" through?

Other Thoughts
- How many countries to use? or should I narrow it down to continent totals?
- Should I let the user pick specific countries to display? If so, how would this work?

Research Proposal

The research proposal as submitted on 25th August 2006.

Research Proposal PDF 2.37MB

Disc Jockey Setup

Beatmatching involves synchronising two or more tracks so they have the same tempo and are in phase. The DJ adjusts the speed of one track to meet the other using the pitch slider.

Scratching involves two tasks:
- moving a record back and forth
- moving a crossfader back and forth.

I also just wanted to post some images that show a traditional scratch disc jockey's setup.

A general setup will have two turntables connected via an audio mixer, hooked up to an amplifier. The DJ also sets up headphones so he/she can listen to a track independently of the music being heard by the audience, in order to cue, beat-match and set it up for mixing.

Each turntable will have a different vinyl with a different musical track on it. The vinyl is read according to the needle's position on the vinyl. There is also a pitch slider on the turntable, which is used for beat-matching. The faster the speed the higher the pitch and vice versa.

The audio mixer comprises of a cross-fader (for overlaying the audio from each track). There is also a channel volume fader for each turntable.




A few other things to note: The platter spins at a constant normal speed. This allows for continuous manual modification of speed, including a complete stop.

There is visual feedback when a DJ is mixing. The needle indicated the temporal position within the song. The pitch slider position indicates the platter speed.

There is quick random access to a particular point within a song through needle placement. There is auditory feedback through changes in the musical output when a record is accelerated, slowed or scratched.

Research Plan

Currently I have almost completed my Research Proposal which is due on the 25th August (this Friday). As part of the proposal I have had to come up with a research plan and gantt chart for the remainder of the course schedule. Final Presentation is set for the 23rd of October. Below is an excerpt from the draft Research Proposal, outlining my research plan for this project. The Gantt Chart shows where I am currently with the project.

3.1 Research Plan Outline
The research plan for this project is defined in terms of objectives and sub-objectives as follows:

1. Design and Project Preparation
(a) Research existing software and hardware
Find existing software and hardware that is
suitable for the project’s implementation.
(b) Obtain access to required software and
hardware.
Ensure required software and hardware is
easily accessible at University and Home.
(c) Design interaction component
Define the interactive device and determine
it’s mapping to the navigational component
of the design.

2. Develop Visualisation
(a) Research Visualisation Techniques
Find a suitable technique that will be used in
the project to test the interaction component.
(b) Find appropriate dataset(s)
Determine the dataset(s) to be visualised in
the project.
(c) Develop Visualisation Concept
Determine how the dataset will be
appropriately mapped.

3. Implement Prototype
(a) Develop interaction device(s)
Construct chosen interaction device as
determined in 1(c).
(b) Integrate software and hardware
Interface software with interaction device.
(c) Programming: Visualisation and Navigation
Program the visualisation and interaction as
developed in stages 1 and 2.

4. Performance
(a) Test prototype with real users
Allow a set of users to test the prototype.
(b) Observe, record and reflect on findings from
performance.


5. Documentation
(a) Progress Documentation
Set up progress “Blog” to continuously
document project progress.
(b) Performance Documentation
Document performance results via video,
pictures, observational notes.
(c) Produce Research Report
Compile and compose research report and
presentation for final submission.


Gantt Chart

Disc Jockeying Research

Disc Jockeying – Background Research

What is a Disc Jockey?
A person who selects and plays sound recordings. There are several different types of DJ’s, which depend on a number of factors such as: selected music genre, intended audience, the performance setting, the preferred medium and the development of sound manipulation.

Equipment used:
1. Sound recordings in a preferred medium (Vinyls, CD’s, mp3’s)
2. A minimum of two devices for playback of sound recordings for alternating back and forth for continuous playback
3. A sound system for amplification of the recordings
Other: Mixer (to mix the sound of the playback devices), a microphone, headphones, samplers, drum machines, effects processors, computerised performance systems.

Techniques:
Cueing
Equalisation
Audio mixing
Turntable DJ specific
Slip cueing, phrasing, cutting, beat juggling, scratching, beatmatching, needle drops phrase shifting + more for transitions and overdubs.

SCRATCHING
- A technique originated by Grand Wizard Theodore, an early hip-hop DJ from New York.
- Almost all scratches are produced by moving a vinyl record back and forth with your hand while it is playing on a turntable.
- Ideally scratching does not damage the record because the needle stays within the groove and does not move horizontally across the record’s surface.
- Basic equipment: two turntables, a mixer with a cross fader.
- Crossfader is used in conjunction with the scratching hand to cut in and out of the scratched record.

Sounds and Techniques
Baby Scratch – Simplest scratch form, basis for all other scratch forms. Performed with the scratching hand only (no crossfader used). The scratching hand slowly moves the record back and forth (moving slowly important, otherwise it becomes a scribble scratch).

Release – sometimes called “cutting” or “sampling”, a simple scratch made by releasing the record from the beginning of a particular sound. Fader open at the beginning of the release, then closed while the record is pulled back to the beginning of the sound.

Tear Scratch – performed without the crossfader – simple forward-back-back or forward-forward-back motion, breaking the sound into triplets,

Scribble Scratch – performed without the crossfader – tensing the forearm muscle of the scratch handle and rapidly jiggling the record back and forth in minute movements.

Chirp Scratch – Involves fading sound in and out with the crossfader hand while the scratching hand performs a baby scratch

Transform Scratch – Starts with crossfader closed, involves moving the record with the scratching hand while periodically “tapping” the crossfader open and immediately closing it again.

Flare Scratch – Begins with crossfader open, involves moving the record while closing the fader one or more times to cut the sound out

Crab Scratch – moving the record while quickly tapping the crossfader open with each finger of the cross fader hand. – for fast flare scratches.

Orbit Scratch – any scratch (commonly the flare) done forward and immediately backwards along the record’s surface.

Tweak Scratch – performed with the turntable motor off. Record platter set in motion manually.

First Research Notes

This was from about 2 weeks ago...

Deco3003 research notes:

vvvv: multipurpose toolkit for real time video synthesis.
Designed to facilitate the handling of large media environments with physical interfaces, real time motion graphics, audio + video and can interact with many users simultaneously.

Can receive input and generate output to external devices.

Can handle multiple objects with “spreads” – an abstraction referring to the act of distributing different values across a set of objects. Generators make it easy to program complex behaviours.

Real time video analysis – nodes are implemented via the “freeframe plugin” can be developed independently of vvvv. Currently existing nodes: Contour tracking, Color Tracking, Motion Detection.

Audio capabilities – uses Microsoft’s DirectShow technology – suitable for the playback and simple mixing of sound files. Input devices with WDM will work with vvvv. Has the ability to analyse incoming sound – volume, frequency analysis.

Interface with wide variety of external devices and protocols.

EXAMPLES

Open Visuals – Cologne
Project displayed at museums in Cologne with the objective of allowing everyone to change the parameters of a vj tool with a behringer BCR2000 midi interface.

Carefullcarelessness
Video to surround sound by nt – soundtracks analysed… camera motion result of real time interaction with vvvv via a 6dof tracker

Wibutee va Watz
Live visuals for an electronic jazz group using sound analysis to influence abstract graphic systems

Electrochoc, sanch 2004 – sound responsive live visuals for electroparty

Videomixer- catweasel 2005

Dual Fotonic Bloc, Meso, 2002

ArKaos Visual Technologies – COMMERCIAL

Gephex – open source
Base visuals from video files or cameras, can be modified by filters and mixers. Parameters can be controlled by signal generators or input devices such as joysticks, sound cards, midi devices. MAC OSX compatible.

Arduino

Carnivore PE

So far...

After meeting with my supervisor, I have sorted out the specifics of my research project. This semester I will be designing a small prototype that uses the Real Time Video Synthesis program "VVVV" (vvvv.org), in conjunction with Arduino (arduino.cc) and experimentation with the idea of "data-scratching".

I will be playing around with the concept of "DJ scratching", exchanging the medium of sound with real datasets. The idea is to allow the user to "scratch" through a dataset as though it were a music track. As with video jockeying displays real-time video derived from the music in the form of a display or projection, the project will be similarly derived, but from the datasets being explored. From this interface the user will be able to set down different "tracks" (datasets), overlay them, "scratch" through them, in an attempt to find correlations, outliers, clusters, patterns, and any other items of interest.

The Arduino program will be used to interface a motion sensor (which will detect X,Y position, acceleration, etc). It is intended for this motion sensor to be the user input device that allows the user to "scratch" the datasets.

Currently I am further exploring the VVVV program, to get used to the interface and discover what is possible, and also looking into how it connects to the Arduino program. I am also conducting background research into the art of "DJing" and particularly "scratching", so that I can generate an appropriate metaphor for the project.

Further I will be starting to compile my Research Proposal, which is due on the 25th August.