.

Friday, October 27, 2006

DUE DUE DUE

So all the due dates are creeping up on me. The research report is now due a week from now (Friday Nov 3rd)... only TWO days after the presentation. Looking back on the course outline though, the presentation is only worth 10%, and the report is 80%. The presentation for this project will be during the next Wednesday Seminar (yikes), on November 1st.

As for the project, here are a few update images of the aesthetics. I FINALLY got rid of the black (ugly!, i don't know what i was thinking), and I've also added interactive labelling.





The scratch device is all complete, though I am considering adding one more functionality - and that is a pause button, so you can stop the movement when you find something interesting, so that you can study it/print it/screenshot it. I've bought a switch for it, but still have to work out the inner workings. I am also considering creating a slider sticker that shows where the placement of the slider should be for the various d/sets.

More on this soon.

Thursday, October 19, 2006

Busy Busy Busy

It has been a long time since the last update. I must admit, I am not a "blogging" type person, and so it has slipped my mind a little. But I promise i will try to keep the last stages constantly tracked, and I will do an update post right now that documents what I have accomplished thus far.

DATASETS
Rather than the datasets I initially proposed, I decided that it would make more sense to use a set of unrelated datasets, and then try to create interesting correlations through experimenting through scratching. Thus I found a set of data from ABS including overseas arrivals and departures, export data, and crime data over a ten year period from July 1996 to June 2006.

VVVV PROGRESS
Well, it has been a very steep learning curve with this program. Apart from the documentation and examples presented on the site and within the program, there is very little help with this program. Also just to note, as an open source project, there are still a few modules that are a bit buggy. Also, being a graphical programming language, it was a little difficult to translate "pseudocode" into modules that work. The easy things in programming such as if, then, else, becomes a little more tricky to work out in VVVV.

There was several tossing about of which type of renderer to use. Initially, I started out using the Direct-X Renderer (EX9). At this time, I only knew enough VVVV to draw straight line graphs as you can see below. Though it was a small idea of what i wanted to play around with, I decided that the lines were rather ugly, and discontinuous with the sharp peaks. Also, I need to think about "zooming in" to the data, and displaying say, 3 data points on the screen at once, rather than almost the whole dataset.



The next stage, i tried to figure out how to draw bezier lines within VVVV using the datasets. At this point I only had enough knowledge to draw Bezier lines using the GDI renderer. The issue with this was that the GDI Renderer renders on the CPU rather than the GPU and this would cause the system to slow down immensely and "jitter". It would also render the whole line rather than only what was visible on the screen. Thus I needed to figure out a way to render only the section of the dataset visible on the screen, whilst allowing it to update the line as we scratch through.



The next prototype saw this problem solved, as I managed to render only sections of the line visible on the screen. However I was not entirely happy with the results and with using the GDI renderer. It had alot of limitations such as not being able to use the capabilities of textures and alpha channels. I knew if I only could figure out bezier lines in Direct X, that many "tricks" could then be employed to make the aesthetics much more pleasing.



Finally after much stress and searching, I found an example on the SanchTV Website that created Bezier Lines in Direct X. Sanch is a VVVV programmer that posts his VVVV work online for other to learn off. Thankfully, I was able to extract the information I needed from his Bezier Line Editor 2D. The results are below, however there were still several bugs that needed addressing. The line drawn would have a giant gap in the centre, which though I tried to cover up by rendering the line as a dashed line, the aesthetics suffered because of it. Further, I could only get text working in the GDI renderer, with the same issues as previously, which caused the system to once again slow down when text was added.



So this brings us up to present. The current prototype fixed a couple of the bugs. I used rope instead of the line module, and this eliminated the gap in the line, and it also rendered alot smoother, meaning that increase in line size does not show so obviously the gaps between the lines that form the bezier curves. Further, I got EX9 text working, which means that everything is rendering fast again. I further increased the speed by ensuring only the text within the screen was rendered.



The aesthetics are getting there! It is improving, though I am not entirely happy with the representation of text as yet, but I am working on it!

INTERACTION PROGRESS
I realised that the microcontroller was not working the way that I needed it to. It would only measure according to tilt, rather than an x and y position relative to a starting point, which is what i imagined it would do.

Thus I began brainstorming new methods for this "scratch" interaction using Arduino. After reading the Physical computing book by Dan O'Sullivan, I realised I could use a potentiometer to read rotation values.

With the help of my boyfriend Graham, we devised a method for attaching the potentiometer to a turntable, such that when the turntable was rotated, the potentiometer would turn, sending out a different resistance value.

We used a die to cut threads on the shaft of the pot, and then used mudguard washers and locknuts to attach the pot to the turntable. We then superglued the base of the potentiometer to the base of the turntable, so only the shaft turned when rotated.

We also added a slider on the turntable by jigsawing a slot for it to sit and mounting it. The slider will be used to select different datasets to scratch. (The slider is not shown in the image).



There are only a few issues with the setup. It is a little bit fragile at the moment, and the turning mechanism is resting entirely on the potentiometer. Thus we are considering adding rollers to take the weight off the pot.



Also the pot is not a continous rotation pot. Thus it has a maximum point and a minimum point. People attempting to push past this point may break it. For the presentation, I will be the only one demonstrating and this will not be an issue. However for the final year exhibition, this may be a problem.

Friday, September 01, 2006

Arduino Microcontroller

Just thought I would post a picture up of the microcontroller I am using. At the moment it spits out an x val, x velocity and x acceleration, y val, y velocity and y acceleration.

I still have to decide how to map these in the program. Hopefully I will be simply limiting it to two movements: foward/backwards movement and side to side. Though talking to some people, some feel the "forward/backward" makes more sense as scratching, other feel side to side feels more like scratching.



Many thanks to Justin Clayden for constructing and coding the microcontroller in Arduino for me! :)

Wednesday, August 30, 2006

vvvv progress

I have been very productive today and organized my datasets into text-files to be read into vvvv. I decided that for now I will be working with two specific datasets: Arrivals and Departures by Country, but narrowed down to regions such as Oceania, the Americas, etc.

I have written a patch which reads these text files into vvvv and separates the data into slices.


So one challenge down, a zillion to go!

The next thing I decided I really needed to spend time on was to play around with the system to see what sorts of visualisations I could create with it, and try and broaden my knowledge of patching in general. I experimented with a few things - had a fit trying to draw bezier curves (evil). I particularly liked the rope patch for DX9.





Another VERY interesting thing i found was the "Camera (SoftImage)" patch, which essentially links keyboard input and mouse clicks to control zoom, pan, rotation etc of the camera! What is great about this is that all patches can be edited... thus I am considering using this existing patch, and editing it to read off the arduino inputs rather than the keyboard/mouse inputs. If this works, it could be a giant leap in my project.... :)

So that's all for now. Hopefully next post I will have an update on the camera patch.

Datasets

After much consideration and looking at several different datasets, I have finally decided upon a dataset for the project.

I was thinking about the DJ metaphor quite a little when considering which dataset would be appropriate. DJ'ing involves mixing music, but the underlying factor which must be "correlated" is the rhythm of the music. Rhythm and movement play an important part in the overall DJ setup.

After scouring through the Australian Bureau of Statistics for many hours, I found a particularly interesting dataset that tells the story of the rhythm and movement of people.

The dataset is the "Overseas Arrivals and Departures" within Australia. It is a very complete set of data based on the arrival and departure cards one must fill out when entering or leaving the country.

Some initial thoughts on the use of this dataset:
- considering having the two different "vinyls" being mixed, the arrivals and the departures. (i.e. imagine one vinyl was full of arrival information and one vinyl full of departure information).
- the dataset is very huge - i will have to narrow it down. I was considering using the country specific data - i.e. WHERE people are going to, and WHERE people are coming from.
- Also some thoughts on overlaying this data with data on the general distance from Australia to the countries. This could add more dimension to the data, so you can also see the distances being travelled and by how many people over time.

Visualisation Thoughts

- The user will need some sort of cue as to where within the tracks they are. Normally a DJ will have the visual cue of the record needle, as well as the audio cue of the output music.
- I want the visualisation to be quite minimalistic, perhaps just using lines and the different attributes of a line such as stroke, colour, thickness, direction etc.
- I was thinking what could be interesting would be to map the total amount of people into line thicknesses (such as in Minard's depiction of the fate of Napoleon's army)
- Country locations could also perhaps be used for mapping (though perhaps too literal if it were lines drawn from an imaginary globe...) Maybe if this was abstracted... if australia was the central point of the visualisation, and lines were drawn in the general direction of the country relative to australia?
- Will time be mapped? or will I just let the visualisation change as the user "scratches" through?

Other Thoughts
- How many countries to use? or should I narrow it down to continent totals?
- Should I let the user pick specific countries to display? If so, how would this work?

Research Proposal

The research proposal as submitted on 25th August 2006.

Research Proposal PDF 2.37MB

Tuesday, August 22, 2006

Disc Jockey Setup

Beatmatching involves synchronising two or more tracks so they have the same tempo and are in phase. The DJ adjusts the speed of one track to meet the other using the pitch slider.

Scratching involves two tasks:
- moving a record back and forth
- moving a crossfader back and forth.

I also just wanted to post some images that show a traditional scratch disc jockey's setup.

A general setup will have two turntables connected via an audio mixer, hooked up to an amplifier. The DJ also sets up headphones so he/she can listen to a track independently of the music being heard by the audience, in order to cue, beat-match and set it up for mixing.

Each turntable will have a different vinyl with a different musical track on it. The vinyl is read according to the needle's position on the vinyl. There is also a pitch slider on the turntable, which is used for beat-matching. The faster the speed the higher the pitch and vice versa.

The audio mixer comprises of a cross-fader (for overlaying the audio from each track). There is also a channel volume fader for each turntable.




A few other things to note: The platter spins at a constant normal speed. This allows for continuous manual modification of speed, including a complete stop.

There is visual feedback when a DJ is mixing. The needle indicated the temporal position within the song. The pitch slider position indicates the platter speed.

There is quick random access to a particular point within a song through needle placement. There is auditory feedback through changes in the musical output when a record is accelerated, slowed or scratched.

Research Plan

Currently I have almost completed my Research Proposal which is due on the 25th August (this Friday). As part of the proposal I have had to come up with a research plan and gantt chart for the remainder of the course schedule. Final Presentation is set for the 23rd of October. Below is an excerpt from the draft Research Proposal, outlining my research plan for this project. The Gantt Chart shows where I am currently with the project.

3.1 Research Plan Outline
The research plan for this project is defined in terms of objectives and sub-objectives as follows:

1. Design and Project Preparation
(a) Research existing software and hardware
Find existing software and hardware that is
suitable for the project’s implementation.
(b) Obtain access to required software and
hardware.

Ensure required software and hardware is
easily accessible at University and Home.
(c) Design interaction component
Define the interactive device and determine
it’s mapping to the navigational component
of the design.

2. Develop Visualisation
(a) Research Visualisation Techniques
Find a suitable technique that will be used in
the project to test the interaction component.
(b) Find appropriate dataset(s)
Determine the dataset(s) to be visualised in
the project.
(c) Develop Visualisation Concept
Determine how the dataset will be
appropriately mapped.

3. Implement Prototype
(a) Develop interaction device(s)
Construct chosen interaction device as
determined in 1(c).
(b) Integrate software and hardware
Interface software with interaction device.
(c) Programming: Visualisation and Navigation
Program the visualisation and interaction as
developed in stages 1 and 2.

4. Performance
(a) Test prototype with real users
Allow a set of users to test the prototype.
(b) Observe, record and reflect on findings from
performance.



5. Documentation
(a) Progress Documentation
Set up progress “Blog” to continuously
document project progress.
(b) Performance Documentation
Document performance results via video,
pictures, observational notes.
(c) Produce Research Report
Compile and compose research report and
presentation for final submission.


Gantt Chart

Monday, August 14, 2006

Disc Jockeying Research

Disc Jockeying – Background Research

What is a Disc Jockey?
A person who selects and plays sound recordings. There are several different types of DJ’s, which depend on a number of factors such as: selected music genre, intended audience, the performance setting, the preferred medium and the development of sound manipulation.

Equipment used:
1. Sound recordings in a preferred medium (Vinyls, CD’s, mp3’s)
2. A minimum of two devices for playback of sound recordings for alternating back and forth for continuous playback
3. A sound system for amplification of the recordings
Other: Mixer (to mix the sound of the playback devices), a microphone, headphones, samplers, drum machines, effects processors, computerised performance systems.

Techniques:
Cueing
Equalisation
Audio mixing
Turntable DJ specific
Slip cueing, phrasing, cutting, beat juggling, scratching, beatmatching, needle drops phrase shifting + more for transitions and overdubs.

SCRATCHING
- A technique originated by Grand Wizard Theodore, an early hip-hop DJ from New York.
- Almost all scratches are produced by moving a vinyl record back and forth with your hand while it is playing on a turntable.
- Ideally scratching does not damage the record because the needle stays within the groove and does not move horizontally across the record’s surface.
- Basic equipment: two turntables, a mixer with a cross fader.
- Crossfader is used in conjunction with the scratching hand to cut in and out of the scratched record.

Sounds and Techniques
Baby Scratch – Simplest scratch form, basis for all other scratch forms. Performed with the scratching hand only (no crossfader used). The scratching hand slowly moves the record back and forth (moving slowly important, otherwise it becomes a scribble scratch).

Release – sometimes called “cutting” or “sampling”, a simple scratch made by releasing the record from the beginning of a particular sound. Fader open at the beginning of the release, then closed while the record is pulled back to the beginning of the sound.

Tear Scratch – performed without the crossfader – simple forward-back-back or forward-forward-back motion, breaking the sound into triplets,

Scribble Scratch – performed without the crossfader – tensing the forearm muscle of the scratch handle and rapidly jiggling the record back and forth in minute movements.

Chirp Scratch – Involves fading sound in and out with the crossfader hand while the scratching hand performs a baby scratch

Transform Scratch – Starts with crossfader closed, involves moving the record with the scratching hand while periodically “tapping” the crossfader open and immediately closing it again.

Flare Scratch – Begins with crossfader open, involves moving the record while closing the fader one or more times to cut the sound out

Crab Scratch – moving the record while quickly tapping the crossfader open with each finger of the cross fader hand. – for fast flare scratches.

Orbit Scratch – any scratch (commonly the flare) done forward and immediately backwards along the record’s surface.

Tweak Scratch – performed with the turntable motor off. Record platter set in motion manually.

First Research Notes

This was from about 2 weeks ago...

Deco3003 research notes:

vvvv: multipurpose toolkit for real time video synthesis.
Designed to facilitate the handling of large media environments with physical interfaces, real time motion graphics, audio + video and can interact with many users simultaneously.

Can receive input and generate output to external devices.

Can handle multiple objects with “spreads” – an abstraction referring to the act of distributing different values across a set of objects. Generators make it easy to program complex behaviours.

Real time video analysis – nodes are implemented via the “freeframe plugin” can be developed independently of vvvv. Currently existing nodes: Contour tracking, Color Tracking, Motion Detection.

Audio capabilities – uses Microsoft’s DirectShow technology – suitable for the playback and simple mixing of sound files. Input devices with WDM will work with vvvv. Has the ability to analyse incoming sound – volume, frequency analysis.

Interface with wide variety of external devices and protocols.

EXAMPLES

Open Visuals – Cologne
Project displayed at museums in Cologne with the objective of allowing everyone to change the parameters of a vj tool with a behringer BCR2000 midi interface.

Carefullcarelessness
Video to surround sound by nt – soundtracks analysed… camera motion result of real time interaction with vvvv via a 6dof tracker

Wibutee va Watz
Live visuals for an electronic jazz group using sound analysis to influence abstract graphic systems

Electrochoc, sanch 2004 – sound responsive live visuals for electroparty

Videomixer- catweasel 2005

Dual Fotonic Bloc, Meso, 2002

ArKaos Visual Technologies – COMMERCIAL

Gephex – open source
Base visuals from video files or cameras, can be modified by filters and mixers. Parameters can be controlled by signal generators or input devices such as joysticks, sound cards, midi devices. MAC OSX compatible.

Arduino

Carnivore PE

So far...

After meeting with my supervisor, I have sorted out the specifics of my research project. This semester I will be designing a small prototype that uses the Real Time Video Synthesis program "VVVV" (vvvv.org), in conjunction with Arduino (arduino.cc) and experimentation with the idea of "data-scratching".

I will be playing around with the concept of "DJ scratching", exchanging the medium of sound with real datasets. The idea is to allow the user to "scratch" through a dataset as though it were a music track. As with video jockeying displays real-time video derived from the music in the form of a display or projection, the project will be similarly derived, but from the datasets being explored. From this interface the user will be able to set down different "tracks" (datasets), overlay them, "scratch" through them, in an attempt to find correlations, outliers, clusters, patterns, and any other items of interest.

The Arduino program will be used to interface a motion sensor (which will detect X,Y position, acceleration, etc). It is intended for this motion sensor to be the user input device that allows the user to "scratch" the datasets.

Currently I am further exploring the VVVV program, to get used to the interface and discover what is possible, and also looking into how it connects to the Arduino program. I am also conducting background research into the art of "DJing" and particularly "scratching", so that I can generate an appropriate metaphor for the project.

Further I will be starting to compile my Research Proposal, which is due on the 25th August.

Friday, July 28, 2006

DECO3003 Design Computing Research Opportunity

This elective gives me the chance to gain experience in and explore the world of research, by choosing a topic I am interested in, and for one semester (in conjunction with an experienced researcher/supervisor) work on a research project of my own choosing.

The field of study chosen is Information Visualization.

More specifically, I will be exploring existing Video-Jockeying software and its potential in data visualization applications.

Supervisor: Andrew Vande Moere