Dynamic Data Visualisation Object.

Cinema’s Voice Vectors is an object that visualizes the audio-visual synthesis which happens in a viewer’s perception when he is watching a movie in a movie theatre.

Glowing glass rods peek through a screen everywhere an actor’s mouth would be at any time in a two-minute excerpt of Orson Welles’ “Citizen Kane”. The length of the glass rods correlates to the actor’s distance to the camera respectively the screen.

While the object is running, the movie scene can not be seen on the canvas anymore, but the soundtrack of the scene can be heard. When an actor speaks, the dedicated glass rod lights up, indicating where from on the screen the actually heard voice comes, or, would come if we saw the movie in a theatre. If an actor moves or the camera moves relatively to the actors, the voice jumps across several rods, following the actor’s position on screen.

Cinema’s Voice Vectors was developed in the course of the semester’s topic “Beauty of Data” at the UDK Berlin, Digital Class of Experimental Media Design. The goal was to create a dynamic data sculpture that visualizes a specific data set in a specific way. Having a personal background in the range of sound and cinema, I searched for something related to that. After thinking about how data could be displayed through sound for quite some time, I finally arrived at the idea to better display data that is inside sound.

In cinema, the actor’s voices is almost always reproduced by a speaker in the center of the big screen, while the actor’s faces can be seen distributed all over it. The allocation which voice belongs to where or whom happens in the human audiovisual reception. Because we are used to it, the brain puts the acoustic sensation (voice) to the place where the visual sensation (mouth) is seen - it “drags” the voice out of the center to where it sees the actor’s mouth. The perception so synthesizes one coherent perception out of two incoherent sensations. To gain my data, I analyzed a 2-minute excerpt of Orson Welles’ “Citizen Kane” with respect to where and when the actors are speaking. The result was a sequence of data that described at which time where on the canvas somebody was speaking, plus additionally how far away from the camera that somebody was standing while speaking. This data was then transformed into a xml-table that the sculpture can access.

The sculpture has the abstracted form of a movie theatre, with an upright canvas and a tilted plane where the viewers would be sitting. From that plane, the bundle of glass rods reaches towards the canvas and peeks through it. When the sculpture is running, the soundtrack of the scene can be heard and the glass rods light up in sequence. The canvas is made from translucent acrylic glass, which allows to see through it and observe the glass rods after they have penetrated the canvas, but still partly reflects the rods back at the viewer.

The sculpture is driven by a computer that fits inside it’s belly. On that computer, a processing sketch is running through the xml-table containing the data. Corresponding to that data, the sketch via an arduino board turns on and off the light emitting diodes that are at the ends of the glass rods, which then light up the glass rods.

Wood / Acrylic Glass / Glass rods / LEDs / Electronics / Computer / Processing Software / Amplifier / Speaker. 2 minute loop. 120 cm x 60 cm x 60 cm (L x D x H).