Embodiment & Architecture I: Computer Vision
Since my last exhibition in June, myself, Colin and Monika have been working on a new performance piece. The project is currently, and somewhat scientifically, entitled "Embodiment and Architecture"— I figure when we flesh out the conceptual and physical side of the oeuvre the packaging will fall into place of its own accord. For a deeper aesthetic analogy of the project please visit the concepts section of this website, as this blog section is reserved for a more open, friendly, conversational discussion of my work.
The main difference between this project and any previous motion-tracking projects we have conducted is that when tracking a dancer you want to literally “cut out”—for want of a better term—the shape (contour) of her body. Up till now we have only ever tracked a single, central point of light (known as the centroid), but if we could generate a bounding box around the dancer this would allow for much more interesting graphical generation.
We attacked the project from 2 different angles:
- We first considered modifying TouchLib software — a very slick solution for tracking a single point of light. However this proved too difficult as the TouchLib project literally has reams of code. Note: one very handy feature of TouchLib is that you can callibrate the camera to see exactly what you tell it to and disregard the rest of the frame (this is what we used in the video below).
- We then turned to the more flexible solution of OpenFrameworks, which we have used before on several projects. OF is an open-source C++ project aimed promoting C++ development in the arts. You must build your application from the ground up and it comes packaged with an OpenCV (Open Computer Vision) plugin ready to go. So you just need to modify it to suit your needs.
The dificulty we found with OpenFrameworks was that it’s complicated for generating graphics. Although it is very fast and powerful, C++ code is messy to work with, the build can get very confusing, and the error messages (unless you are really used to compiling your own programs) are abstruse to say the least. I have no doubt that if you are a C++ wisard it would be better to build the whole project with OF, but I’m not, I’m an artist. Note: For anybody embarking on an OF project for the first time, the OFWiki is a great jumping off point and will help you with the complex teething problems associated with building your own C++ projects.
For this reason we decided to build the graphics in Processing; we're used to processing as we've used it several times before. This decision also served in keeping the code modular for different aspects of the project, making it easier to debug problems as they arose.
Openframeworks with OpenCV allows you to track with pinpoint accuracy the location of a given performer. We used OSC messages to send the cordinates of the dancer’s location (i.e. the bounding box) to the processing patch whereby we could generate in realtime, procedural graphics in response to her movement.
Monika Training with the virtual set from Neill O'Dwyer on Vimeo.