Shift
From Medialab Prado
Contents |
Descripción del proyecto / Descripction of the project
Shift, a follow-up to Body/traces, is a multimedia performance piece that pursues the theme of the body as object and how identity is manifested through it. In particular, we will be focusing on the space between physical and virtual existence and questioning the notion of “enduring identity” in light of memory and our experience of change. This project will be co-directed by Sophie and Lisa, collaborating with the digital artist Kyle McDonald, to create a performance piece involving real-time, interactive scanning for capturing imagery of the moving body. The process involves bathing the entire scene in light from a projector in a technique called "structured light scanning.” The high frame rate and transparent scanning process will generate 3D models alongside dancers to create a shared space of movement, images and sound in real-time. The 3D representations of the body that we will be generating emphasize the shape of the gestures rather than an idealized abstract form. The performance content will comprise a visual narrative based on a series of photographs of a woman in a dress. Successive images show the same woman, but each time in a different place, in a kind of geographical fragmentation which runs parallel to the framing of moments in her life in memory. We envision the final piece as a performance installation in which life-size projected images will occupy the space at the same time as the dancers themselves, blurring the distinction between physical and represented presence in movement.
Documentación (gráficos, fotos y vídeos) / Documentation (graphics, pictures and videos)
Photos from some preparation work before HelloWorld!, and from some scans during HelloWorld!, can be found on Flickr [1]
Videos of the preparation work are presented at the beginning of this short video on vimeo [2]
Technical goals in preparation for workshop II
Chris Sugrue made some significant progress towards identifying essential requirements for distributing a useful DIY 3D Scanning tool. From our discussions, we've aimed at releasing an app for OSX and Windows with the following features before workshop II:
- Realtime 3d preview
- This is implemented best in Chris' application cap_realtime_v3_cb, produced in conjunction with Arturo Castro. There isn't an obvious cross-platform solution to this right now as there isn't a single solution to displaying on a main monitor and projector simultaneously that also leaves vertical sync enabled.
- In-application capture of structured light sequences, saved to movie files
- Good progress on this, structuredLightCapture, saving to jpg sequences instead of movie files for now as it is more cross-platform.
- Playback of recorded structured light sequences in movie files
- A single three phase sequence (three images) is decoded by structuredLightDecoder; now working towards moving sequences...
- Controls for phase shift rate vs capture rate, and a phase offset parameter for manually keeping them in sync
- Implemented in structuredLightCapture
- Controls for the frequency of the sine patterns, zskew, zscale, and projector/camera orientation
- Added this to the interface in structuredLightDecoder
- Controls for denoising input (fast median filtering)
- Semi-automatic calibration of projector-camera system brightness curve
- Support for .obj or .ply export of point clouds and triangulated meshes
- Wrote code for this (.obj and .ply as point clouds and meshes, .ply with color, also image depth maps), posted example output to Google code. Future improvement includes using 16-bit .png depth output. Now added to the structuredLightDecoder interface.
- Gray code scanning with the appropriate controls (number of levels, horizontal or vertical, or both)
Most importantly, this will all be accompanied by documentation describing how to use the software, and how to arrange the 3d scanner and camera.
Less essential goals for the future include:
- Camera lens distortion modeling for removing "bulging"
- Automatic camera calibration for determining absolute spatial coordinates in the case of gray codes, or at least automatically deriving zskew/zscale in the case of three phase (a la the black/white markers from DAVID)
- Hacking the PS3Eye to allow trigger input from a VGA or DVI signal
- Some progress here -- no trigger input has been found, but a VSYNC output has been found. This can be used to drive a microcontroller that is generating the video signal in a phase locked loop. This has the added advantage of freeing up the computer for processing instead of display.
Autor/es del proyecto / Author(s) of the project
Kyle McDonald, Sophie Kahn, Lisa Parra y Chris Sugrue
Colaboradores / Collaboratos
Links
The applications mentioned above can temporarily be downloaded from [3] while in early development, and will be moved to [4] after the workshop is over.
