MAT 256, Heawon Kang

3D Parameter Interaction / Visualization

MAT 256 Visual Design Through Algorithms: Explorations in Visual Perception
Winter 2005

Kang, Heawon
Media Arts and Technology
heawon@umail.ucsb.edu

Project Responsibility

1) Interaction One of aims of this class was to make an interactive installation visual art, including camera motion tracking in 3D space or employing other device to track the motion of audience in 3D space. So, we purchased a device that is sensing the angles of the rotation about x, y, z axis.

2) Mapping 2D into 3D I have thought of mapping a 2-dimensional image into a 3-dimensional image. I found out a Jitter patch that is built in as a sample patch. This patch maps a video input in 2D into 3D. The 3D world generated in this patch can be rotated by the mouse about x, y, z axis. I have come up with an idea that we can control the newly generated 3D world by this device instead of a mouse. We started to study how this patch works.

Project Description

1) OpenGL

To create a world in 3D, OpenGL is in need. OpenGL, or Open Graphics Library is a 3D graphics language developed by Silicon Graphics. Before OpenGL, developers had to write unique 3D graphics code for each graphics hardware and operating system platform. However, the birth of OpenGL allows an identical description language for graphics and special effects across platforms and hardware.

In OpenGL, each command executes a drawing action or a special effect such as texture mapping, transparency, hidden surface removal, antialiasing, fog, and lighting effects. And these things are rendered as a 3D world.

This Jitter patch renders a nurb and creates a 3D world. A nurb has control points. By connecting control points, the shape of the nurb is decided. In pictures above, blue points are control points. The right picture is the rotation version of the left picture. You can see each point has a height, which makes the nurb come alive in 3D world. If the control point is high, it looks extruding. If the control point is low or a minus value, it looks going backwards.

2) How to Map 2D into 3D

How can we create a 3D world from a 2D image? There may be many ways to do it. One of the most common is to take luminance from an image. Then, in this patch, the luminance of the image is reflected into the height of the control point of the nurb. If the luminance of the pixel is high (bright), the height of the control point of the nurb for that pixel becomes high. On the opposite, the luminance of the pixel is low, the height of the control point of the nurb for that pixel becomes low.

Luminance X 1.0

Luminance X 2.0

Luminance X 1.0

Also, the number of the control point can be controlled. As the number of the control point reduces, the size of the rectangle generated by connecting each control point around it is getting bigger.

3) Installation

For the installation, we programmed to capture a video input and render 3D polygons in OpenGL. The resolution in Jitter is limited to 320*240 for the real time display. We can increase the resolution, but it is limited by the hardware. So, Will and Zach developed in order to increase the flexibility and efficiency of the system as well as the resolution. For the real installation, we takes the average of RGB values instead of luminance, and creates polygons instead of nurbs. The device controls the size of polygons and the height of polygons. These parts are explained in Will and Zach’s webpage.

Additional Works

Eunsu and I developed a patch to detect people from a background. This patch becomes a part of “foggy mirror”, which is modified to capture people in the desired condition of a light by Wes. It is very simple but exact for the performance. Take the background ahead of the installation and subtract it from a live input. By chromakeying, the human part turns white and the background turns black. The patch is attached to Eunsu’s webpage.

Also, I developed a logical patch to get the pitch, yaw, and roll intuitively to replace the mouse movement.

Reference

Mapping 2D into 3D world
http://xenia.media.mit.edu/~sbeck

Algorithmic Assemblage by Gary Thomas
http://www.mat.ucsb.edu/~g.thomas/projects/mat594e/index.html
http://www.mat.ucsb.edu/%7Eg.thomas/courses/594e/proposal-orig.html

OpenGL tutorial
http://www.eecs.tulane.edu/www/Terry/OpenGL/Introduction.html

3D Parameter Interaction / Visualization MAT 256 Visual Design Through Algorithms: Explorations in Visual Perception Winter 2005 Kang, Heawon Media Arts and Technology heawon@umail.ucsb.edu
Project Responsibility 1) Interaction One of aims of this class was to make an interactive installation visual art, including camera motion tracking in 3D space or employing other device to track the motion of audience in 3D space. So, we purchased a device that is sensing the angles of the rotation about x, y, z axis. 2) Mapping 2D into 3D I have thought of mapping a 2-dimensional image into a 3-dimensional image. I found out a Jitter patch that is built in as a sample patch. This patch maps a video input in 2D into 3D. The 3D world generated in this patch can be rotated by the mouse about x, y, z axis. I have come up with an idea that we can control the newly generated 3D world by this device instead of a mouse. We started to study how this patch works.
Project Description 1) OpenGL To create a world in 3D, OpenGL is in need. OpenGL, or Open Graphics Library is a 3D graphics language developed by Silicon Graphics. Before OpenGL, developers had to write unique 3D graphics code for each graphics hardware and operating system platform. However, the birth of OpenGL allows an identical description language for graphics and special effects across platforms and hardware. In OpenGL, each command executes a drawing action or a special effect such as texture mapping, transparency, hidden surface removal, antialiasing, fog, and lighting effects. And these things are rendered as a 3D world.

This Jitter patch renders a nurb and creates a 3D world. A nurb has control points. By connecting control points, the shape of the nurb is decided. In pictures above, blue points are control points. The right picture is the rotation version of the left picture. You can see each point has a height, which makes the nurb come alive in 3D world. If the control point is high, it looks extruding. If the control point is low or a minus value, it looks going backwards.
2) How to Map 2D into 3D
How can we create a 3D world from a 2D image? There may be many ways to do it. One of the most common is to take luminance from an image. Then, in this patch, the luminance of the image is reflected into the height of the control point of the nurb. If the luminance of the pixel is high (bright), the height of the control point of the nurb for that pixel becomes high. On the opposite, the luminance of the pixel is low, the height of the control point of the nurb for that pixel becomes low.
Luminance X 1.0
Luminance X 2.0
Luminance X 1.0
Also, the number of the control point can be controlled. As the number of the control point reduces, the size of the rectangle generated by connecting each control point around it is getting bigger.



3) Installation For the installation, we programmed to capture a video input and render 3D polygons in OpenGL. The resolution in Jitter is limited to 320240 for the real time display. We can increase the resolution, but it is limited by the hardware. So, Will and Zach developed in order to increase the flexibility and efficiency of the system as well as the resolution. For the real installation, we takes the average of RGB values instead of luminance, and creates polygons instead of nurbs. The device controls the size of polygons and the height of polygons. These parts are explained in Will and Zach’s webpage. Additional Works* Eunsu and I developed a patch to detect people from a background. This patch becomes a part of “foggy mirror”, which is modified to capture people in the desired condition of a light by Wes. It is very simple but exact for the performance. Take the background ahead of the installation and subtract it from a live input. By chromakeying, the human part turns white and the background turns black. The patch is attached to Eunsu’s webpage. Also, I developed a logical patch to get the pitch, yaw, and roll intuitively to replace the mouse movement. Reference Mapping 2D into 3D world http://xenia.media.mit.edu/~sbeck Algorithmic Assemblage by Gary Thomas http://www.mat.ucsb.edu/~g.thomas/projects/mat594e/index.html http://www.mat.ucsb.edu/%7Eg.thomas/courses/594e/proposal-orig.html OpenGL tutorial http://www.eecs.tulane.edu/www/Terry/OpenGL/Introduction.html