MULTIMEDIA

iPhone 3D Programming : Reflections with Cube Maps

6/16/2011 3:25:15 PM
We rendered an upside-down object to simulate reflection. This was sufficient for reflecting a limited number of objects onto a flat plane, but if you’d like the surface of a 3D object to reflect a richly detailed environment, as shown in Figure 1, a cube map is required. Cube maps are special textures composed from six individual images: one for each of the six axis-aligned directions in 3D space. Cube maps are supported only in OpenGL ES 2.0.
Figure 1. Reflection sample


Cube maps are often visualized using a cross shape that looks like an unfolded box, as shown in Figure 2.

Figure 2. Cube map cross of the Minnehaha Falls (courtesy of Edward Fink)


The cross shape is for the benefit of humans only; OpenGL does expect it when you give it the image data for a cube map. Rather, it requires you to upload each of the six faces individually, like this:

glTexImage2D(GL_TEXTURE_CUBE_MAP_POSITIVE_X, mip, format, 
w, h, 0, format, type, data[0]);
glTexImage2D(GL_TEXTURE_CUBE_MAP_NEGATIVE_X, mip, format,
w, h, 0, format, type, data[1]);
glTexImage2D(GL_TEXTURE_CUBE_MAP_POSITIVE_Y, mip, format,
w, h, 0, format, type, data[2]);
glTexImage2D(GL_TEXTURE_CUBE_MAP_NEGATIVE_Y, mip, format,
w, h, 0, format, type, data[3]);
glTexImage2D(GL_TEXTURE_CUBE_MAP_POSITIVE_Z, mip, format,
w, h, 0, format, type, data[4]);
glTexImage2D(GL_TEXTURE_CUBE_MAP_NEGATIVE_Z, mip, format,
w, h, 0, format, type, data[5]);

Note that, for the first time, we’re using a texture target other than GL_TEXTURE_2D. This can be a bit confusing because the function call name still has the 2D suffix. It helps to think of each face as being 2D, although the texture object itself is not.

The enumerants for the six faces have contiguous values, so it’s more common to upload the faces of a cube map using a loop. For an example of this, see Example 1, which creates and populates a complete mipmapped cube map.

Example 1. CreateCubemap function
{
GLuint textureObject;
glGenTextures(1, &textureObject);
glBindTexture(GL_TEXTURE_CUBE_MAP, textureObject);
for (int f = 0; f < 6; ++f) {
GLenum face = GL_TEXTURE_CUBE_MAP_POSITIVE_X + f;
glTexImage2D(face, 0, format, size, size, 0, format, type, faceData[f]);
}
glTexParameteri(GL_TEXTURE_CUBE_MAP,
GL_TEXTURE_MIN_FILTER,
GL_LINEAR_MIPMAP_LINEAR);
glTexParameteri(GL_TEXTURE_CUBE_MAP, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glGenerateMipmap(GL_TEXTURE_CUBE_MAP);
return textureObject;
}





In Example 1, the passed-in size parameter is the width (or height) of each cube map face. Cube map faces must be square. Additionally, on the iPhone, they must have a size that’s a power-of-two.

Example 2 shows the vertex shader that can be used for cube map reflection.

Example 2. Vertex shader (cube map sample)
attribute vec4 Position;
attribute vec3 Normal;

uniform mat4 Projection;
uniform mat4 Modelview;
uniform mat3 Model;
uniform vec3 EyePosition;

varying vec3 ReflectDir;

void main(void)
{
gl_Position = Projection * Modelview * Position;

// Compute eye direction in object space:
mediump vec3 eyeDir = normalize(Position.xyz - EyePosition);

// Reflect eye direction over normal and transform to world space:
ReflectDir = Model * reflect(eyeDir, Normal);
}

Newly introduced in Example 8-9 is GLSL’s built-in reflect function, which is defined like this:

float reflect(float I, float N)
{
return I - 2.0 * dot(N, I) * N;
}

N is the surface normal; I is the incident vector, which is the vector that strikes the surface at the point of interest (see Figure 3).

Figure 3. The GLSL “reflect” function



Note:

Cube maps can also be used for refraction, which is useful for creating glass or other transparent media. GLSL provides a refract function to help with this.


The fragment shader for our cube mapping example is fairly simple; see Example 3.

Example 3. Fragment shader (cube map sample)
varying mediump vec3 ReflectDir;

uniform samplerCube Sampler;

void main(void)
{
gl_FragColor = textureCube(Sampler, ReflectDir);
}

Newly introduced in Example 3 is a new uniform type called samplerCube. Full-blown desktop OpenGL has many sampler types, but the only two sampler types supported on the iPhone are samplerCube and sampler2D. Remember, when setting a sampler from within your application, set it to the stage index, not the texture handle!

The sampler function in Example 3 is also new: textureCube differs from texture2D in that it takes a vec3 texture coordinate rather than a vec2. You can think of it as a direction vector emanating from the center of a cube. OpenGL finds which of the three components have the largest magnitude and uses that to determine which face to sample from.

A common gotcha with cube maps is incorrect face orientation. I find that the best way to test for this issue is to render a sphere with a simplified version of the vertex shader that does not perform true reflection:

//ReflectDir = Model * reflect(eyeDir, Normal);
ReflectDir = Model * Position.xyz; // Test the face orientation.

Using this technique, you’ll easily notice seams if one of your cube map faces needs to be flipped, as shown on the left in Figure 4. Note that only five faces are visible at a time, so I suggest testing with a negated Position vector as well.

Figure 4. From left to right: incorrect face orientation, corrected faces, corrected faces with reflection


1. Render to Cube Map

Instead of using a presupplied cube map texture, it’s possible to generate a cube map texture in real time from the 3D scene itself. This can be done by rerendering the scene six different times, each time using a different model-view matrix.

GLenum attachment = GL_COLOR_ATTACHMENT0;
GLenum textureTarget = GL_TEXTURE_2D;
GLuint textureHandle = myTextureObject;
GLint mipmapLevel = 0;
glFramebufferTexture2D(GL_FRAMEBUFFER, attachment,
textureTarget, textureHandle, mipmapLevel);

The textureTarget parameter is not limited to GL_TEXTURE_2D; it can be any of the six face enumerants (GL_TEXTURE_CUBE_MAP_POSITIVE_X and so on). See Example 4 for a high-level overview of a render method that draws a 3D scene into a cube map.

Example 4. Rendering to a cube map
glBindFramebuffer(GL_FRAMEBUFFER, fboHandle);
glViewport(0, 0, fboWidth, fboHeight);

for (face = 0; face < 6; face++) {

// Change the FBO attachment to the current face:
GLenum textureTarget = GL_TEXTURE_CUBE_MAP_POSITIVE_X + face;
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
textureTarget, textureHandle, 0);

// Set the model-view matrix to point toward the current face:
...

// Render the scene:
...
}


Warning:

Rendering to a cube map texture is supported only in iPhone OS 3.1 and newer.

Other  
  •  Silverlight Recipes : Networking and Web Service Integration - Accessing Resources over HTTP
  •  Silverlight Recipes : Networking and Web Service Integration - Using JSON Serialization over HTTP
  •  Microsoft XNA Game Studio 3.0 : Displaying Images - Using Resources in a Game (part 4) - Filling the Screen
  •  Microsoft XNA Game Studio 3.0 : Displaying Images - Using Resources in a Game (part 3) - Sprite Drawing with SpriteBatch
  •  Microsoft XNA Game Studio 3.0 : Displaying Images - Using Resources in a Game (part 2) - Positioning Your Game Sprite on the Screen
  •  Microsoft XNA Game Studio 3.0 : Displaying Images - Using Resources in a Game (part 1) - Loading XNA Textures
  •  iPhone 3D Programming : Holodeck Sample (part 5) - Overlaying with a Live Camera Image
  •  iPhone 3D Programming : Holodeck Sample (part 4) - Replacing Buttons with Orientation Sensors
  •  iPhone 3D Programming : Holodeck Sample (part 3) - Handling the Heads-Up Display
  •  iPhone 3D Programming : Holodeck Sample (part 2) - Rendering the Dome, Clouds, and Text
  •  iPhone 3D Programming : Holodeck Sample (part 1) - Application Skeleton
  •  Building LOB Applications : Printing in a Silverlight LOB Application
  •  Building LOB Applications : Data Validation through Data Annotation
  •  Building LOB Applications : Implementing CRUD Operations in RIA Services
  •  Microsoft XNA Game Studio 3.0 : Displaying Images - Resources and Content (part 2) - Adding Resources to a Project
  •  Microsoft XNA Game Studio 3.0 : Displaying Images - Resources and Content (part 1)
  •  iPhone 3D Programming : Blending and Augmented Reality - Rendering Anti-Aliased Lines with Textures
  •  Programming with DirectX : Game Math - Bounding Geometry (part 2) - Bounding Spheres & Bounding Hierarchies
  •  Programming with DirectX : Game Math - Bounding Geometry (part 1) - Bounding Boxes
  •  Programming with DirectX : Game Math - Matrices
  •  
    Most View
    Golden Media Spark One - Plenty To Offer Out Of The Box (Part 1)
    Introducing UEFI BIOS (Part 2)
    How To Automate Your Web With ifttt (Part 1)
    Corsair Carbide 200r - Joy To Build
    Panasonic Lumix DMC-SZ9 - Lots Of Smart Features In A Very Small Camera
    HTC One - A Huge Leap For Android Smartphones
    Windows Server 2003 : Advanced Backup and Restore (part 1) - Backup Options, The Ntbackup Command
    Linux vs Windows 8 (Part 5)
    Gigabyte Osmium Aivia Mechanical Keyboard
    The Complete Guide To Photography On Your Mac! (Part 2)
    Top 10
    Does Microsoft Have An Image Problem? (Part 2)
    Does Microsoft Have An Image Problem? (Part 1)
    Time For A Bigger iPhone?
    99 Mac Secrets (Part 5) - Top ten third-party apps
    99 Mac Secrets (Part 4) - iMovie secrets, GarageBand secrets, iWork secrets
    99 Mac Secrets (Part 3) : Safari secrets, Mail secrets, Safari shortcuts, Mail shortcuts, iPhoto secrets
    99 Mac Secrets (Part 2) : Customizing, Best menu bar add-ons, Quick Look secrets
    99 Mac Secrets (Part 1) : General OS X tips, Security tips, System shortcuts
    iMovie Trailers And Audio Premastered
    PowerTraveller Powerchimp 4A Battery Charger