iPhone 3D Programming : Blending and Augmented Reality - Poor Man’s Reflection with the Stencil Buffer

One use for blending in a 3D scene is overlaying a reflection on top of a surface, as shown on the left of Figure 1. Remember, computer graphics is often about cheating! To create the reflection, you can redraw the object using an upside-down projection matrix. Note that you need a way to prevent the reflection from “leaking” outside the bounds of the reflective surface, as shown on the right in Figure 1. How can this be done?

Figure 1. Left: reflection with stencil; right: reflection without stencil

It turns out that third-generation iPhones and iPod touches have support for an OpenGL ES feature known as the stencil buffer, and it’s well-suited to this problem. The stencil buffer is actually just another type of renderbuffer, much like color and depth. But instead of containing RGB or Z values, it holds a small integer value at every pixel that you can use in different ways. There are many applications for the stencil buffer beyond clipping.

---

Note:

To accommodate older iPhones, we’ll cover some alternatives to stenciling later in the chapter.

---

To check whether stenciling is supported on the iPhone, check for the GL_OES_stencil8 extension using the method in this article. At the time of this writing, stenciling is supported on third-generation devices and the simulator, but not on first- and second-generation devices.

The reflection trick can be achieved in four steps (see Figure 2):

1. Render the disk to stencil only.

2. Render the reflection of the floating object with the stencil test enabled.

3. Clear the depth buffer, and render the actual floating object.

4. Render the disk using front-to-back blending.

Figure 2. Rendering a reflection in four steps

Note that the reflection is drawn before the textured podium, which is the reason for the front-to-back blending. We can’t render the reflection after the podium because blending and depth-testing cannot both be enabled when drawing complex geometry.

First let’s take a look at the creation of the stencil buffer itself. The first few steps are generating a renderbuffer identifier, binding it, and allocating storage. This may look familiar if you remember how to create the depth buffer:

GLuint stencil;
glGenRenderbuffersOES(1, &stencil);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, stencil);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_STENCIL_INDEX8_OES, width, height);


					  

Next, attach the stencil buffer to the framebuffer object, shown in bold here:

GLuint framebuffer;
glGenFramebuffersOES(1, &framebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES,
                             GL_RENDERBUFFER_OES, color);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES,
                             GL_RENDERBUFFER_OES, depth);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_STENCIL_ATTACHMENT_OES,
                             GL_RENDERBUFFER_OES, stencil);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, color);

As always, remember to omit the OES endings when working with ES 2.0.

To save memory, sometimes you can interleave the depth buffer and stencil buffer into a single renderbuffer. This is possible only when the OES_packed_depth_stencil extension is supported. At the time of this writing, it’s available on third-generation devices, but not on the simulator or older devices. To see how to use this extension, see Example 1. Relevant portions are highlighted in bold.

Example 1. Using packed depth stencil

GLuint depthStencil; glGenRenderbuffersOES(1, &depthStencil); glBindRenderbufferOES(GL_RENDERBUFFER_OES, depthStencil); glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_DEPTH24_STENCIL8_OES, width, height); GLuint framebuffer; glGenFramebuffersOES(1, &framebuffer); glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, color); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES, GL_RENDERBUFFER_OES, depthStencil); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_STENCIL_ATTACHMENT_OES, GL_RENDERBUFFER_OES, depthStencil); glBindRenderbufferOES(GL_RENDERBUFFER_OES, color);

1. Rendering the Disk to Stencil Only

Recall that step 1 in our reflection demo renders the disk to the stencil buffer. Before drawing to the stencil buffer, it needs to be cleared, just like any other renderbuffer:

glClearColor(0, 0, 0, 1);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

Next you need to tell OpenGL to enable writes to the stencil buffer, and you need to tell it what stencil value you’d like to write. Since you’re using an 8-bit buffer in this case, you can set any value between 0x00 and 0xff. Let’s go with 0xff and set up the OpenGL state like this:

glEnable(GL_STENCIL_TEST);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
glStencilFunc(GL_ALWAYS, 0xff, 0xff);

The first line enables GL_STENCIL_TEST, which is a somewhat misleading name in this case; you’re writing to the stencil buffer, not testing against it. If you don’t enable GL_STENCIL_TEST, then OpenGL assumes you’re not working with the stencil buffer at all.

The next line, glStencilOp, tells OpenGL which stencil operation you’d like to perform at each pixel. Here’s the formal declaration:

void glStencilOp(GLenum fail, GLenum zfail, GLenum zpass);

GLenum fail

Specifies the operation to perform when the stencil test fails

GLenum zfail

Specifies the operation to perform when the stencil test passes and the depth test fails

GLenum zpass

Specifies the operation to perform when the stencil test passes and the depth test passes

Since the disk is the first draw call in the scene, we don’t care whether any of these tests fail, so we’ve set them all to the same value.

Each of the arguments to glStencilOp can be one of the following:

GL_REPLACE

Replace the value that’s currently in the stencil buffer with the value specified in glStencilFunc.

GL_KEEP

Don’t do anything.

GL_INCR

Increment the value that’s currently in the stencil buffer.

GL_DECR

Decrement the value that’s currently in the stencil buffer.

GL_INVERT

Perform a bitwise NOT operation with the value that’s currently in the stencil buffer.

GL_ZERO

Clobber the current stencil buffer value with zero.

Again, this may seem like way too much flexibility, more than you’d ever need. Later in this book, you’ll see how all this freedom can be used to perform interesting tricks. For now, all we’re doing is writing the shape of the disk out to the stencil buffer, so we’re using the GL_REPLACE operation.

The next function we called to set up our stencil state is glStencilFunc. Here’s its function declaration:

void glStencilFunc(GLenum func, GLint ref, GLuint mask);

GLenum func

This specifies the comparison function to use for the stencil test.

GLint ref

This “reference value” actually serves two purposes:

• Comparison value to test against if func is something other than GL_ALWAYS or GL_NEVER

• The value to write if the operation is GL_REPLACE

GLuint mask

Before performing a comparison, this bitmask gets ANDed with both the reference value and the value that’s already in the buffer.

Again, this gives the developer quite a bit of power, but in this case we only need something simple.

Getting back to the task at hand, check out Example 2 to see how to render the disk to the stencil buffer only. I adjusted the indentation of the code to show how certain pieces of OpenGL state get modified before the draw call and then restored after the draw call.

Example 2. Rendering the disk to stencil only

// Prepare the render state for the disk. glEnable(GL_STENCIL_TEST); glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE); glStencilFunc(GL_ALWAYS, 0xff, 0xff); // Render the disk to the stencil buffer only. glDisable(GL_TEXTURE_2D); glTranslatef(0, DiskY, 0); glDepthMask(GL_FALSE); glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE); RenderDrawable(m_drawables.Disk); // private method that calls glDrawElements glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE); glDepthMask(GL_TRUE); glTranslatef(0, -DiskY, 0); glEnable(GL_TEXTURE_2D);

Two new function calls appear in Example 6-3: glDepthMask and glColorMask. Recall that we’re interested in affecting values in the stencil buffer only. It’s actually perfectly fine to write to all three renderbuffers (color, depth, stencil), but to maximize performance, it’s good practice to disable any writes that you don’t need.

The four arguments to glColorMask allow you to toggle each of the individual color channels; in this case we don’t need any of them. Note that glDepthMask has only one argument, since it’s a single-component buffer. Incidentally, OpenGL ES also provides a glStencilMask function, which we’re not using here.

2. Rendering the Reflected Object with Stencil Testing

Step 2 renders the reflection of the object and uses the stencil buffer to clip it to the boundary of the disk. Example 3 shows how to do this.

Example 3. Rendering the reflection

glTranslatef(0, KnotY, 0); glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); glStencilFunc(GL_EQUAL, 0xff, 0xff); glEnable(GL_LIGHTING); glBindTexture(GL_TEXTURE_2D, m_textures.Grille); const float alpha = 0.4f; vec4 diffuse(alpha, alpha, alpha, 1 - alpha); glMaterialfv(GL_FRONT_AND_BACK, GL_DIFFUSE, diffuse.Pointer()); glMatrixMode(GL_PROJECTION); glLoadMatrixf(m_mirrorProjection.Pointer()); RenderDrawable(m_drawables.Knot); // private method that calls glDrawElements glLoadMatrixf(m_projection.Pointer()); glMatrixMode(GL_MODELVIEW);

This time we don’t need to change the values in the stencil buffer, so we use GL_KEEP for the argument to glStencilOp. We changed the stencil comparison function to GL_EQUAL so that only the pixels within the correct region will pass.

There are several ways you could go about drawing an object upside down, but I chose to do it with a quick-and-dirty projection matrix. The result isn’t a very accurate reflection, but it’s good enough to fool the viewer! Example 4 shows how I did this using a mat4 method from the C++ vector library in the appendix. (For ES 1.1, you could simply use the provided glFrustum function.)

Example 4. Computing two projection matrices

Code View: Scroll / Show All const float AspectRatio = (float) height / width; const float Shift = -1.25; const float Near = 5; const float Far = 50; m_projection = mat4::Frustum(-1, 1, -AspectRatio, AspectRatio, Near, Far); m_mirrorProjection = mat4::Frustum(-1, 1, AspectRatio + Shift, -AspectRatio + Shift, Near, Far);

3. Rendering the “Real” Object

The next step is rather mundane; we simply need to render the actual floating object, without doing anything with the stencil buffer. Before calling glDrawElements for the object, we turn off the stencil test and disable the depth buffer:

glDisable(GL_STENCIL_TEST);
glClear(GL_DEPTH_BUFFER_BIT);

For the first time, we’ve found a reason to call glClear somewhere in the middle of the Render method! Importantly, we’re clearing only the depth buffer, leaving the color buffer intact.

Remember, the reflection is drawn just like any other 3D object, complete with depth testing. Allowing the actual object to be occluded by the reflection would destroy the illusion, so it’s a good idea to clear the depth buffer before drawing it. Given the fixed position of the camera in our demo, we could actually get away without performing the clear, but this allows us to tweak the demo without breaking anything.

4. Rendering the Disk with Front-to-Back Blending

The final step is rendering the marble disk underneath the reflection. Example 5 sets this up.

Example 5. Render the disk to the color buffer

glTranslatef(0, DiskY - KnotY, 0); glDisable(GL_LIGHTING); glBindTexture(GL_TEXTURE_2D, m_textures.Marble); glBlendFuncSeparateOES(GL_DST_ALPHA, GL_ONE, // RGB factors GL_ZERO, GL_ONE_MINUS_SRC_ALPHA); // Alpha factors glEnable(GL_BLEND);