MULTIMEDIA

iPhone 3D Programming : Blending and Augmented Reality - Poor Man’s Reflection with the Stencil Buffer

2/23/2011 7:54:33 PM
One use for blending in a 3D scene is overlaying a reflection on top of a surface, as shown on the left of Figure 1. Remember, computer graphics is often about cheating! To create the reflection, you can redraw the object using an upside-down projection matrix. Note that you need a way to prevent the reflection from “leaking” outside the bounds of the reflective surface, as shown on the right in Figure 1. How can this be done?
Figure 1. Left: reflection with stencil; right: reflection without stencil


It turns out that third-generation iPhones and iPod touches have support for an OpenGL ES feature known as the stencil buffer, and it’s well-suited to this problem. The stencil buffer is actually just another type of renderbuffer, much like color and depth. But instead of containing RGB or Z values, it holds a small integer value at every pixel that you can use in different ways. There are many applications for the stencil buffer beyond clipping.


Note:

To accommodate older iPhones, we’ll cover some alternatives to stenciling later in the chapter.


To check whether stenciling is supported on the iPhone, check for the GL_OES_stencil8 extension using the method in this article. At the time of this writing, stenciling is supported on third-generation devices and the simulator, but not on first- and second-generation devices.

The reflection trick can be achieved in four steps (see Figure 2):

  1. Render the disk to stencil only.

  2. Render the reflection of the floating object with the stencil test enabled.

  3. Clear the depth buffer, and render the actual floating object.

  4. Render the disk using front-to-back blending.

Figure 2. Rendering a reflection in four steps


Note that the reflection is drawn before the textured podium, which is the reason for the front-to-back blending. We can’t render the reflection after the podium because blending and depth-testing cannot both be enabled when drawing complex geometry.

First let’s take a look at the creation of the stencil buffer itself. The first few steps are generating a renderbuffer identifier, binding it, and allocating storage. This may look familiar if you remember how to create the depth buffer:

GLuint stencil;
glGenRenderbuffersOES(1, &stencil);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, stencil);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_STENCIL_INDEX8_OES, width, height);


Next, attach the stencil buffer to the framebuffer object, shown in bold here:
GLuint framebuffer;
glGenFramebuffersOES(1, &framebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES,
GL_RENDERBUFFER_OES, color);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES,
GL_RENDERBUFFER_OES, depth);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_STENCIL_ATTACHMENT_OES,
GL_RENDERBUFFER_OES, stencil);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, color);

As always, remember to omit the OES endings when working with ES 2.0.

To save memory, sometimes you can interleave the depth buffer and stencil buffer into a single renderbuffer. This is possible only when the OES_packed_depth_stencil extension is supported. At the time of this writing, it’s available on third-generation devices, but not on the simulator or older devices. To see how to use this extension, see Example 1. Relevant portions are highlighted in bold.

Example 1. Using packed depth stencil
GLuint depthStencil;
glGenRenderbuffersOES(1, &depthStencil);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, depthStencil);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_DEPTH24_STENCIL8_OES, width, height);

GLuint framebuffer;
glGenFramebuffersOES(1, &framebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES,
GL_RENDERBUFFER_OES, color);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES,
GL_RENDERBUFFER_OES, depthStencil);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_STENCIL_ATTACHMENT_OES,
GL_RENDERBUFFER_OES, depthStencil);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, color);



1. Rendering the Disk to Stencil Only

Recall that step 1 in our reflection demo renders the disk to the stencil buffer. Before drawing to the stencil buffer, it needs to be cleared, just like any other renderbuffer:

glClearColor(0, 0, 0, 1);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

Next you need to tell OpenGL to enable writes to the stencil buffer, and you need to tell it what stencil value you’d like to write. Since you’re using an 8-bit buffer in this case, you can set any value between 0x00 and 0xff. Let’s go with 0xff and set up the OpenGL state like this:

glEnable(GL_STENCIL_TEST);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
glStencilFunc(GL_ALWAYS, 0xff, 0xff);

The first line enables GL_STENCIL_TEST, which is a somewhat misleading name in this case; you’re writing to the stencil buffer, not testing against it. If you don’t enable GL_STENCIL_TEST, then OpenGL assumes you’re not working with the stencil buffer at all.

The next line, glStencilOp, tells OpenGL which stencil operation you’d like to perform at each pixel. Here’s the formal declaration:

void glStencilOp(GLenum fail, GLenum zfail, GLenum zpass);

GLenum fail

Specifies the operation to perform when the stencil test fails

GLenum zfail

Specifies the operation to perform when the stencil test passes and the depth test fails

GLenum zpass

Specifies the operation to perform when the stencil test passes and the depth test passes

Since the disk is the first draw call in the scene, we don’t care whether any of these tests fail, so we’ve set them all to the same value.

Each of the arguments to glStencilOp can be one of the following:

GL_REPLACE

Replace the value that’s currently in the stencil buffer with the value specified in glStencilFunc.

GL_KEEP

Don’t do anything.

GL_INCR

Increment the value that’s currently in the stencil buffer.

GL_DECR

Decrement the value that’s currently in the stencil buffer.

GL_INVERT

Perform a bitwise NOT operation with the value that’s currently in the stencil buffer.

GL_ZERO

Clobber the current stencil buffer value with zero.

Again, this may seem like way too much flexibility, more than you’d ever need. Later in this book, you’ll see how all this freedom can be used to perform interesting tricks. For now, all we’re doing is writing the shape of the disk out to the stencil buffer, so we’re using the GL_REPLACE operation.

The next function we called to set up our stencil state is glStencilFunc. Here’s its function declaration:

void glStencilFunc(GLenum func, GLint ref, GLuint mask);
GLenum func

This specifies the comparison function to use for the stencil test.

GLint ref

This “reference value” actually serves two purposes:

  • Comparison value to test against if func is something other than GL_ALWAYS or GL_NEVER

  • The value to write if the operation is GL_REPLACE

GLuint mask

Before performing a comparison, this bitmask gets ANDed with both the reference value and the value that’s already in the buffer.

Again, this gives the developer quite a bit of power, but in this case we only need something simple.

Getting back to the task at hand, check out Example 2 to see how to render the disk to the stencil buffer only. I adjusted the indentation of the code to show how certain pieces of OpenGL state get modified before the draw call and then restored after the draw call.

Example 2. Rendering the disk to stencil only
// Prepare the render state for the disk.
glEnable(GL_STENCIL_TEST);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
glStencilFunc(GL_ALWAYS, 0xff, 0xff);

// Render the disk to the stencil buffer only.
glDisable(GL_TEXTURE_2D);
glTranslatef(0, DiskY, 0);
glDepthMask(GL_FALSE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
RenderDrawable(m_drawables.Disk); // private method that calls glDrawElements
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glDepthMask(GL_TRUE);
glTranslatef(0, -DiskY, 0);
glEnable(GL_TEXTURE_2D);

Two new function calls appear in Example 6-3: glDepthMask and glColorMask. Recall that we’re interested in affecting values in the stencil buffer only. It’s actually perfectly fine to write to all three renderbuffers (color, depth, stencil), but to maximize performance, it’s good practice to disable any writes that you don’t need.

The four arguments to glColorMask allow you to toggle each of the individual color channels; in this case we don’t need any of them. Note that glDepthMask has only one argument, since it’s a single-component buffer. Incidentally, OpenGL ES also provides a glStencilMask function, which we’re not using here.

2. Rendering the Reflected Object with Stencil Testing

Step 2 renders the reflection of the object and uses the stencil buffer to clip it to the boundary of the disk. Example 3 shows how to do this.

Example 3. Rendering the reflection
glTranslatef(0, KnotY, 0);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glStencilFunc(GL_EQUAL, 0xff, 0xff);
glEnable(GL_LIGHTING);
glBindTexture(GL_TEXTURE_2D, m_textures.Grille);

const float alpha = 0.4f;
vec4 diffuse(alpha, alpha, alpha, 1 - alpha);
glMaterialfv(GL_FRONT_AND_BACK, GL_DIFFUSE, diffuse.Pointer());

glMatrixMode(GL_PROJECTION);
glLoadMatrixf(m_mirrorProjection.Pointer());
RenderDrawable(m_drawables.Knot); // private method that calls glDrawElements
glLoadMatrixf(m_projection.Pointer());
glMatrixMode(GL_MODELVIEW);

This time we don’t need to change the values in the stencil buffer, so we use GL_KEEP for the argument to glStencilOp. We changed the stencil comparison function to GL_EQUAL so that only the pixels within the correct region will pass.

There are several ways you could go about drawing an object upside down, but I chose to do it with a quick-and-dirty projection matrix. The result isn’t a very accurate reflection, but it’s good enough to fool the viewer! Example 4 shows how I did this using a mat4 method from the C++ vector library in the appendix. (For ES 1.1, you could simply use the provided glFrustum function.)

Example 4. Computing two projection matrices
const float AspectRatio = (float) height / width;
const float Shift = -1.25;
const float Near = 5;
const float Far = 50;

m_projection = mat4::Frustum(-1, 1,
-AspectRatio, AspectRatio,
Near, Far);

m_mirrorProjection = mat4::Frustum(-1, 1,
AspectRatio + Shift, -AspectRatio + Shift,
Near, Far);

3. Rendering the “Real” Object

The next step is rather mundane; we simply need to render the actual floating object, without doing anything with the stencil buffer. Before calling glDrawElements for the object, we turn off the stencil test and disable the depth buffer:

glDisable(GL_STENCIL_TEST);
glClear(GL_DEPTH_BUFFER_BIT);

For the first time, we’ve found a reason to call glClear somewhere in the middle of the Render method! Importantly, we’re clearing only the depth buffer, leaving the color buffer intact.

Remember, the reflection is drawn just like any other 3D object, complete with depth testing. Allowing the actual object to be occluded by the reflection would destroy the illusion, so it’s a good idea to clear the depth buffer before drawing it. Given the fixed position of the camera in our demo, we could actually get away without performing the clear, but this allows us to tweak the demo without breaking anything.

4. Rendering the Disk with Front-to-Back Blending

The final step is rendering the marble disk underneath the reflection. Example 5 sets this up.

Example 5. Render the disk to the color buffer
glTranslatef(0, DiskY - KnotY, 0);
glDisable(GL_LIGHTING);
glBindTexture(GL_TEXTURE_2D, m_textures.Marble);
glBlendFuncSeparateOES(GL_DST_ALPHA, GL_ONE, // RGB factors
GL_ZERO, GL_ONE_MINUS_SRC_ALPHA); // Alpha factors
glEnable(GL_BLEND);

Other  
  •  Microsoft XNA Game Studio 3.0 : Getting Player Input - Reading a Gamepad
  •  iPhone 3D Programming : Blending and Augmented Reality - Shifting Texture Color with Per-Vertex Color
  •  iPhone 3D Programming : Blending and Augmented Reality - Blending Extensions and Their Uses
  •  iPhone 3D Programming : Blending and Augmented Reality - Blending Caveats
  •  Building LOB Applications : Using Visual Studio 2010 WCF RIA Data Services Tooling
  •  Building LOB Applications : Implementing CRUD Operations in WCF Data Services
  •  iPhone 3D Programming : Blending and Augmented Reality - Wrangle Premultiplied Alpha
  •  iPhone 3D Programming : Blending and Augmented Reality - Blending Recipe
  •  Microsoft XNA Game Studio 3.0 : Controlling Color (part 3)
  •  Microsoft XNA Game Studio 3.0 : Controlling Color (part 2)
  •  Microsoft XNA Game Studio 3.0 : Controlling Color (part 1) - Games and Classes & Classes as Offices
  •  Microsoft XNA Game Studio 3.0 : Working with Colors
  •  iPhone 3D Programming : Textures and Image Capture - Creating Textures with the Camera
  •  iPhone 3D Programming : Textures and Image Capture - Dealing with Size Constraints
  •  Programming with DirectX : Game Math - Vectors
  •  iPhone 3D Programming : Textures and Image Capture - Generating and Transforming OpenGL Textures with Quartz
  •  iPhone 3D Programming : Textures and Image Capture - The PowerVR SDK and Low-Precision Textures
  •  Building LOB Applications : Using Visual Studio 2010 WCF Data Services Tooling
  •  Building LOB Applications : Accessing RESTful Data using OData
  •  Programming with DirectX : Additional Texture Mapping - Image Filters
  •  
    Most View
    Executing Work on a Background Thread with Updates
    SQL Server 2008 : Transact-SQL Programming - The max Specifier
    Information Theory
    Buying Guide: Mid-Price Flashguns (Part 2) : Nissin D1866 Mark Ii, Sigma EF-610 DG Super, Sunpak PZ42X
    Seagate Backup Plus Portable – A Huge Plus To Storage
    Using Group Policy in Windows Vista
    Windows 7 : Command-Line and Automation Tools - Windows Script Host
    Lian Li PC-TU200
    Server 2008 : Using the Integrated Windows Firewall with Advanced Security
    Musical Fidelity M1 CLiC Universal Music Controller (Part 2)
    Top 10
    ADO.NET Programming : Microsoft SQL Server (part 4) - Working with Typed Data Sets
    ADO.NET Programming : Microsoft SQL Server (part 3) - Using Stored Procedures with DataSet Objects
    ADO.NET Programming : Microsoft SQL Server (part 2) - Using SQL Server Stored Procedures
    ADO.NET Programming : Microsoft SQL Server (part 1) - Connecting to SQL Server, Creating Command Objects
    Windows Phone 8 In-Depth Review (Part 6)
    Windows Phone 8 In-Depth Review (Part 5)
    Windows Phone 8 In-Depth Review (Part 4)
    Windows Phone 8 In-Depth Review (Part 3)
    Windows Phone 8 In-Depth Review (Part 2)
    Windows Phone 8 In-Depth Review (Part 1)