iPhone 3D Programming : Anti-Aliasing Tricks with Offscreen FBOs (part 1) - A Super Simple Sample App for Supersampling

The iPhone’s first-class support for framebuffer objects is perhaps its greatest enabler of unique effects. In every sample presented so far in this book, we’ve been using a single FBO, namely, the FBO that represents the visible Core Graphics layer. It’s important to realize that FBOs can also be created as offscreen surfaces, meaning they don’t show up on the screen unless bound to a texture. In fact, on most platforms, FBOs are always offscreen. The iPhone is rather unique in that the visible layer is itself treated as an FBO (albeit a special one).

Binding offscreen FBOs to textures enables a whole slew of interesting effects, including page-curling animations, light blooming, and more. Several sneaky tricks with FBOs can be used to achieve full-scene anti-aliasing, even though the iPhone does not directly support anti-aliasing! We’ll cover two of these techniques in the following subsections.

1. A Super Simple Sample App for Supersampling

The easiest and crudest way to achieve full-scene anti-aliasing on the iPhone is to leverage bilinear texture filtering. Simply render to an offscreen FBO that has twice the dimensions of the screen, and then bind it to a texture and scale it down, as shown in Figure 1. This technique is known as supersampling.

Figure 1. Supersampling

To demonstrate how to achieve this effect, we’ll walk through the process of extending the stencil sample to use supersampling. As an added bonus, we’ll throw in an Apple-esque flipping animation, as shown in Figure 2. Since we’re creating a secondary FBO anyway, flipping effects like this come virtually for free.

Figure 2. Flipping transition with FBO

Example 1 shows the RenderingEngine class declaration and related type definitions. Class members that carry over from previous samples are replaced with an ellipses for brevity.

Example 1. RenderingEngine declaration for the anti-aliasing sample

struct Framebuffers { GLuint Small; GLuint Big; }; struct Renderbuffers { GLuint SmallColor; GLuint BigColor; GLuint BigDepth; GLuint BigStencil; }; struct Textures { GLuint Marble; GLuint RhinoBackground; GLuint TigerBackground; GLuint OffscreenSurface; }; class RenderingEngine : public IRenderingEngine { public: RenderingEngine(IResourceManager* resourceManager); void Initialize(); void Render(float objectTheta, float fboTheta) const; private: ivec2 GetFboSize() const; Textures m_textures; Renderbuffers m_renderbuffers; Framebuffers m_framebuffers; // ... };

First let’s take a look at the GetFboSize implementation (Example 2), which returns a width-height pair for the size. The return type is an instance of ivec2, one of the types defined in the C++ vector library in the appendix.

Example 2. GetFboSize() implementation

ivec2 RenderingEngine::GetFboSize() const { ivec2 size; glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &size.x); glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &size.y); return size; }

Next let’s deal with the creation of the two FBOs. Recall the steps for creating the on-screen FBO used in almost every sample so far:

1. In the RenderingEngine constructor, generate an identifier for the color renderbuffer, and then bind it to the pipeline.

2. In the GLView class (Objective-C), allocate storage for the color renderbuffer like so:

   [m_context renderbufferStorage:GL_RENDERBUFFER fromDrawable:eaglLayer]

   
   

3. In the RenderingEngine::Initialize method, create a framebuffer object, and attach the color renderbuffer to it.

4. If desired, create and allocate renderbuffers for depth and stencil, and then attach them to the FBO.

For the supersampling sample that we’re writing, we still need to perform the first three steps in the previous sequence, but then we follow it with the creation of the offscreen FBO. Unlike the on-screen FBO, its color buffer is allocated in much the same manner as depth and stencil:

glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_RGBA8_OES, width, height);

See Example 3 for the Initialize method used in the supersampling sample.

Example 3. Initialize() for supersampling

void RenderingEngine::Initialize() { // Create the on-screen FBO. glGenFramebuffersOES(1, &m_framebuffers.Small); glBindFramebufferOES(GL_FRAMEBUFFER_OES, m_framebuffers.Small); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, m_renderbuffers.SmallColor); // Create the double-size off-screen FBO. ivec2 size = GetFboSize() * 2; glGenRenderbuffersOES(1, &m_renderbuffers.BigColor); glBindRenderbufferOES(GL_RENDERBUFFER_OES, m_renderbuffers.BigColor); glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_RGBA8_OES, size.x, size.y); glGenRenderbuffersOES(1, &m_renderbuffers.BigDepth); glBindRenderbufferOES(GL_RENDERBUFFER_OES, m_renderbuffers.BigDepth); glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_DEPTH_COMPONENT24_OES, size.x, size.y); glGenRenderbuffersOES(1, &m_renderbuffers.BigStencil); glBindRenderbufferOES(GL_RENDERBUFFER_OES, m_renderbuffers.BigStencil); glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_STENCIL_INDEX8_OES, size.x, size.y); glGenFramebuffersOES(1, &m_framebuffers.Big); glBindFramebufferOES(GL_FRAMEBUFFER_OES, m_framebuffers.Big); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, m_renderbuffers.BigColor); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES, GL_RENDERBUFFER_OES, m_renderbuffers.BigDepth); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_STENCIL_ATTACHMENT_OES, GL_RENDERBUFFER_OES, m_renderbuffers.BigStencil); // Create a texture object and associate it with the big FBO. glGenTextures(1, &m_textures.OffscreenSurface); glBindTexture(GL_TEXTURE_2D, m_textures.OffscreenSurface); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, size.x, size.y, 0, GL_RGBA, GL_UNSIGNED_BYTE, 0); glFramebufferTexture2DOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_TEXTURE_2D, m_textures.OffscreenSurface, 0); // Check FBO status. GLenum status = glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES); if (status != GL_FRAMEBUFFER_COMPLETE_OES) { cout << "Incomplete FBO" << endl; exit(1); } // Load textures, create VBOs, set up various GL state. ... }

You may have noticed two new FBO-related function calls in Example 6-10: glFramebufferTexture2DOES and glCheckFramebufferStatusOES. The formal function declarations look like this:

void glFramebufferTexture2DOES(GLenum target, 
                               GLenum attachment, GLenum textarget,
                               GLuint texture, GLint level);

GLenum glCheckFramebufferStatusOES(GLenum target);

(As usual, the OES suffix can be removed for ES 2.0.)

The glFramebufferTexture2DOES function allows you to cast a color buffer into a texture object. FBO texture objects get set up just like any other texture object: they have an identifier created with glGenTextures, they have filter and wrap modes, and they have a format that should match the format of the FBO. The main difference with FBO textures is the fact that null gets passed to the last argument of glTexImage2D, since there’s no image data to upload.

Note that the texture in Example 3 has non-power-of-two dimensions, so it specifies clamp-to-edge wrapping to accommodate third-generation devices. For older iPhones, the sample won’t work; you’d have to change it to POT dimensions.

The other new function, glCheckFramebufferStatusOES, is a useful sanity check to make sure that an FBO has been set up properly. It’s easy to bungle the creation of FBOs if the sizes of the attachments don’t match up or if their formats are incompatible with each other. glCheckFramebufferStatusOES returns one of the following values, which are fairly self-explanatory:

• GL_FRAMEBUFFER_COMPLETE

• GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT

• GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT

• GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS

• GL_FRAMEBUFFER_INCOMPLETE_FORMATS

• GL_FRAMEBUFFER_UNSUPPORTED

General OpenGL Errors OpenGL ES also supports a more generally useful diagnostic function called glGetError. Its function declaration is simple: GLenum glGetError(); The possible return values are: GL_NO_ERROR GL_INVALID_ENUM GL_INVALID_VALUE GL_INVALID_OPERATION GL_STACK_OVERFLOW GL_STACK_UNDERFLOW GL_OUT_OF_MEMORY Although this book doesn’t call glGetError in any of the sample code, always keep it in mind as a useful debugging tool. Some developers like to sprinkle it throughout their OpenGL code as a matter of habit, much like an assert. Aside from building FBO objects, another error-prone activity in OpenGL is building shader objects in ES 2.0.

Next let’s take a look at the render method of the supersampling sample. Recall from the class declaration that the application layer passes in objectTheta to control the rotation of the podium and passes in fboTheta to control the flipping transitions. So, the first thing the Render method does is look at fboTheta to determine which background image should be displayed and which shape should be shown on the podium. See Example 4.

Example 4. Render() for supersampling

void RenderingEngine::Render(float objectTheta, float fboTheta) const { Drawable drawable; GLuint background; vec3 color; // Look at fboTheta to determine which "side" should be rendered: // 1) Orange Trefoil knot against a Tiger background // 2) Green Klein bottle against a Rhino background if (fboTheta > 270 || fboTheta < 90) { background = m_textures.TigerBackground; drawable = m_drawables.Knot; color = vec3(1, 0.5, 0.1); } else { background = m_textures.RhinoBackground; drawable = m_drawables.Bottle; color = vec3(0.5, 0.75, 0.1); } // Bind the double-size FBO. glBindFramebufferOES(GL_FRAMEBUFFER_OES, m_framebuffers.Big); glBindRenderbufferOES(GL_RENDERBUFFER_OES, m_renderbuffers.BigColor); ivec2 bigSize = GetFboSize(); glViewport(0, 0, bigSize.x, bigSize.y); // Draw the 3D scene - download the example to see this code. ... // Render the background. glColor4f(0.7, 0.7, 0.7, 1); glBindTexture(GL_TEXTURE_2D, background); glMatrixMode(GL_PROJECTION); glLoadIdentity(); glFrustumf(-0.5, 0.5, -0.5, 0.5, NearPlane, FarPlane); glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glTranslatef(0, 0, -NearPlane * 2); RenderDrawable(m_drawables.Quad); glColor4f(1, 1, 1, 1); glDisable(GL_BLEND); // Switch to the on-screen render target. glBindFramebufferOES(GL_FRAMEBUFFER_OES, m_framebuffers.Small); glBindRenderbufferOES(GL_RENDERBUFFER_OES, m_renderbuffers.SmallColor); ivec2 smallSize = GetFboSize(); glViewport(0, 0, smallSize.x, smallSize.y); // Clear the color buffer only if necessary. if ((int) fboTheta % 180 != 0) { glClearColor(0, 0, 0, 1); glClear(GL_COLOR_BUFFER_BIT); } // Render the offscreen surface by applying it to a quad. glDisable(GL_DEPTH_TEST); glRotatef(fboTheta, 0, 1, 0); glBindTexture(GL_TEXTURE_2D, m_textures.OffscreenSurface); RenderDrawable(m_drawables.Quad); glDisable(GL_TEXTURE_2D); }

Most of Example 6-11 is fairly straightforward. One piece that may have caught your eye is the small optimization made right before blitting the offscreen FBO to the screen:

// Clear the color buffer only if necessary.
if ((int) fboTheta % 180 != 0) {
    glClearColor(0, 0, 0, 1);
    glClear(GL_COLOR_BUFFER_BIT);
}

This is a sneaky little trick. Since the quad is the exact same size as the screen, there’s no need to clear the color buffer; unnecessarily issuing a glClear can hurt performance. However, if a flipping animation is currently underway, the color buffer needs to be cleared to prevent artifacts from appearing in the background; flip back to Figure 2 and observe the black areas. If fboTheta is a multiple of 180, then the quad completely fills the screen, so there’s no need to issue a clear.

Figure 3. Left: normal rendering; right: 2× supersampling

That’s it for the supersampling sample. The quality of the anti-aliasing is actually not that great; you can still see some “stair-stepping” along the bottom outline of the shape in Figure 3. You might think that creating an even bigger offscreen buffer, say quadruple-size, would provide higher-quality results. Unfortunately, using a quadruple-size buffer would require two passes; directly applying a 1280×1920 texture to a 320×480 quad isn’t sufficient because GL_LINEAR filtering only samples from a 2×2 neighborhood of pixels. To achieve the desired result, you’d actually need three FBOs as follows:

• 1280×1920 offscreen FBO for the 3D scene

• 640×960 offscreen FBO that contains a quad with the 1280×1920 texture applied to it

• 320×480 on-screen FBO that contains a quad with the 640×960 texture applied to it

Not only is this laborious, but it’s a memory hog. Older iPhones don’t even support textures this large! It turns out there’s another anti-aliasing strategy called jittering, and it can produce high-quality results without the memory overhead of supersampling.