Is a texture a collection of discrete texels,
or is it a continuous function across [0, 1]? This is a dangerous question
to ask a graphics geek; it’s a bit like asking a physicist if a photon is
a wave or a particle.When you upload a texture to OpenGL using
glTexImage2D, it’s a collection of discrete texels.
When you sample a texture using normalized texture coordinates, it’s a bit
more like a continuous function. You might recall these two lines from the
rendering engine:
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
What’s going on here? The first line sets the
minification filter; the second line sets the
magnification filter. Both of these tell OpenGL how
to map those discrete texels into a continuous function.
More precisely, the minification filter
specifies the scaling algorithm to use when the texture size in screen
space is smaller than the original image; the magnification filter tells
OpenGL what to do when the texture size is screen space is larger than the
original image.
The magnification filter can be one of two
values:
- GL_NEAREST
Simple and crude; use the color of the
texel nearest to the texture coordinate.
- GL_LINEAR
Indicates bilinear filtering. Samples the
local 2×2 square of texels and blends them together using a weighted
average. The image on the far right in Figure 1 is an example of bilinear
magnification applied to a simple 8×8 monochrome texture.
The minification filter supports the same
filters as magnification and adds four additional filters that rely on
mipmaps, which are “preshrunk” images that you need
to upload separately from the main image. More on mipmaps soon.
The available minification modes are as
follows:
- GL_NEAREST
As with magnification, use the color of
the nearest texel.
- GL_LINEAR
As with magnification, blend together the
nearest four texels. The middle image in Figure 5-4 is an example of bilinear
minification.
- GL_NEAREST_MIPMAP_NEAREST
Find the mipmap that best matches the
screen-space size of the texture, and then use
GL_NEAREST filtering.
- GL_LINEAR_MIPMAP_NEAREST
Find the mipmap that best matches the
screen-space size of the texture, and then use
GL_LINEAR filtering.
- GL_LINEAR_MIPMAP_LINEAR
Perform GL_LINEAR
sampling on each of two “best fit” mipmaps, and
then blend the result. OpenGL takes eight samples for this, so it’s
the highest-quality filter. This is also known as
trilinear filtering.
- GL_NEAREST_MIPMAP_LINEAR
Take the weighted average of two samples,
where one sample is from mipmap A, the other from mipmap B.
Figure 2 compares
various filtering schemes.
Deciding on a filter is a bit of a black art;
personally I often start with trilinear filtering
(GL_LINEAR_MIPMAP_LINEAR), and I try cranking down to a
lower-quality filter only when I’m optimizing my frame rate. Note that
GL_NEAREST is perfectly acceptable in some scenarios,
such as when rendering 2D quads that have the same size as the source
texture.
First- and second-generation devices have some
restrictions on the filters:
If magnification is
GL_NEAREST, then minification must be one of
GL_NEAREST,
GL_NEAREST_MIPMAP_NEAREST, or
GL_NEAREST_MIPMAP_LINEAR.
If magnification is
GL_LINEAR, then minification must be one of
GL_LINEAR, GL_LINEAR_MIPMAP_NEAREST, or
GL_LINEAR_MIPMAP_LINEAR.
This isn’t a big deal since you’ll almost never
want a different same-level filter for magnification and minification.
Nevertheless, it’s important to note that the iPhone Simulator and newer
devices do not have these restrictions.
1. Boosting Quality and Performance with Mipmaps
Mipmaps help with both quality and
performance. They can help with performance especially when large
textures are viewed from far away. Since the graphics hardware performs
sampling on an image potentially much smaller than the original, it’s
more likely to have the texels available in a nearby memory cache.
Mipmaps can improve quality for several reasons; most importantly, they
effectively cast a wider net, so the final color is less likely to be
missing contributions from important nearby texels.
In OpenGL, mipmap zero is the original image,
and every following level is half the size of the preceding level. If a
level has an odd size, then the floor function is used, as in Mipmap sizes.
Mipmap sizes
Watch out though, because sometimes you need
to ensure that all mipmap levels have an even size. In another words,
the original texture must have dimensions that are powers of two. Figure 3 depicts a popular way of neatly visualizing
mipmaps levels into an area that’s 1.5 times the original width.
To upload the mipmaps levels to OpenGL, you
need to make a series of separate calls to
glTexImage2D, from the original size all the way down
to the 1×1 mipmap:
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 16, 16, 0, GL_RGBA, GL_UNSIGNED_BYTE,
pImageData0);
glTexImage2D(GL_TEXTURE_2D, 1, GL_RGBA, 8, 8, 0, GL_RGBA,
GL_UNSIGNED_BYTE, pImageData1);
glTexImage2D(GL_TEXTURE_2D, 2, GL_RGBA, 4, 4, 0, GL_RGBA,
GL_UNSIGNED_BYTE, pImageData2);
glTexImage2D(GL_TEXTURE_2D, 3, GL_RGBA, 2, 2, 0, GL_RGBA,
GL_UNSIGNED_BYTE, pImageData3);
glTexImage2D(GL_TEXTURE_2D, 4, GL_RGBA, 1, 1, 0, GL_RGBA,
GL_UNSIGNED_BYTE, pImageData4);
Usually code like this occurs in a loop. Many
OpenGL developers like to use a right-shift as a sneaky way of halving
the size at each iteration. I doubt it really buys you anything, but
it’s great fun:
for (int level = 0;
level < description.MipCount;
++level, width >>= 1, height >>= 1, ppData++)
{
glTexImage2D(GL_TEXTURE_2D, level, GL_RGBA, width, height,
0, GL_RGBA, GL_UNSIGNED_BYTE, *ppData);
}
If you’d like to avoid the tedium of creating
mipmaps and loading them in individually, OpenGL ES can generate mipmaps
on your behalf:
// OpenGL ES 1.1
glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP, GL_TRUE);
glTexImage2D(GL_TEXTURE_2D, 0, ...);
// OpenGL ES 2.0
glTexImage2D(GL_TEXTURE_2D, 0, ...);
glGenerateMipmap(GL_TEXTURE_2D);
In ES 1.1, mipmap generation is part of the
OpenGL state associated with the current texture object, and you should
enable it before uploading level zero. In ES 2.0,
mipmap generation is an action that you take after
you upload level zero.
You might be wondering why you’d ever want to
provide mipmaps explicitly when you can just have OpenGL generate them
for you. There are actually a couple reasons for this:
There’s a performance hit for mipmap
generation at upload time. This could prolong your application’s
startup time, which is something all good iPhone developers obsess
about.
When OpenGL performs mipmap generation
for you, you’re (almost) at the mercy of whatever filtering
algorithm it chooses. You can often produce higher-quality results
if you provide mipmaps yourself, especially if you have a very
high-resolution source image
or a vector-based source.
Later we’ll learn about a couple free tools
that make it easy to supply OpenGL with ready-made, preshrunk
mipmaps.
By the way, you do have some control over the
mipmap generation scheme that OpenGL uses. The following lines are valid
with both ES 1.1 and 2.0:
glHint(GL_GENERATE_MIPMAP_HINT, GL_FASTEST);
glHint(GL_GENERATE_MIPMAP_HINT, GL_NICEST);
glHint(GL_GENERATE_MIPMAP_HINT, GL_DONT_CARE); // this is the default
If you’re a control freak and you’d like to
tweak the way OpenGL chooses mipmap levels to sample from, you’ll be
glad to hear the iPhone supports an extension that can shift which
mipmap level(s) get sampled. This is useful for intentional blurring
or pseudosharpening. For more information, head over to the extension
registry on the Khronos site: - http://www.khronos.org/registry/gles/extensions/EXT/texture_lod_bias.txt
This is an ES 1.1 extension only; it’s not
necessary for ES 2.0 because you can bias the mipmap level from within
the fragment shader using an optional third argument to
texture2D. The full function signature looks like
this: vec4 texture2D(sampler2D sampler, vec2 coord, float bias = 0)
Incidentally, don’t confuse
texture2D, which is a shader function for sampling,
and glTexImage2D, which a C function for
uploading. |
2. Modifying ModelViewer to Support Mipmaps
It’s easy to enable mipmapping in the
ModelViewer sample. For the ES 1.1 rendering engine, enable mipmap
generation after binding to the texture object, and then replace the
minification filter:
glGenTextures(1, &m_gridTexture);
glBindTexture(GL_TEXTURE_2D, m_gridTexture);
glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP, GL_TRUE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
// ...
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, size.x,
size.y, 0, GL_RGBA, GL_UNSIGNED_BYTE, pixels);
For the ES 2.0 rendering engine, replace the
minification filter in the same way, but call
glGenerateMipmap after uploading the texture
data:
glGenTextures(1, &m_gridTexture);
glBindTexture(GL_TEXTURE_2D, m_gridTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
// ...
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, size.x,
size.y, 0, GL_RGBA, GL_UNSIGNED_BYTE, pixels);
glGenerateMipmap(GL_TEXTURE_2D);
