iPhone 3D Programming : Textures and Image Capture - Texture Formats and Types

2/5/2011 9:53:01 AM

Recall that two of the parameters to glTexImage2D stipulate format, and one stipulates type, highlighted in bold here:

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 16, 16, 0, 
             GL_RGBA, GL_UNSIGNED_BYTE, pImageData0);

The allowed formats are as follows:

GL_RGB: Three-component color.
GL_RGBA: Four-component color; includes alpha.
GL_BGRA: Same as GL_RGBA but with the blue and red components swapped. This is a nonstandard format but available on the iPhone because of the GL_IMG_texture_format_BGRA8888 extension.
GL_ALPHA: Single component format used as an alpha mask.
GL_LUMINANCE: Single component format used as grayscale.
GL_LUMINANCE_ALPHA: Two component format: grayscale + alpha. This is very useful for storing text.

Don’t dismiss the non-RGB formats; if you don’t need color, you can save significant memory with the one- or two-component formats.

The type parameter in glTexImage2D can be one of these:

GL_UNSIGNED_BYTE: Each color component is 8 bits wide.
GL_UNSIGNED_SHORT_5_6_5: Each pixel is 16 bits wide; red and blue have five bits each, and green has six. Requires the format to be GL_RGB. The fact that green gets the extra bit isn’t random—the human eye is more sensitive to variation in green hues.
GL_UNSIGNED_SHORT_4_4_4_4: Each pixel is 16 bits wide, and each component is 4 bits. This can be used only with GL_RGBA.
GL_UNSIGNED_SHORT_5_5_5_1: This dedicates only one bit to alpha; a pixel can be only fully opaque or fully transparent. Each pixel is 16 bits wide. This requires format to be GL_RGBA.

It’s also interesting to note the various formats supported by the PNG file format, even though this has nothing to do with OpenGL:

Five grayscale formats: each pixel can be 1, 2, 4, 8, or 16 bits wide.
Two RGB formats: each color component can be 8 or 16 bits.
Two “gray with alpha” formats: each component can be 8 or 16 bits.
Two RGBA formats: each component can be 8 or 16 bits.
Paletted formats—we’ll ignore these.

Warning:

Just because a PNG file looks grayscale doesn’t mean that it’s using a grayscale-only format! The iPhone SDK includes a command-line tool called pngcrush that can help with this. You can also right-click an image file in Mac OS X and use the Get Info option to learn about the internal format.

1. Hands-On: Loading Various Formats

We can start by enhancing the IResourceManager interface so that it returns some format information in an API-agnostic way. (Remember, we’re avoiding all platform-specific code in our interfaces.) For simplicity’s sake, let’s support only the subset of formats that are supported by both OpenGL and PNG. Open Interfaces.hpp, and make the changes shown in Example 1 . New and modified lines are shown in bold. Note that the GetImageSize method has been removed because size is part of TextureDescription.

Example 1. Adding format support to IResourceManager

enum TextureFormat {
    TextureFormatGray,
    TextureFormatGrayAlpha,
    TextureFormatRgb,
    TextureFormatRgba,
};

struct TextureDescription {
    TextureFormat Format;
    int BitsPerComponent;
    ivec2 Size;
};

struct IResourceManager {
    virtual string GetResourcePath() const = 0;
    virtual TextureDescription LoadPngImage(const string& filename) = 0;
    virtual void* GetImageData() = 0;
    virtual void UnloadImage() = 0;
    virtual ~IResourceManager() {}
};

Example 2 shows the implementation to the new LoadPngImage method. Note the Core Graphics functions used to extract format and type information, such as CGImageGetAlphaInfo, CGImageGetColorSpace, and CGColorSpaceGetModel. I won’t go into detail about these functions because they are fairly straightforward; for more information, look them up on Apple’s iPhone Developer site.

Example 2. Update to ResourceManager.mm

TextureDescription LoadPngImage(const string& file)
{
    NSString* basePath = [NSString stringWithUTF8String:file.c_str()];
    NSString* resourcePath = [[NSBundle mainBundle] resourcePath];
    NSString* fullPath = [resourcePath stringByAppendingPathComponent:basePath];

    NSLog(@"Loading PNG image %s...", fullPath);

    UIImage* uiImage = [UIImage imageWithContentsOfFile:fullPath];
    CGImageRef cgImage = uiImage.CGImage;
    m_imageData = CGDataProviderCopyData(CGImageGetDataProvider(cgImage));

    TextureDescription description;
    description.Size.x = CGImageGetWidth(cgImage);
    description.Size.y = CGImageGetHeight(cgImage);
    bool hasAlpha = CGImageGetAlphaInfo(cgImage) != kCGImageAlphaNone;
    CGColorSpaceRef colorSpace = CGImageGetColorSpace(cgImage);
    switch (CGColorSpaceGetModel(colorSpace)) {
        case kCGColorSpaceModelMonochrome:
            description.Format = 
              hasAlpha ? TextureFormatGrayAlpha : TextureFormatGray;
            break;
        case kCGColorSpaceModelRGB:
            description.Format = 
              hasAlpha ? TextureFormatRgba : TextureFormatRgb;
            break;
        default:
            assert(!"Unsupported color space.");
            break;
    }
    description.BitsPerComponent = CGImageGetBitsPerComponent(cgImage);

    return description;
}

Next, we need to modify the rendering engines so that they pass in the correct arguments to glTexImage2D after examining the API-agnostic texture description. Example 3 shows a private method that can be added to both rendering engines; it works under both ES 1.1 and 2.0, so add it to both renderers (you will also need to add its signature to the private: section of the class declaration).

Example 3. RenderingEngine::SetPngTexture()

private:
    void SetPngTexture(const string& name) const;
    // ...
    
void RenderingEngine::SetPngTexture(const string& name) const
{
    TextureDescription description = m_resourceManager->LoadPngImage(name);
    
    GLenum format;
    switch (description.Format) {
        case TextureFormatGray:      format = GL_LUMINANCE;       break;
        case TextureFormatGrayAlpha: format = GL_LUMINANCE_ALPHA; break;
        case TextureFormatRgb:       format = GL_RGB;             break;
        case TextureFormatRgba:      format = GL_RGBA;            break;
    }

    GLenum type;
    switch (description.BitsPerComponent) {
        case 8: type = GL_UNSIGNED_BYTE; break;
        case 4:
            if (format == GL_RGBA) {
                type = GL_UNSIGNED_SHORT_4_4_4_4;
                break;
            }
            // intentionally fall through
        default:
            assert(!"Unsupported format.");
    }

    void* data = m_resourceManager->GetImageData();
    ivec2 size = description.Size;
    glTexImage2D(GL_TEXTURE_2D, 0, format, size.x, size.y, 
                 0, format, type, data);
    m_resourceManager->UnloadImage();
}

Now you can remove the following snippet in the Initialize method (both rendering engines, but leave the call to glGenerateMipmap(GL_TEXTURE_2D) in the 2.0 renderer):

m_resourceManager->LoadPngImage("Grid16");
void* pixels = m_resourceManager->GetImageData();
ivec2 size = m_resourceManager->GetImageSize();
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, size.x, size.y, 0, 
GL_RGBA, GL_UNSIGNED_BYTE, pixels);
m_resourceManager->UnloadImage();

Replace it with a call to the new private method: