Programming the Mobile Web : Multimedia and Streaming

2/6/2011 4:13:21 PM
Serving audio and video content to mobile devices is very important for many portals and content providers.

We can provide multimedia content in three formats:

  • Downloadable content

  • On-demand streaming content

  • Live streaming content

For downloadable content, there are many formats and codecs that we can use. Not all devices support all of them, so we should check the documentation for our target devices or use WURFL properties to check for support on the fly.

Video and audio files come with two technologies: a container format and one or more codecs inside. The most compatible container formats for mobile devices are 3GP and 3GP2, created by the 3GPP organization (formed with 3G companies). They are very similar to the MPEG-4 (MP4) format, so many devices support both.

There are also devices with support for MPEG, Flash Video, AVI, Real Audio, Real Video, MOV, and Windows Media Audio/Video containers. Most devices support the H.263 and H.264 codecs.

In the audio world, the most standard formats today are MP3 and MIDI, but some devices also support MP4, Real Audio, WAV, AAC, and other audio formats.

1. Delivering Multimedia Content

If we want to deliver multimedia content, we need to first look at the Accept header or WURFL properties to determine whether the device supports the format we’re using. If so, we can use the delivery methods defined earlier. Depending on the device’s capabilities, it may download the entire file before playing it or it may try to play it while downloading using HTTP streaming techniques.


Multimedia files are generally large. If we deliver noncompatible files, the user will be paying for non-useful traffic and will not be happy with us.

For the best HTTP streaming technique, we need the server to support partial downloads. If your server doesn’t support it, there is a great PHP script available at

The direct download technique works in almost every compatible phone: Symbian, iPhone, Android, Nokia Series 40, etc.

2. Embedding Audio and Video

Not all devices support embedded multimedia in web pages. On some devices, the display and CPU constraints make this feature impossible.

2.1. Flash Video

Flash Lite (from version) 3.0 and Flash Player for mobile devices support Flash Video format (FLV). For the compatible devices , like Symbian devices from 3rd edition FP1, you can embed any Flash content, although it is better if you compile your SWFs for Flash Player 8. In fact, the YouTube desktop version works well on Flash Lite 3–enabled devices (it’s a bit slow on some devices, but it works).

2.2. Object embedding

You can use an object tag to include a video in a web page:

<object data="video.mp4" type="video/mp4" width="300" height="300" />

Safari will show an image with a play button. When the user presses play, the video will open in a full-screen QuickTime Player window instead of being embedded in the browser.

An alternative is the embed tag, preferred for iOS 1 and 2.X:

<embed src="poster.jpg" href="video.m4v" type="video/x-m4v" />

The object tag also works on Symbian devices.

2.3. HTML 5

Although at the time of this writing they are only compatible with some mobile browsers, such as Safari on iOS 3.0 and later, the audio and video tags defined in HTML 5 can also be used to embed multimedia content. The behavior is similar to using the object tag. The video element supports the usage of a source child tag that allows us to define different media files in different codecs and bit rates for best compatibility. The normal syntax is:

<video src="url"
height="240" />

Safari supports the formats 3GP, MOV, and MPEG-4, and the H.264 and AAC-LC codecs.

2.4. Reference movies for iPhone

Safari on iOS also supports “reference movies,” created with QuickTime Pro or a similar tool. A reference movie provides a list of movie URLs with different bit rates (for example, for WiFi, 3G, or EDGE), so QuickTime can select the correct one for the device. In our embed or video tag, we point to this new file.


At you can find an Objective-C Mac tool provided by Apple to generate iPhone reference movie files from the command line.

3. Streaming

Streaming audio or video is a difficult solution if we want to be compatible with all devices. Different platforms support different streaming technologies.

Some devices, including Symbian, Windows Mobile, and BlackBerry devices, support the Real Time Streaming Protocol (RTSP). When a link with this protocol is used (e.g., rtsp://server/content), the default media player—Real Player, Windows Media, etc.—is opened. The content can be a file to be streamed (a prerecorded audio or video file) or a live event (radio or TV show, sports event, etc.).


RTSP is an open standard for establishing and controlling sessions between two points. Most platforms use the Real-time Transport Protocol (RTP) for media streaming (audio and video) and can deliver either live events or on-demand multimedia content.

We should expect more Flash streaming services for mobile devices when the full Flash Player is available for mobile devices. Today, Flash Lite 3.0 devices should work with Adobe Flash Media Server or the Red5 open source alternative (

For general audio and video streaming there are also other streaming solutions, like the commercial Helix Media Delivery Platform ( and QuickTime Streaming Server (

Apple also maintains an open source alternative called Darwin Streaming Server. (

A streaming server uses TCP or UDP but generally does not use HTTP. Some proxies may have problems redirecting the server’s TCP or UDP packages, so HTTP has some advantages. However, it also has more overhead than TCP or UDP and it is not prepared for live streaming events. Some mobile devices that use HTTP have to download the entire file before playing it. Other devices will start playing the file while downloading it.

3.1. HTTP Live Streaming

Apple has created a new way to deliver live streaming using HTTP, called HTTP Live Streaming, which it has presented to the IETF as a proposed Internet standard. It is supported by iOS 3.0 and allows the transmission of live events using the same HTTP protocol we know. In fact, this is the only streaming solution that works on the iPhone.

Implementing HTTP Live Streaming requires some changes on the web server end. The simplified explanation of the protocol is that on the server, the live stream is buffered in little packages sent to the client. It’s like transmitting a live radio show by sending a series of 10-second MP3s.


The well-known Akamai application acceleration service provider offers live streaming services for the iPhone from Influxis ( also offers mobile streaming services as a shared hosting solution, for the iPhone/iPad and BlackBerry and Android devices.

HTTP Live Streaming supports the H.264 codec for video and AAC or MP3 for live audio streaming, as well as a bandwidth switcher for different qualities. However, the best feature is that it passes any firewall or proxy because it is HTTP-based.


Carson McDonald has developed an open source Ruby and C solution for doing HTTP Live Streaming from a server, and even using Amazon S3 services. You can check out his blog at or read more about the project at

Apple offers a prerelease toolkit for this solution called HTTP Live Streaming Tools. You need to be part of the Apple Developer Connection to download it. The open source IceCast Server also supports iPhone streaming, from version 2.3.2. You can find more information at


For Android, iPhone, and Flash Lite 3 or Flash Player 10 devices, we can safely upload video content to YouTube and embed it in our websites. The devices will render this content properly using the internal Flash Player or a native YouTube application.

  •  Mobile Application Security : BlackBerry Security - Development and Security Testing
  •  Mobile Application Security : BlackBerry Security - Introduction to Platform
  •  Windows Phone 7 Development : Using a WebBrowser Control to Display Dynamic Content
  •  Windows Phone 7 Development : Using a WebBrowser Control to Display Local HTML Content
  •  Windows Mobile Security - Networking
  •  Windows Mobile Security - Local Data Storage
  •  Windows Mobile Security - Permissions and User Controls
  •  Windows Phone 7 Development : Using a WebBrowser Control to Display Web Content
  •  Windows Phone 7 Development : Adding a WebBrowser Control
  •  Programming the Mobile Web : Content Delivery (part 3)
  •  Programming the Mobile Web : Content Delivery (part 2) - File Delivery
  •  Programming the Mobile Web : Content Delivery (part 1) - Defining MIME Types
  •  iPhone Application Development : Using Switches, Segmented Controls, and Web Views (part 3)
  •  iPhone Application Development : Using Switches, Segmented Controls, and Web Views (part 2)
  •  iPhone Application Development : Using Switches, Segmented Controls, and Web Views (part 1)
  •  iPhone Application Development : Using Advanced Interface Objects and Views - User Input and Output
  •  Windows Phone 7 Development : Wiring Up Events to an Application Bar ( part 2)
  •  Windows Phone 7 Development : Wiring Up Events to an Application Bar ( part 1) - Reacting to Add Button Events
  •  Adding an Application Bar to a Windows Phone 7 Application (part 3) - Adding an Application Bar Using Managed Code
  •  Adding an Application Bar to a Windows Phone 7 Application (part 2) - Adding a Local Application Bar Using XAML & Adding Menu Items
    Top 10
    Nikon 1 J2 With Stylish Design And Dependable Image And Video Quality
    Canon Powershot D20 - Super-Durable Waterproof Camera
    Fujifilm Finepix F800EXR – Another Excellent EXR
    Sony NEX-6 – The Best Compact Camera
    Teufel Cubycon 2 – An Excellent All-In-One For Films
    Dell S2740L - A Beautifully Crafted 27-inch IPS Monitor
    Philips 55PFL6007T With Fantastic Picture Quality
    Philips Gioco 278G4 – An Excellent 27-inch Screen
    Sony VPL-HW50ES – Sony’s Best Home Cinema Projector
    Windows Vista : Installing and Running Applications - Launching Applications
    Most View
    Bamboo Splash - Powerful Specs And Friendly Interface
    Powered By Windows (Part 2) - Toshiba Satellite U840 Series, Philips E248C3 MODA Lightframe Monitor & HP Envy Spectre 14
    MSI X79A-GD65 8D - Power without the Cost
    Canon EOS M With Wonderful Touchscreen Interface (Part 1)
    Windows Server 2003 : Building an Active Directory Structure (part 1) - The First Domain
    Personalize Your iPhone Case
    Speed ​​up browsing with a faster DNS
    Using and Configuring Public Folder Sharing
    Extending the Real-Time Communications Functionality of Exchange Server 2007 : Installing OCS 2007 (part 1)
    Google, privacy & you (Part 1)
    iPhone Application Development : Making Multivalue Choices with Pickers - Understanding Pickers
    Microsoft Surface With Windows RT - Truly A Unique Tablet
    Network Configuration & Troubleshooting (Part 1)
    Panasonic Lumix GH3 – The Fastest Touchscreen-Camera (Part 2)
    Programming Microsoft SQL Server 2005 : FOR XML Commands (part 3) - OPENXML Enhancements in SQL Server 2005
    Exchange Server 2010 : Track Exchange Performance (part 2) - Test the Performance Limitations in a Lab
    Extra Network Hardware Round-Up (Part 2) - NAS Drives, Media Center Extenders & Games Consoles
    Windows Server 2003 : Planning a Host Name Resolution Strategy - Understanding Name Resolution Requirements
    Google’s Data Liberation Front (Part 2)
    Datacolor SpyderLensCal (Part 1)