Building Custom Players with the Silverlight Media Framework

Streaming media has become ubiquitous on the Web

It seems like everyone—from news sites to social networks to your next-door neighbor—is involved in the online video experience. Due to this surge in popularity, most sites want to present high-quality video—and often high-quality bandwidth-aware video—to their consumers in a reliable and user-friendly manner.

A key element in the online media delivery experience is the player itself. The player is what the customer interacts with, and it drives every element ofthe user's online experience. With so much attention centered on the player, it's no surprise that modern, Web-based media players have become a great deal more complicated to implement than they were even a couple years ago. As a result, developers need a robust framework on which they can build their players.

The Silverlight Media Framework (SMF) is an open source project that was released by Microsoft at the 2009 Microsoft Professional Developers Conference. It is an extensible and highly scalable Silverlight video framework that directly answers the need for a stable core upon which developers and designers can create their own players. The code at the center of the Silverlight Media Framework has been refined based on lessons learned from the NBC Olympics and Sunday Night Football Web video projects.

This article will explain the basic elements of SMF, demonstrate how you can integrate SMF into your own player projects and walk you through a simple project that uses SMF to create a custom player experience. I'll show you how to use the logging, settings, and event-handling features of SMF. Finally, I'll create a player application that displays suggested videos for further viewing when the current video ends.

Getting Started with SMF

To get started, the first thing you'll want to do is download the framework from Codeplex (smf.codeplex.com). You also need to download the Smooth Streaming Player Development Kit (iis.net/ expand/smoothplayer) and reference it in any projects using SMF. The Smooth Streaming Player Development Kit is not part of SMF—it's a completely separate, closed-source component. However, SMF leverages a core set of functionality from the kit, in particular the video player itself. As of the writing of this article, the Smooth Streaming Player Development Kit is in beta 2.

Figure 1 The Silverlight Media Framework Assemblies

SMF consists ofa number ofMicrosoft .NET assemblies (as shown in Figure 1), each a different functional part ofthe overall framework.

The core assembly is Microsoft.SilverlightMediaFramework.dll, which comprises a number of utility classes and types referenced throughout the rest of the framework. When using any aspect of SMF, you must also reference the Microsoft.SilverlightMedia-Framework.dll assembly.

The Microsoft.SilverlightMediaFramework.Data namespace provides helper classes for consuming data external to the player and for encapsulating data within the player. The data can be general, with any form, but it can also be settings information for the player itself. There's another namespace, Microsoft.Silverlight-MediaFramework.Data.Settings, for types representing and dealing with player settings.

Apart from data used for settings, the type within the Data namespace you'll most likely interact with is the out-of-stream DataClient class, which can retrieve data from an external source. You reference this assembly if you want to download and use data external to the player.

The SMF player includes the robust Microsoft.Silverlight-MediaFramework.Logging framework that uses a callback-style paradigm in which writing to the logging infrastructure raises events. You register your own callback methods with the logging system, and these callbacks carry out additional operations once invoked—such as posting information to a Web service or displaying information to a text box. You reference this assembly if you wish to use the built-in logging facilities of SMF.

The Microsoft.SilverlightMediaFramework.Player assembly implements the player itself. It also provides a number of controls the player relies on, such as a scrubber, volume control and timeline markers. The default SMF player is sleek and clean, a great starting point for any project requiring a Silverlight player. However, central to all controls defined within SMF is the notion of control templating, so each control can be themed by using tools such as Expression Blend or Visual Studio.

Building and Referencing SMF

SMF downloads as a single .zip file in which you'll find a solution file, a project for each output library, and test projects for running and verifying the player itself.

SMF relies on the Smooth Streaming Player Development Kit. To reference the kit, move the Smooth Streaming assembly (Microsoft.Web.Media.SmoothStreaming.dll) into the \Lib folder of the SMF project.

Next, open the SMF solution in Visual Studio and build it, creating all the assemblies needed to leverage the framework. To verify that everything executes as expected, press F5 to begin debugging. The solution will build and the Microsoft.SilverlightMediaFramework. Test.Web target will execute, presenting you with the default SMF player streaming a "Big Buck Bunny" video (see Figure 2). Note how complete the default player already is, with a position element for scrubbing, play/stop/pause buttons, volume controls, full screen controls and so forth.

Figure 2 The SMF Player and the Big Buck Bunny Video

The next step is to create your own separate Silverlight project and leverage SMF from within it. In Visual Studio click File | New | Project | Silverlight Application. Call the solution SMFPlayerTest and click OK. A modal dialog will pop up, asking whether you wish to host the Silverlight application in a new Web site. Click OK and you'll see a basic Silverlight application solution consisting of two projects, SMFPlayerTest and SMFPlayerTest.Web.

The final step is to reference the Smooth Streaming Player Development Kit and SMF assemblies from your newly created project. Copy the output SMF assemblies and Smooth Streaming Player Development Kit from the SMF solutions Debug folder and paste them into your new project as shown in Figure 3. Your new solution now includes all the assembly references required to take full advantage of the SMF.

Figure 3 Referencing the Required Assemblies

Displaying the Player

To begin using the SMF, include the SMF players namespace within your MainPage.xaml page. This ensures that all references resolve properly:

Now insert player's XAML within the page's LayoutRoot Grid control.

Pressing F5 will launch the project and bring up the SMF player. However, because the player hasn't been told what to play, it does nothing. All you get is a player with no content to play.

SMF uses SmoothStreamingMediaElement (from the Smooth Streaming Player Development Kit) to play video. From SmoothStreamingMediaElement, SMF inherits its own player, called CoreSmoothStreamingMediaElement. This object is required if you want the player to stream content. Be sure to set the Smooth-StreamingSource property to a valid smooth streaming media URL:

As mentioned earlier, Microsoft provides the "Big Buck Bunny" sample video stream, which developers can use to test Silverlight projects. To use this test stream, set the SmoothStreamingSource property on the CoreSmoothStreamingMediaElement to:

Once again, press F5 to build and run the project. The browser will execute with the same player as before, but this time the "Big Buck Bunny" video will begin streaming moments after the player has fully loaded. If your task was to create a basic Silverlight player to stream content, you've done it.

However, the SMF offers quite a bit more than we've seen thus far. Let's add some basic logging.

Logging in the Player

Logging in SMF is simple—whenever an event is logged, it raises a Log-Received event. You register an event handler for this event, and thereby receive a notification for each logging event as it's raised. What you do with the notification is up to you; you can display it in a new window within the player, filter the events and notify a Web service whenever a certain event gets raised, or do whatever is necessary for your scenario.

The LogReceived event is statically defined on the Logger class itself(defined within Microsoft.SilverlightMediaFramework.Logg-ing.dll), so it's possible to register for logging events anywhere within the project. Here's an example of registering for and defining the event handler within the MainPage.xaml file of the SMFPlayerTest project:

SMF raises quite a few events out of the box. To see them, create a breakpoint within the Logger_LogReceived method and run the player once again in Debug mode. Almost immediately your breakpoint will get hit, allowing you to step through the method's parameters and see the information passed to it.

Log event data is packaged within a special messaging object whose type must inherit from an abstract class named Log. This abstract Log type has three properties: Sender, Message and TimeStamp. Sender references the object that raised the event. Message is an object of type System.String that holds the text for the logging event. TimeStamp simply holds the date and time at which the logging object was first instantiated. The SimpleEventArgs<> object passed as the second parameter to your event handler holds a reference to the Log object through its Result property.

To raise a log event, all that's required is to instantiate a type that inherits from the Log base class, then pass this type to the statically defined Log method on the Logger type. The framework supplies a DebugLog class that already inherits from the Log base type. What's special about the DebugLog type, however, is that if the libraries being referenced by your Silverlight project were created under a Debug build of the SMF, passing a DebugLog type to the SMF logging framework will raise a corresponding logging event (and therefore invoke your event handlers). On the other hand, a Release build of the SMF will ignore any call to the Log method that gets passed the DebugLog class. In short, ifyou have debugging statements you only want to use Debug builds, with the Debug-Log object as the log event argument; otherwise you will need to construct your own type that inherits from the abstract Log type.

Here's an example that raises a Listening event through the SMF event system by instantiating a DebugLog object and passing it to the Logger's static Log method (be sure your Smooth Streaming Player Development Kit files were built under Debug settings):

Inheriting from the Player Class

Although logging is a central feature of the player, the SMF playback features are only accessible when you inherit from and begin extending the SMF Player type itself.

To see how this works, you need to create a new class called SMF-Player that inherits from the Player type.

The new SMFPlayer class looks like this:

Every FrameworkElement type (such as Player in SMF) has an OnApplyTemplate method that is called whenever the Apply-Template event is raised. This method often serves as a useful starting point when initializing a FrameworkElement type.

In this case, I override the default OnApplyTemplate method from within the new SMFPlayer class. To demonstrate that the new SMFPlayer type is executed instead of the default Player type, you can set a breakpoint within the override. When you debug the player in Visual Studio, this breakpoint will be enountered when Silverlight executes the SMFPlayer.

Now update the MainPage.xaml file to use the new player class. First, include the players namespace in the list of namespaces already referenced (just as you did the player namespace earlier):

Then simply update the Player tags within the XAML to use SMFPlayer instead of Player:

Next, instantiate a DebugLog class and pass it to the Log method as shown earlier. Doing so

To listen specifically for this event from within the event handler, filter the Message property of the DebugLog object itself. In this example, look for any message that contains "OnApplyTemplate":

Using Settings Data

A mature framework for dealing with settings is crucial to most large-scale software projects. The code for handling settings in SMF is built on the Microsoft.SilverlightMediaFramework.Data.dll assembly, which allows you to download generic, external data. The settings layer of SMF uses this infrastructure to reach out and download a specially formatted XML settings file hosted on a Web server. Once the settings data has been successfully downloaded and read, the SMF settings layer encapsulates it with a SettingsBase object whose methods are then used to retrieve the settings values.

The SettingsBase class, as the name suggests, serves as a base for a more specific class that can provide strongly typed access to your settings values. Here's an example of a class that inherits from SettingsBase. It has two properties, one for retrieving a video player source URL and another for retrieving a Boolean value that indicates whether the video player should start automatically or wait for the viewer to press the play button:

The property methods use functions implemented by the SettingsBase class to inspect the underlying collection of settings name/value pairs loaded into the type (through a mechanism discussed shortly). This provides a type-safe and IntelliSense-friendly method of retrieving settings information.

Now create a new XML file in the SMFPlayerTest.Web project, name it SMFPlayerSettings.xml, and add the following to it:

Next, create a SettingsClient object into which you'll load the settings XML. SettingsClient takes a URI pointing to the settings file:

The process of retrieving the settings data is asynchronous, so a callback method must be assigned to the RequestCompleted method on SettingsClient:

The last step is to invoke the parameterless Fetch method on the SettingsClient object. When the data is retrieved, the settings-Getter_RequestCompleted event handler will be invoked and a SettingsBase object will be passed to it:

The SettingsBase object passed to the settingsGetter_Request-Completed method is loaded with the name/value pairs parsed for you by the underlying framework from the file SMFPlayerSettings.xml.

In order to load this data into your SMFPlayerTestSettings object, you simply call the Merge method, which merges settings information from one SettingsBase-derived object with that of another:

You no longer have to hard-code the AutoPlay and Smooth-StreamingSource properties on the CoreSmoothStreamingMediaElement within the page XAML, because the player settings are being downloaded from within the OnApplyTemplate method. This is all you need for the player XAML:

When you run the player, all the settings data will load, the callback will load the values into the players media element, and the video will begin to stream just as it did before.

Extending the SMF Player

On many popular video sites, when video playback has completed, you see a list of similar or recommended videos. To illustrate how easy it is to extend the SMF player, let's walk through the steps to build a similar suggested-viewing feature into the SMFPlayerTest project.

Start by adding an x:Name attribute to the Player element in the MainPage.xaml file:

This makes it easier to refer to the SMFPlayer object by name within both Visual Studio and Expression Blend.

Now, right-click on the MainPage.xaml file in Solution Explorer and select Open in Expression Blend. Expression Blend 3 will launch and display a design interface to the SMF player. In the Objects and Timeline section, you'll find a myPlayer node in the tree of visual objects that corresponds to the name given to the SMFPlayer object previously. The goal is to create a template for SMFPlayer, then to add three Suggestion buttons to the template. By using a template in Expression Blend, you can add, edit or remove controls built into the player itself.

To create a template, right-click myPlayer in the Objects and Timeline window and select Edit Template | Edit a Copy. A Create Style Resource dialog will be displayed, click OK. To insert the three buttons on top ofthe video player, double-click the button icon in the Tools window for each button you want to add. Three buttons should now be visible in the tree of controls that make up the player template (see Figure 4).

Figure 4 Button Controls Added to the Control Tree

Select all three buttons in the tree, go to the properties window for the controls and set the horizontal and vertical alignment to be centered (see Figure 5), thus aligning the buttons down the center and middle of the video player.

Figure 5 Setting Button Control Alignment

The buttons are the default size and lie on top of each other. Set the width of each button to 400, and the height to 75. Next, adjust the margins so that one button has a 175-pixel offset from the bottom, another 175-pixel offset from the top and the last has no margin offsets at all. The end result will look like Figure 6.

Figure 6 The Centered Buttons in Expression Blend

To verify the buttons have been properly placed on the player, save all open files in Expression Blend and return to Visual Studio. Visual Studio may prompt you to reload documents that were changed by Expression Blend. If so, click OK. From within Visual Studio, press F5 to relaunch the SMF player in Debug mode. The player should now appear with three buttons aligned down the center of the video screen as shown in Figure 7.

Figure 7 The Centered Buttons in the SMF Player

Hooking up Event Handlers

Event handlers must now be associated with the buttons. To reference the buttons from code, you need to assign names to them, which you do via the Name text box in the Properties tab. For simplicity, name the buttons Button1, Button2 and Button3. When you're done, the Objects and Timeline window should update and display the button names adjacent to the button icons in the visual tree.

Within the Properties tab for each button you'll find an Events button that's used to assign event handlers for a visual component. Select one of the buttons, click the Event button within the Properties tab, and double-click the Click text box to auto-generate an event handler within the MainPage.xaml.cs. The properties window for each button will now have an event handler assigned to its Click event (see Figure 8), and the MainPage.xaml.cs file will have event handlers assigned to each buttons Click event.

Figure 8 Setting the Event Handler

You can now debug the player. Clicking any of the buttons on the screen will raise a Click event, which is now handled by the auto-generated methods within MainPage.xaml.cs.

Suggested Videos

Now let's use these buttons to enable the suggested video feature. The following XML will represent the suggestions:

The value of the Url attribute will specify the video the player is to load when the button is clicked, and the DisplayName attribute is the text to be written on the button. Save this file with the name Suggestions.xml in the SMFPlayerTest.Web project.

The DataClient type (within the Microsoft.Silverlight-MediaFramework.Data namespace) will be used to download the XML document and to represent the content in a type-safe manner. To represent each Suggestion read from the XML file in a strongly typed fashion, create a class called SMFPlayerTestSuggestion in your Silverlight project:

DataClient, like SettingsBase, is intended to be derived from by a class that enables a strongly typed representation of the data from the XML content (in this case, an array of SMFPlayerTest-Suggestion objects).

Create another class file within the SMFPlayerTest project called SMFPlayerTestDataClient:

 SMFPlayerTestDataClient inherits from DataClient and sets its template argument to an array of SMFPlayerTestSuggestion types. The DataClient base class provides all the necessary asynchronous networking logic to go online and download the external XML file. Once the content has been downloaded, however, the DataClient base will invoke OnRequestCompleted and expect all processing of the XML data to take place then. In other words, the DataClient base class downloads the content, but the implementer is responsible for doing something with it.

Here's a more complete implementation of OnRequestCompleted:

For the sake of simplicity, I've used LINQ to XML in this implementation to parse the required elements and attributes in the XML. Once the DisplayName and Url attribute values from each Suggestion node have been retrieved, a SMFPlayerTestSuggestion object is instantiated and the values are assigned.

The final step is the invocation of OnFetchCompleted event. Outside consumers of SMFPlayerTestDataClient may register event handlers to the FetchCompleted event to be notified when the suggested video data has been downloaded. Because OnRequestCompleted has packaged the XML data in a type-safe manner, each event handler will receive a handy array of SMFPlayerTestSuggestion objects, one for each Suggestion element in the XML document the DataClient base class downloaded.

The underlying DataClient provides a method called Fetch that, once invoked, begins the process of asynchronously downloading content. To begin downloading the suggestion data when the video has ended, attach an event handler called mediaElement_MediaEnded to the MediaEnded event on the MediaElement object:

The mediaElement_MediaEnded method creates an instance of the SMFPlayerTestDataClient type, assigns another event handler to the FetchCompleted event, and then invokes Fetch to begin the download process. The FetchComplet-ed handler will be invoked by the call to OnFetchCompleted implemented previously within OnRequestCom-pleted (which is invoked by the Data-Client base type once the content has downloaded).

The implementation of suggestion_ FetchCompleted, registered within mediaElement_MediaEnded, takes the strongly typed array of Suggestion data and assigns one Suggestion to each button:

GetTemplateChild, a method on the underlying FrameworkElement type, gets a reference to each of the buttons defined in the MainPage XAML. For each button, the display text is assigned to the Content property, and the URI is assigned to the Tag property. Each button's click event handler can then pull the URI from the Tag property and assign the URL to the player's Media-Element to play the stream:

Showing the Buttons

The final step is to hide the buttons until the currently streaming video has ended, at which point the buttons become visible. Once a user clicks a button, the buttons are hidden again.

Within Visual Studio, edit the SMFPlayer class by decorating it with two TemplateVisualState attributes:

TemplateVisualState is a fascinatingly powerful attribute that defines visual states under which an object may exist. Once a visual state becomes active, Silverlight will update properties of visual elements belonging to the class as instructed—such as the visibility of a child button control.

To set the current visual state, use the static GoToState method of the VisualStateManager class (a native Silverlight type). The GroupName property ofthe TemplateVisualState groups like states together, whereas the Name property of the TemplateVisualState specifies the individual state.

Return to Expression Blend. In the myPlayer template, click myPlayer directly above the designer window, then click Edit Template | Edit Current. Click the States tab and scroll down SuggestionStates as shown in Figure 9.

Figure 9 Visual States for SuggestionStates

The two SuggestionStates created by the attributes appear as Hide and Show. Ifyou click on Hide, a red circle appears just to the left, indicating Expression Blend is recording any property changes made within the designer. Expression Blend continues to record property changes until Hide is clicked again, which causes the red recording circle to disappear.

With Expression Blend actively recording for the Hide visual state, set the buttons to Collapsed. Select all three buttons under the Objects and Timeline window and choose Collapsed as their Visibility in the Properties tab. Stop recording for the Hide visual state by clicking the Hide button once again. Now click Show so that a red circle appears to the left of the Show visual state. This time explicitly record Visible as the visibility status by clicking the Advanced Property Options button just to the right of the Visibility drop-down and selecting Record Current Value. Save all open documents and once again return to Visual Studio.

The native Silverlight class, VisualStateManager, is used to explicitly set a currently active visual state. From within the OnApplyTemplate method of the player, set Hide as the currently active visual state:

Within suggestion_FetchCompleted, set Show as the currently active state to display the buttons once the stream has ended and the Suggestion data download has completed:

To hide the buttons once a button is clicked (or the original stream is replayed), create a new event handler for the MediaElement's MediaOpened event, and set the visual state to Hide.

Launch and debug the player one final time. You'll see the buttons are invisible until the very end of the video, at which point they become visible. Clicking a button navigates the player to whatever URL was specified in the button's corresponding Suggestion setting.

The SMF project space on Codeplex gives you access to the code base, documentation, discussions and the issue tracker. Take a look and contribute what you can. The more creative minds applied to the project, the better the result for everyone.

BEN RUSHis an 18-year veteran software developer specializing in the Microsoft .NET Framework and related Microsoft technologies. He enjoys smart code and fast bike rides.