Building Out Of Browser Silverlight Applications - Using COM Interoperability and File System Access

1. Problem

You need to interoperate with COM based APIs and access the file system from an out of browser Silverlight application.

2. Solution

Use the built-in support for COM interoperability and file system access from an out of browser Silverlight application running with elevated trust.

3. How It Works

3.1. COM Interoperability

A large number of system services and platform features on Microsoft Windows are exposed through an integration technology called COM. Additionally, many applications, both from Microsoft (such as Microsoft Office), as well as a multitude of 3^rd party applications for Windows also enable extensibility and programmability by exposing COM based APIs.

Silverlight 4 introduces the ability to interoperate with some of these system services and application API's through a COM Interoperability layer built into the Silverlight runtime. Before we progress in describing how it all works, there are three very important points to be noted here:

• COM is a technology available on Microsoft Windows only. So if you build a Silverlight application with features that take advantage of COM Interop, those features of your application will only work when it runs on Windows.

• COM Interop through Silverlight 4 is only available when the application is running out of browser with elevated trust.

• Not all COM components can be access through the Silverlight COM Interop feature. Only COM objects that support COM Automation are accessible through Silverlight. COM Automation capable COM objects implement a COM interface named Idispatch (or IDispatchEx) and are scriptable through scripting languages such as JavaScript.

NOTE

A detailed treatment of COM is out of scope for this book. For more details on COM, you can refer to msdn.microsoft.com/en-us/library/ms680573(VS.85).aspx. For more details on COM Automation, you can refer to msdn.microsoft.com/en-us/library/ms221375.aspx.

3.2. Instantiating a COM object

Silverlight 4 exposes the COM Interoperability mechanism through the AutomationFactory class in the System.Runtime.InteropServices.Automation namespace. The static CreateObject() method defined on AutomationFactory accepts the ProgID of the COM object you are trying to instantiate and returns the newly instantiated COM objected as a dynamic type instance on success.

The dynamic type is newly introduced in .Net 4 and Silverlight 4 runtime offers it as well. A variable of type dynamic bypasses static (compile time) type checking. This makes the dynamic type especially suited for representing COM types, as due to the implementation differences between native APIs like COM and the common language runtime, the exact signature of a COM type is not known while compiling the managed code, but only at runtime. Keep in mind that while authoring code using a dynamic type, you can call any method or access any property on the variable of type dynamic without the compiler checking whether the member actually exists on the underlying implementing COM object. If the call is erroneous, your application fails at runtime. Also keep in mind that, because of this lack of type description information during authoring, Visual Studio offers no IntelliSense on dynamic typed variables. Consequently, having the API documentation available for any COM API that you may want to access is very important for authoring correct Silverlight-based COM Interop code.

Once you acquire the returned object from the AutomationFactory.CreateObject(), you can call methods and access properties on it just like you would on any managed object, as long as the members are implemented by the underlying COM object. Remember that these properties and methods will also use dynamic types as return values, so feel free to cast them to appropriate CLR types, and Silverlight will do the conversion for you.

You can also use AutomationFactory.GetObject() to acquire a reference to a COM object. While CreateObject() will load the COM server containing the COM object you requested and start the containing application if it is an out of process executable, GetObject() expects the COM server to be already running. For example, calling CreateObject() to acquire a handle to a COM object defined in the Microsoft Excel COM object model would cause Excel to start up, while GetObject() can be called if you know Excel to already be running. The snippet below shows an example of creating a COM object, accessing a property and calling a method on it:

dynamic devManager = AutomationFactory.CreateObject("WIA.DeviceManager");
dynamic DeviceInfoCollection = devManager.DeviceInfos;
devManager.RegisterEvent("{A28BBADE-64B6-11D2-A231-00C04FA31809}");

Note that AutomationFactory also exposes a property named IsAvailable that indicates if the COM automation feature is available to your application at runtime. Before you attempt to create your first COM object in your application, you should check the value of the property and ensure that COM Interop is available to you in the current environment.

3.3. Handling a COM event

There are two ways to handle an event raised from the COM object in your Silverlight code. In the first approach, you can use the static GetEvent() method on the AutomationFactory object to search for a declared event by its string name. The first parameter to GetEvent() accepts the object returned from a CreateObject() or a GetObject() call, and the second parameter accepts the string name of the event you want to look for. If the event is found, an AutomationEvent instance is returned from GetEvent(). You can then add a handler to the AutomationEvent.EventRaised event to handle the occurrence of the COM event. The AutomationEventArgs type parameter passed into your event handler implementation exposes an Arguments property that contains any event parameters passed in from COM as a collection of objects. The snippet below shows an example of searching for an event named OnEvent, registering a handler, and accessing the event arguments inside the handler:

dynamic devManager = AutomationFactory.CreateObject("WIA.DeviceManager");
AutomationEvent evt = AutomationFactory.GetEvent(devManager, "OnEvent");
evt.EventRaised += new EventHandler<AutomationEventArgs>((s, e) =>
{
  string EventID = e.Arguments[0] as string;
  string DeviceID = e.Arguments[1] as string;
  string ItemID = e.Arguments[2] as string;
});

The other approach is to attach a handler to the event directly, using the dynamic instance of the COM object. You would, of course, need to declare a delegate that matches the event signature as documented for the COM object in question. The snippet below shows an example:

private delegate void OnEventHandler
  (string EventID, string DeviceID, string ItemID);

dynamic devManager = AutomationFactory.CreateObject("WIA.DeviceManager");
devManager.OnEvent += new OnEventHandler((evtID, DevID, ItemID) =>
{
  ...
});

3.4. File System Access

Although File System Access does not have anything to do with COM Interop, the code sample later in the recipe uses both features, and so we thought it prudent to cover this topic in the same recipe. Note that file system access as well requires that the application be running out of browser and with elevated trust.

Up until Silverlight 3, the OpenFileDialog and SaveFileDialog types have been the only ways for Silverlight applications to access any file information on the local file system, and only through user initiated code such as a button click. With a Silverlight 4 application running with elevated trust, you now have the option of using the classes in System.IO for a much deeper access to the file system. You can create new files and directories, enumerate the contents of a directory, get detailed file information, etc. A full discussion of the types in System.IO is out of scope for this recipe, but you can refer to msdn.microsoft.com/en-us/library/ms404278(VS.100).aspx for more details on the common I/O tasks that you can perform using these types.

Note that your file system access is limited to the MyDocuments, MyMusic, MyVideos, and MyPictures system folders and any sub folders and files within. To standardize the path to these folders, Silverlight defines a SpecialFolder enumeration within the Environment type where the above mentioned folders correspond to SpecialFolder.MyDocuments, SpecialFolder.MyMusic, ans so on. To make sure that your code remains cross-platform, you should always use Environment.GetFolderPath() and pass in one of these values to get the corresponding path for that platform. The snippet below shows an example of enumerating the sub-folders for the MyDocuments folder:

string MyDocumentsPath = Environment.GetFolderPath
    (Environment.SpecialFolder.MyDocuments);
IEnumerable<string> SubFolders = System.IO.Directory.
  EnumerateDirectories(MyDocumentsPath);

4. The Code

The code sample in this recipe shows an application for viewing photos from a digital camera that is connected to your computer. The application offers the option of saving the photos to your local file system.

4.1. Windows Image Acquisition

Windows Image Acquisition (WIA) is an API built into Windows that provides a standard mechanism for acquiring digital images from devices connected to your computer. These devices could be digital cameras that store captured images, or scanners that can scan digital images of documents. WIA exposes a COM Automation API and consequently is well suited for COM Interop-based access from within Silverlight. To ease its use from within our application and to facilitate property binding, you wrap the necessary parts of the WIA object model to create a strongly typed version. This wrapper code is found in a file named wiaom.cs in the sample project for this recipe. We describe parts of it here to illustrate the COM Interop aspects, but encourage you to look through the WIA Automation API at msdn.microsoft.com/en-us/library/ms630827(VS.85).aspx as well the code in wiaom.cs for more details on the wrapping approach.

The top level object in the WIA API is called the DeviceManager, and you wrap it in a CLR type named WIADeviceManager. Listing 1 shows the WIADeviceManager class and a few related classes in our wrapper object model.

Listing 1. WIA wrappers

public class WIAObject : INotifyPropertyChanged { //hold the COM native object protected dynamic WIASource { get; private set; } // public WIAObject(dynamic Source) { WIASource = Source; Validate(); } //validate the COM object protected virtual void Validate() { if (WIASource == null) throw new ArgumentNullException("Null source"); } #region INotifyPropertyChanged Members protected void RaisePropertyChanged(string PropName) { if (PropertyChanged != null) PropertyChanged(this, new PropertyChangedEventArgs(PropName)); } public event PropertyChangedEventHandler PropertyChanged; #endregion } public class WIADeviceManager : WIAObject { //raise a CLR event on handling a WIA Event public event EventHandler<WIAOnEventArgs> OnEvent; //delegate for handling DeviceManager.OnEvent private delegate void OnEventHandler (string EventID, string DeviceID, string ItemID); //get all the devices public IEnumerable<WIADeviceInfo> DeviceInfos { get { return (COMHelpers.COMIndexedPropertyToList(WIASource.DeviceInfos) as List<dynamic>).Select( (DeviceInfo) => new WIADeviceInfo(DeviceInfo)); } } //construct private WIADeviceManager(dynamic Source) : base((object)Source) { //attach handler to onEvent event Source.OnEvent += new OnEventHandler((eID, dID, iID) => { //raise our own OnEvent wrapper if (OnEvent != null) OnEvent(this, new WIAOnEventArgs() { EventID = eID, DeviceID = dID, ItemID = iID }); }); } //static factory method public static WIADeviceManager Create() { if (!AutomationFactory.IsAvailable) throw new InvalidOperationException ("COM Automation is not available"); return new WIADeviceManager(AutomationFactory.CreateObject("WIA.DeviceManager")); } //register for a WIA event public void RegisterEvents(string DeviceID, IEnumerable<string> Events) { Events.Any((ev) => { WIASource.RegisterEvent(ev, DeviceID); return false; }); } //unregister events public void UnregisterEvents(string DeviceID, IEnumerable<string> Events) { Events.Any((ev) => { WIASource.UnregisterEvent(ev, DeviceID); return false; }); } } public class COMHelpers { public static List<dynamic> COMIndexedPropertyToList( dynamic IndexedPropertyCollection) { List<dynamic> RetVal = null; if (RetVal == null) RetVal = new List<dynamic>(IndexedPropertyCollection.Count); else RetVal.Clear(); for (int i = 1; i <= IndexedPropertyCollection.Count; i++) RetVal.Add(IndexedPropertyCollection[i]); return RetVal; } } public class WIADeviceInfo : WIAObject { public WIADeviceInfo(dynamic Source) : base((object)Source) { } public WIADevice Connect() { WIADevice retval = new WIADevice(WIASource.Connect()); return retval; } public WIADeviceType DeviceType { get { return (WIADeviceType)WIASource.Type; } } public string DeviceID { get { return (string)WIASource.DeviceID; } } public IEnumerable<WIAProperty> Properties { get { return (COMHelpers.COMIndexedPropertyToList(WIASource.Properties) as List<dynamic>).Select((Prop) => new WIAProperty(Prop)); } } }

The WIADeviceManager.Create() is a wrapper factory method that creates an instance of the DeviceManager COM automation type and returns an instance of a WIADeviceManager. The WIAObject base class is used to hold the dynamic-typed COM automation object instance, implement property change notification, and validate it to make sure it is not null on creation. You check AutomationFactory.IsAvailable to make sure COM Automation is available in the environment you are running before you attempt to create the COM object.

The DeviceManager COM object exposes a COM indexed property named DeviceInfos that represents a collection of DeviceInfo COM objects, each entry representing a device connected to the machine and recognizable by WIA. You wrap this property using WIADeviceManager.DeviceInfos. In the property implementation, you use the COMIndexedPropertyToList() static method on the COMHelpers class. In COMIndexedPropertyToList() you enumerate a COM indexed property and return an CLR List instance populated with the same items.

The DeviceManager COM object also implements a RegisterEvent and an UnregisterEvent method that allows for callers to register and unregister for specific events defined in the WIA automation API as GUIDs. You define a RegisterEvents() wrapper method on WIADeviceManager that accepts a collection of event GUID's and registers each of them. Similarly, WIADeviceManager.UnregisterEvents() unregisters an already registered set of events. The first parameter to the RegisterEvent() and UnregisterEvent() methods are DeviceID's for the device in whose events you may be interested. You can pass the string "*" if you are interested in a specific event for all connected devices at the moment. The DeviceManager object also raises a COM event named OnEvent. You attach a handler to OnEvent in the WIADeviceManager constructor, and raise your own OnEvent implementation, passing in a new instance of WIAEventArgs type populated with the individual parameters obtained from the COM OnEvent handler.

There are several more WIA automation types that you have wrapped around in your wrapper object model, but we do not list all of them here for brevity. The purpose of this section was to show you a sample of COM Interop code, and the rest of the wrapper object model follows the exact same principles and techniques shown so far. For the rest of the recipe, whenever you encounter a type with a name starting with the string "WIA", know that it is one of your wrapper types defined in wiaom.cs wrapping around a corresponding automation type.

4.2. The Application Code

Figure 1 shows the application user interface.

Figure 1. The client application user interface

On the left is a list of devices connected to the computer that WIA recognizes. The right side shows the photo viewer with a pager control at the bottom to navigate through the photos on a connected camera, plus as a button that allows the user to save a specific photo to the disk. To test the application's functionality, you will need a digital camera with some images already stored in it. Run the application with the camera connected, or connect it to your PC with the application running. Then select the camera device; the right hand pane should let you navigate through the images on the camera as well as save them to your computer's local file system

Note that Figure 1 shows both a scanner and a camera connected to the computer. Recall from the earlier section that WIA recognizes scanners as source devices for digital images as well. In this sample, however, you will only deal with digital camera-specific features. Listing 2 shows relevant portions of the code for the MainPage.xaml from the application.

Listing 2. Portions of MainPage.xaml

<Grid x:Name="LayoutRoot" Background="Black"> <Grid.ColumnDefinitions> <ColumnDefinition Width="0.294*"/> <ColumnDefinition Width="0.706*"/> </Grid.ColumnDefinitions> <Border BorderBrush="White" BorderThickness="1" Margin="2"> <ListBox Margin="0" x:Name="lbxDevices" ItemContainerStyle="{StaticResource styleImagingDeviceListBoxItem}" SelectionChanged="lbxDevices_SelectionChanged" Background="Black"/> </Border> <Border BorderBrush="White" BorderThickness="1" Margin="2" Grid.Column="1"> <ContentControl x:Name="cntctlDataPane" HorizontalAlignment="Stretch" VerticalAlignment="Stretch" HorizontalContentAlignment="Stretch" VerticalContentAlignment="Stretch" Margin="0" /> </Border> </Grid>

The ListBox named lbxDevices implements the device list shown in Figure 1 The ContentControl cntctldataPane is used to display the photos; we will discuss the mechanics of that later in the recipe. Listing 3 shows the codebehind for MainPage.xaml.

Listing 3. Codebehind for MainPage.xaml

Code View: Scroll / Show All public partial class MainPage : UserControl { //event handler delegate to handle the WIADeviceManager.OnEvent event public delegate void WIADeviceManageOnEventHandler(string EventID, string DeviceID, string ItemID); //WIADeviceManager singleton WIADeviceManager wiaDeviceManager = null; //the collection of all connected devices ObservableCollection<WIADevice> WIADevicesColl = new ObservableCollection<WIADevice>(); public MainPage() { InitializeComponent(); //create the DeviceManager wiaDeviceManager = WIADeviceManager.Create(); //register for connection and disconnection events for all devices //that WIA might recognize wiaDeviceManager.RegisterEvents("*", new string[]{WIAEventID.DeviceConnected,WIAEventID.DeviceDisconnected}); //attach event handler for OnEvent wiaDeviceManager.OnEvent += new EventHandler<WIAOnEventArgs>(wiaDeviceManager_OnEvent); //get and connect all the devices WIADevicesColl = new ObservableCollection<WIADevice>( wiaDeviceManager.DeviceInfos.Select((di) => di.Connect())); //set the device list lbxDevices.ItemsSource = WIADevicesColl; } void wiaDeviceManager_OnEvent(object sender, WIAOnEventArgs e) { //if a device just connected if (e.EventID == WIAEventID.DeviceConnected ) { //connect it WIADevice wiaDevice = wiaDeviceManager.DeviceInfos. Where((di) => di.DeviceID == e.DeviceID). Select((di) => di.Connect()).First(); //add to the bound collection WIADevicesColl.Add(wiaDevice); //if minimized - show notification if (Application.Current.MainWindow.WindowState == WindowState.Minimized) { DeviceConnectDisconnectNotification notfcontent = new DeviceConnectDisconnectNotification() { DataContext = wiaDevice, Connected = true}; NotificationWindow notfWindow = new NotificationWindow() { Height = 60, Width = 400, Content = notfcontent }; notfcontent.NotificationParent = notfWindow; notfWindow.Show(30000); } } //if device disconnected else if (e.EventID == WIAEventID.DeviceDisconnected && WIADevicesColl.Where((wiaDeviceInfo)=>wiaDeviceInfo.DeviceID == e.DeviceID).Count() > 0) { //remove it WIADevice wiaDevice = WIADevicesColl. Where((de) => de.DeviceID == e.DeviceID).First(); WIADevicesColl.Remove(wiaDevice); } } private void lbxDevices_SelectionChanged(object sender, SelectionChangedEventArgs e) { //get the selected device WIADevice Device = lbxDevices.SelectedValue as WIADevice; //display the content on the right pane DisplayCameraItems(Device); } private void DisplayCameraItems( WIADevice CameraDevice) { //create a new instance of the PhotoItems user control //and bind appropriate data cntctlDataPane.Content = new PhotoItems() { Device = CameraDevice, HorizontalAlignment = HorizontalAlignment.Stretch, VerticalAlignment = VerticalAlignment.Stretch }; } }

In the constructor of the MainPage class, you create an instance of the WIADeviceManager, which, as discussed in the previous section, instantiates the DeviceManager COM object using COM Interop. You also register to receive the OnEvent event for all devices whenever they connect to or disconnect from the machine, and then attach a handler to the OnEvent handler. As discussed before, WIA events are defined as GUIDs, and the WIAEventID type declares the WIA event's GUIDs as named variables. Lastly, you get back and bind the list of connected devices by calling the Connect() wrapper method on each WIADeviceInfo exposed through the WIADeviceManager.DeviceInfos collection property.

In the OnEvent event handler, you either remove a device from your bound device collection if you get a DeviceDisconnected event, or you add a new device if you get a DeviceConnected event. You also show a notification window on device connection; we will discuss this later in the recipe.

In the SelectionChanged event handler for the device ListBox, you acquire the selected WIADevice instance, and call DisplayCameraItems(), which in turn creates and displays a new instance of the PhotoItems user control, passing in the selected WIADevice instance. Listing 4 shows portions of the PhotoItems user control XAML.

Listing 4. PhotoItems user control XAML

<Grid x:Name="LayoutRoot" HorizontalAlignment="Stretch" VerticalAlignment="Stretch"> <Grid.RowDefinitions> <RowDefinition Height="*"/> <RowDefinition Height="Auto"/> </Grid.RowDefinitions> <Image Source= "{Binding Converter={StaticResource REF_WIAImageFileToBitmapConverter}}" Stretch="Uniform" x:Name="Photo" Grid.RowSpan="2"/> <StackPanel Orientation="Horizontal" Grid.Row="2" HorizontalAlignment="Stretch" VerticalAlignment="Stretch"> <Button x:Name="btnSave" Content="Save To Disk" Click="btnSave_Click" Width="90"/> <datacontrols:DataPager x:Name="PhotoPager" PageSize="1" DisplayMode="PreviousNext" IsTotalItemCountFixed="True" HorizontalAlignment="Stretch" VerticalAlignment="Stretch"/> </StackPanel> </Grid>

A DataPager control named PhotoPager is used to navigate through the images. An Image control named Photo is used to display the image, and the Button named btnSave allows the user to save an image to the disk when clicked. Listing 5 shows the relevant portions from the codebehind for PhotoItems.

Listing 5. PhotoItems user control codebehind

void PhotoItems_Loaded(object sender, RoutedEventArgs args) { PhotoPager.Source = new PagedCollectionView( Device.Items.Where((itm) => itm.Formats.Contains(WIAFormatID.JPEG))) { PageSize = 1 }; ShowPhoto(); PhotoPager.PageIndexChanged += new EventHandler<EventArgs>((s, e) => { ShowPhoto(); }); return; } private void ShowPhoto() { try { WIAImageFile img = ((PhotoPager.Source as PagedCollectionView). CurrentItem as WIAItem).Transfer(WIAFormatID.JPEG); if (img == null) Photo.DataContext = null; else Photo.DataContext = img.FileData; } catch (Exception Ex) { Photo.DataContext = null; } }

Each WIADevice instance exposes a property named Items, which is collection of WIAItem objects that wraps around the Items indexed property on the Device COM object. The items on the device (in this case, the device is a camera) can be images or other types of device specific data. In the Loaded event handler of the PhotoItem control, you query the Items property on the selected WIADevice, acquire only the JPEG images, wrap the filtered collection into a PagedCollectionView, and set it as the PhotoPager source. You also set the PageSize on the PhotoPager to one, ensuring that the user pages forward or backward for each image. You then call ShowPhoto(). In ShowPhoto(), you invoke the Transfer() COM method on the current WIAItem on the bound PagedCollectionView, which transfers the physical image as a WIAImageFile instance to the computer from the device. You then bind the WIAImageFile.FileData property to the Photo Image control shown in Listing 4. You call ShowPhoto() once every time a user pages through the DataPager to transfer and display other images. As you can also see in the XAML in Listing 4, a value converter named WIAImageFileToBitmapConverter is used in the image binding to convert the raw image data to a Silverlight bitmap. We do not list that code here, but you can find the converter in the PhotoItems.xaml.cs file in the sample project for this recipe.

4.3. Saving Images to the disk

Listing 6 shows the code to save an image to the disk.

Listing 6. Saving an image to the disk

private void btnSave_Click(object sender, RoutedEventArgs e) { //get the current WIAItem WIAItem itm = ((PhotoPager.Source as PagedCollectionView). CurrentItem as WIAItem); //get the raw data byte[] FileData = Photo.DataContext as byte[]; //get the filename from the Item Properties collection string filename = itm.Properties.ToList().Where((wiaprop) => wiaprop.Name == "Item Name").Select((wiaprop) => (string)wiaprop.Value).FirstOrDefault(); //create a file FileStream fs = File.Create( System.IO.Path.Combine( Environment.GetFolderPath(Environment.SpecialFolder.MyPictures), filename + ".jpeg")); //write fs.Write(FileData, 0, FileData.Length); //close file fs.Close(); }

As you can see in the code shown in bold, you first obtain a target path for the file by combining the path to the MyPicture special folder with the file name obtained from the Item Name WIA property on the current WIAItem. You then use the well-known FileStream API to write the data to the file on the file system, just as you would in a regular desktop .Net application.

4.4. Taskbar Notification

If you refer back to the code in Listing 4, you will note that when you handle a device connection event, you optionally display a notification window if the main application window is minimized. Figure 2 shows this notification window in action.

Figure 2. Notification window showing a device connected event

Silverlight 4 introduces the NotificationWindow type that enables this scenario. The NotificationWindow type exposes a Content property that you can set to any content in your application, and it exposes a Show() method that can be invoked with a timeout parameter in milliseconds so that the notification remains displayed for the specified timeout. As you can see in Listing 4, you create your NotificationWindow instance, set it to some content that displays the device connection notification, and then show it for 30 seconds.