BizTalk 2006 : Managing Exceptions in Orchestrations (part 1) - The Exception Management Challenge

8/7/2012 4:02:12 PM
Designing consistent reusable patterns for exception management within any development project is a fundamental necessity to ensure maintainability and supportability of the application once it's deployed. BizTalk development introduces somewhat new challenges due to the distributed nature of the BizTalk infrastructure. For example, a typical BizTalk application may leverage the BizTalk messaging features, then optionally start an orchestration for processing, call the Business Rule Engine, interact with several lines of business LOB (Line of Business) or ERP systems, and return a response back to another third-party system.

The use case just described is not atypical, and would entail catching and reporting on exceptions possibly generated by several decoupled subsystems within BizTalk as well as numerous third-party LOB systems with their own exception-handling constraints. Building on the previous scenario of the fictitious company ABC, what if the mainframe rejected errors while accepting orders from the web site? How does the web site compensate for this, and how should the user be notified? The developer is faced with several exception-reporting options in a BizTalk environment:

  • Health Activity and Tracker (HAT)

  • BizTalk Administration Console

  • Microsoft Event Log

  • Custom development options)

Given the complexity, the development of a consistent solution for application exception management should embody several common design goals:

  • Standardize how application exceptions are detected and caught in the BizTalk environment, i.e., messaging and orchestration subsystems.

  • Provide common patterns that allow automated processes to react and manage application exceptions.

  • Provide a loosely coupled exception management pattern that facilitates reuse.

  • Develop a common reporting paradigm of application exceptions and their available message state that applies to any BizTalk subsystem.

The BizTalk product team considered some of these points during the development of BizTalk Server 2006. For example, BizTalk Server 2006 introduced the Failed Message Routing feature. This essentially allowed users to create orchestration processes or messaging send ports that could be configured to "subscribe" to any exceptions that occurred at the messaging subsystem level.

Consider the following scenario:

A user submits financial records to BizTalk.During the parsing and validation stage, an exception is thrown.

Using BizTalk Server 2004, the message would be moved to a suspended queue where limited options were available. For instance, an operator would first have to independently detect that an exception did in fact occur. Then, the message would have to be manually saved to disk from the HAT user interface. Next, the message would have to be manually corrected and resubmitted to the system.

In BizTalk Server 2006, an operator can simply create either an orchestration or a messaging send port that subscribes to any failed message, thereby solving the issue of automated error detection and routing the original message state for processing. Unfortunately, there was no time to build a similar feature for orchestrations. This leaves you today with two very different ways in which exceptions are processed and managed within a BizTalk system. It's up to the developer to customize the exception handling as necessary.

Failed Message Routing was not the only new feature that shipped with BizTalk Server 2006. A new BizTalk Administration Console provides a set of Group Hub pages that allows the query of suspended instances and exceptions grouped by either application, service name, error code, or URI, as shown in Figure 1. Although this provides a common user interface to view exceptions, its views are limited to "live" service instances, and examining state can be a bit cumbersome due to the drill-down required.

Figure 1 .BizTalk Server 2006 Administration Console Group Hub query page showing all suspended services grouped by application

Several other factors also limit the BizTalk Administration Console in its application exception reporting role:

  • There is no way to mine the data for business intelligence. For instance, how does someone query to see what the worst offending applications are on a monthly basis or examine quarterly trends of application exceptions?

  • The business may want to be alerted when certain application exceptions occur or when specific thresholds are reached. How does someone subscribe to such exception events? The MMC as a reporting tool is not the ideal interface, since it's not very convenient to access in production environments. You need at minimum to be in the BizTalk Operator's role, and, in production environments, access to the MMC is usually limited to a terminal server client interface, reducing the audience significantly.

  • Only unhandled exceptions, i.e., suspended service instances, are displayed in the Administration Console. If the developer handles the exception within the orchestration, exiting the service, the exception information will never be displayed in the Administration Console.

An obvious suggestion would be to build a custom web portal interface to replace the MMC. However, because of the way data is stored, exposed, and accessed by the BizTalk Administration Console, this is a nearly impossible task. Any unified reporting system would really need to feed off the several sources, including the Microsoft Event Log and HAT.

With BizTalk, you're working in a message-oriented paradigm. Everything in a BizTalk solution is message oriented, and developers think in a message-oriented mindset. Wouldn't it seem natural that exception handling also be done in a message-oriented manner?

Fortunately, BizTalk Server 2006 provides you with a sophisticated infrastructure and object model that will allow you to build a simple API that should address your common goals for application exception management in a message-oriented fashion, for both the BizTalk messaging and orchestration subsystems. The API you build can then be used to take advantage of other features within BizTalk, such as the subscription model and eventbased Business Activity Monitoring. Coupled with Windows SharePoint Services, you should be able to provide a fairly robust reporting portal for all application exceptions that occur within BizTalk.

The Exception Management Challenge

In BizTalk Server 2004, there was no built-in mechanism to make this an easy accomplishment for the developer. For instance, if an error occurred in the BizTalk messaging subsystem, the message would simply be written to BizTalk's suspended queue. If the developer needed to view, repair, or resubmit the message, it entailed some rather elaborate custom development work. In addition, developers would have to write their own solution, subscribing to WMI events, to detect that the suspension of the message actually took place.

This forced some developers to design their applications not with business considerations in mind, but rather with exception management in mind. For instance, because there was no way to trap the exception that a BizTalk map could generate in the messaging subsystem, many developers opted to put maps in the orchestration, rather than in the receive and send ports where they logically should have been placed. They did so simply because BizTalk offered a graphical mechanism for catching and reacting to application exceptions within an orchestration. Some developers even took this a step further: incorporating more code within the orchestration, code that logically should be placed within a custom pipeline component, for no better reason than to have the ability to catch the exception and react to it.

When using an orchestration, developers could graphically define a Scope shape with one or more exception handlers, similar to the Try...Catch...Finally functionality you would incorporate writing any pure .NET application. This gave the developer a way to catch the exceptions, but custom code would still have to be written within the Exception Handler shape to process the exception. By default, just as with a .NET application, BizTalk doesn't do anything with the .NET exception it catches. An example of this is shown in Figure 2. It's up to the developer to do something, i.e., write to Debug.Trace, incorporate custom logging, write to the Microsoft Event Log, and/or write the exception to HAT.

Figure 2. BizTalk Server 2006 Orchestration Designer demonstrating the inclusion of multiple exception handlers

We've seen some fairly well-thought-out and elaborate exception handling implemented in BizTalk, but usually just at the orchestration level. For example, it wasn't uncommon to see the Exception Management Application Block (posted on the Microsoft Patterns and Practices web site)[] used as a common way to control exception reporting. Using it was a little challenging, though due to the overhead and configuration files, it was necessary. Other times we've seen custom exception reporting to a database.

[] www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=8CA8EB6E-6F4A-43DFADEB-8F22CA173E02

However, none of the exception management options we've just described provide a unified pattern for managing and reporting application exceptions in a BizTalk environment that includes both orchestration and messaging subsystems. And neither addresses the issue of managing the state involved in a misbehaving process.

Managing state, or to be clearer, making the state available as part of the exception management process, is fairly important when you consider scenarios that require the current message state when handling the exception. A common scenario that would require message state availability as well as a robust exception management process would be the following:

A user submits an invoice to the system.During the course of processing the invoice within an orchestration, the Business Rule Engine throws an application exception because a piece of the data is incorrect. The business process should catch the exception and send the offending message to another person or system. That person or system can correct and resubmit the message for processing.

Using existing exception management techniques, the exception management logic could be maintained in the current Exception Handler shape on the existing orchestration.Alternatively, you could tightly couple the current orchestration to a secondary orchestration and pass the message over to it for processing.

The downside to the former approach is that handling more elaborate exception use cases is done within the current Orchestration Exception Handler shapes. This means an orchestration starts to grow in complexity due to exception correction logic being embedded in it. What happens if you have five, ten, or even more Exception Handler shapes that need corrective action or management? This slowly becomes difficult to manage as it limits the development of all exception logic to a single developer (due to the single orchestration handling the exception logic).

The latter approach would move the logic out of the exception handlers and into secondary orchestrations by using either the Start Orchestration or Call Orchestration shapes. This would allow additional developers to work with the exception logic but would maintain a fairly tightly coupled process.

Now consider the following scenario where message state is part of the exception reporting process:

There are several BizTalk applications deployed in the environment. Some are generating application exceptions at the messaging subsystem level, while others are generating application exceptions at the orchestration level. The internal IT Operations department needs to be able view a central portal for all exception messages, their state, and application exception trends by fiscal quarter across applications.

Today, there is no BizTalk feature that provides this functionality, since there is no standard and consistent way of reporting and logging all exceptions generated in a BizTalk environment.

Clearly, you would want to accomplish several things by developing a unified pattern and API for business exception handling. The goals of this approach would be

  • Develop loosely coupled asynchronous orchestration processes that have the ability to subscribe to specific business exceptions generated by either the messaging subsystem or an orchestration.

  • Enable identification of common exceptions generated from either orchestration- or messaging-based applications.

  • Allow the development of common business exception orchestration handlers that function across multiple orchestration applications.

  • Provide the ability to create these business exception orchestration handlers independent of the currently deployed orchestration-based applications.

  • Provide a web-based portal that displays application and business exception trends using optional BAM observations models as well as exception messages and application state driven from both orchestration- and messaging-based applications.

  •  D-Link DHP-1565 Wireless N Powerline Router
  •  Application Patterns and Tips : Localize a Windows Forms Application, Localize an ASP.NET Application
  •  Application Patterns and Tips : Use Model-View-ViewModel in WPF
  •  Liquid-Metal
  •  Vigor 2850n
  •  Visual Studio 2010 : Introducing the Visual Studio Extensibility - Extending the Code Editor
  •  Visual Studio 2010 : Managing Extensions with the Extension Manager, Managing Add-Ins with the Add-In Manager
  •  Intel Xeon Phi: Coprocessor speeding at 1 teraflops in a PCIe Card
  •  Visual Studio Team System 2008 : Working with Test Results (part 2) - Build report and test result
  •  Visual Studio Team System 2008 : Working with Test Results (part 1) - Test as part of Team Foundation Server build
  •  Finance - Apple Versus Google
  •  Oracle Coherence 3.5 : Testing and debugging Coherence applications
  •  Oracle Coherence 3.5 : Accessing the data grid (part 6) - Using the Coherence API - Implementing CoherenceTarget, Testing the Cache loader
  •  Oracle Coherence 3.5 : Accessing the data grid (part 5) - Using the Coherence API - Loader design, Implementing CsvSource
  •  Oracle Coherence 3.5 : Accessing the data grid (part 4) - Using the Coherence API - The basics: NamedCache and CacheFactory
  •  Oracle Coherence 3.5 : Accessing the data grid (part 3) - Configuring Coherence
  •  Oracle Coherence 3.5 : Accessing the data grid (part 2) - Configuring the development environment
  •  Oracle Coherence 3.5 : Accessing the data grid (part 1) - Coherence console
  •  Oracle Coherence 3.5 : Installing Coherence, Starting up the Coherence cluster
  •  The Go-To Reference Design Map For The Cloud?
    Top 10
    Nikon 1 J2 With Stylish Design And Dependable Image And Video Quality
    Canon Powershot D20 - Super-Durable Waterproof Camera
    Fujifilm Finepix F800EXR – Another Excellent EXR
    Sony NEX-6 – The Best Compact Camera
    Teufel Cubycon 2 – An Excellent All-In-One For Films
    Dell S2740L - A Beautifully Crafted 27-inch IPS Monitor
    Philips 55PFL6007T With Fantastic Picture Quality
    Philips Gioco 278G4 – An Excellent 27-inch Screen
    Sony VPL-HW50ES – Sony’s Best Home Cinema Projector
    Windows Vista : Installing and Running Applications - Launching Applications
    Most View
    Bamboo Splash - Powerful Specs And Friendly Interface
    Powered By Windows (Part 2) - Toshiba Satellite U840 Series, Philips E248C3 MODA Lightframe Monitor & HP Envy Spectre 14
    MSI X79A-GD65 8D - Power without the Cost
    Canon EOS M With Wonderful Touchscreen Interface (Part 1)
    Windows Server 2003 : Building an Active Directory Structure (part 1) - The First Domain
    Personalize Your iPhone Case
    Speed ​​up browsing with a faster DNS
    Using and Configuring Public Folder Sharing
    Extending the Real-Time Communications Functionality of Exchange Server 2007 : Installing OCS 2007 (part 1)
    Google, privacy & you (Part 1)
    iPhone Application Development : Making Multivalue Choices with Pickers - Understanding Pickers
    Microsoft Surface With Windows RT - Truly A Unique Tablet
    Network Configuration & Troubleshooting (Part 1)
    Panasonic Lumix GH3 – The Fastest Touchscreen-Camera (Part 2)
    Programming Microsoft SQL Server 2005 : FOR XML Commands (part 3) - OPENXML Enhancements in SQL Server 2005
    Exchange Server 2010 : Track Exchange Performance (part 2) - Test the Performance Limitations in a Lab
    Extra Network Hardware Round-Up (Part 2) - NAS Drives, Media Center Extenders & Games Consoles
    Windows Server 2003 : Planning a Host Name Resolution Strategy - Understanding Name Resolution Requirements
    Google’s Data Liberation Front (Part 2)
    Datacolor SpyderLensCal (Part 1)