Designing consistent reusable patterns for exception
management within any development project is a fundamental necessity to
ensure maintainability and supportability of the application once it's
deployed. BizTalk development introduces somewhat new challenges due to
the distributed nature of the BizTalk infrastructure. For example, a
typical BizTalk application may leverage the BizTalk messaging features,
then optionally start an orchestration for processing, call the
Business Rule Engine, interact with several lines of business LOB (Line
of Business) or ERP systems, and return a response back to another
third-party system.
The use case just
described is not atypical, and would entail catching and reporting on
exceptions possibly generated by several decoupled subsystems within
BizTalk as well as numerous third-party LOB systems with their own
exception-handling constraints. Building on the previous scenario of the
fictitious company ABC, what if the mainframe rejected errors while
accepting orders from the web site? How does the web site compensate for
this, and how should the user be notified? The developer is faced with
several exception-reporting options in a BizTalk environment:
Health Activity and Tracker (HAT)
BizTalk Administration Console
Microsoft Event Log
Custom development options)
Given the complexity, the
development of a consistent solution for application exception
management should embody several common design goals:
Standardize how
application exceptions are detected and caught in the BizTalk
environment, i.e., messaging and orchestration subsystems.
Provide common patterns that allow automated processes to react and manage application exceptions.
Provide a loosely coupled exception management pattern that facilitates reuse.
Develop
a common reporting paradigm of application exceptions and their
available message state that applies to any BizTalk subsystem.
The BizTalk product
team considered some of these points during the development of BizTalk
Server 2006. For example, BizTalk Server 2006 introduced the Failed Message Routing
feature. This essentially allowed users to create orchestration
processes or messaging send ports that could be configured to
"subscribe" to any exceptions that occurred at the messaging subsystem
level.
Consider the following scenario:
A user submits financial records to BizTalk.During the parsing and validation stage, an exception is thrown.
Using BizTalk Server
2004, the message would be moved to a suspended queue where limited
options were available. For instance, an operator would first have to
independently detect that an exception did in fact occur. Then, the
message would have to be manually saved to disk from the HAT user
interface. Next, the message would have to be manually corrected and
resubmitted to the system.
In BizTalk Server 2006,
an operator can simply create either an orchestration or a messaging
send port that subscribes to any failed message, thereby solving the
issue of automated error detection and routing the original message
state for processing. Unfortunately, there was no time to build a
similar feature for orchestrations. This leaves you today with two very
different ways in which exceptions are processed and managed within a
BizTalk system. It's up to the developer to customize the exception
handling as necessary.
Failed Message Routing was
not the only new feature that shipped with BizTalk Server 2006. A new
BizTalk Administration Console provides a set of Group Hub pages that
allows the query of suspended instances and exceptions grouped by either
application, service name, error code, or URI, as shown in Figure 1.
Although this provides a common user interface to view exceptions, its
views are limited to "live" service instances, and examining state can
be a bit cumbersome due to the drill-down required.
Several other factors also limit the BizTalk Administration Console in its application exception reporting role:
There is no way to
mine the data for business intelligence. For instance, how does someone
query to see what the worst offending applications are on a monthly
basis or examine quarterly trends of application exceptions?
The
business may want to be alerted when certain application exceptions
occur or when specific thresholds are reached. How does someone
subscribe to such exception events? The MMC as a reporting tool is not
the ideal interface, since it's not very convenient to access in
production environments. You need at minimum to be in the BizTalk
Operator's role, and, in production environments, access to the MMC is
usually limited to a terminal server client interface, reducing the
audience significantly.
Only
unhandled exceptions, i.e., suspended service instances, are displayed
in the Administration Console. If the developer handles the exception
within the orchestration, exiting the service, the exception information
will never be displayed in the Administration Console.
An obvious suggestion would
be to build a custom web portal interface to replace the MMC. However,
because of the way data is stored, exposed, and accessed by the BizTalk
Administration Console, this is a nearly impossible task. Any unified
reporting system would really need to feed off the several sources,
including the Microsoft Event Log and HAT.
With BizTalk, you're
working in a message-oriented paradigm. Everything in a BizTalk solution
is message oriented, and developers think in a message-oriented
mindset. Wouldn't it seem natural that exception handling also be done
in a message-oriented manner?
Fortunately, BizTalk Server
2006 provides you with a sophisticated infrastructure and object model
that will allow you to build a simple API that should address your
common goals for application exception management in a message-oriented
fashion, for both the BizTalk messaging and orchestration subsystems.
The API you build can then be used to take advantage of other features
within BizTalk, such as the subscription model and eventbased Business
Activity Monitoring. Coupled with Windows SharePoint Services, you
should be able to provide a fairly robust reporting portal for all
application exceptions that occur within BizTalk.
The Exception Management Challenge
In BizTalk Server 2004,
there was no built-in mechanism to make this an easy accomplishment for
the developer. For instance, if an error occurred in the BizTalk
messaging subsystem, the message would simply be written to BizTalk's
suspended queue. If the developer needed to view, repair, or resubmit
the message, it entailed some rather elaborate custom development work.
In addition, developers would have to write their own solution,
subscribing to WMI events, to detect that the suspension of the message
actually took place.
This forced some
developers to design their applications not with business
considerations in mind, but rather with exception management in mind.
For instance, because there was no way to trap the exception that a
BizTalk map could generate in the messaging subsystem, many developers
opted to put maps in the orchestration, rather than in the receive and
send ports where they logically should have been placed. They did so
simply because BizTalk offered a graphical mechanism for catching and
reacting to application exceptions within an orchestration. Some
developers even took this a step further: incorporating more code within
the orchestration, code that logically should be placed within a custom
pipeline component, for no better reason than to have the ability to
catch the exception and react to it.
When using an
orchestration, developers could graphically define a Scope shape with
one or more exception handlers, similar to the Try...Catch...Finally
functionality you would incorporate writing any pure .NET application.
This gave the developer a way to catch the exceptions, but custom code
would still have to be written within the Exception Handler shape to
process the exception. By default, just as with a .NET application,
BizTalk doesn't do anything with the .NET exception it catches. An
example of this is shown in Figure 2. It's up to the developer to do something, i.e., write to Debug.Trace, incorporate custom logging, write to the Microsoft Event Log, and/or write the exception to HAT.
We've seen some
fairly well-thought-out and elaborate exception handling implemented in
BizTalk, but usually just at the orchestration level. For example, it
wasn't uncommon to see the Exception Management Application Block
(posted on the Microsoft Patterns and Practices web site)
used as a common way to control exception reporting. Using it was a
little challenging, though due to the overhead and configuration files,
it was necessary. Other times we've seen custom exception reporting to a
database.
However, none of
the exception management options we've just described provide a unified
pattern for managing and reporting application exceptions in a BizTalk
environment that includes both orchestration and messaging subsystems.
And neither addresses the issue of managing the state involved in a
misbehaving process.
Managing state, or to be
clearer, making the state available as part of the exception management
process, is fairly important when you consider scenarios that require
the current message state when handling the exception. A common scenario
that would require message state availability as well as a robust
exception management process would be the following:
A user
submits an invoice to the system.During the course of processing the
invoice within an orchestration, the Business Rule Engine throws an
application exception because a piece of the data is incorrect. The
business process should catch the exception and send the offending
message to another person or system. That person or system can correct
and resubmit the message for processing.
Using
existing exception management techniques, the exception management logic
could be maintained in the current Exception Handler shape on the
existing orchestration.Alternatively, you could tightly couple the
current orchestration to a secondary orchestration and pass the message
over to it for processing.
The downside to the
former approach is that handling more elaborate exception use cases is
done within the current Orchestration Exception Handler shapes. This
means an orchestration starts to grow in complexity due to exception
correction logic being embedded in it. What happens if you have five,
ten, or even more Exception Handler shapes that need corrective action
or management? This slowly becomes difficult to manage as it limits the
development of all exception logic to a single developer (due to the
single orchestration handling the exception logic).
The latter approach would
move the logic out of the exception handlers and into secondary
orchestrations by using either the Start Orchestration or Call
Orchestration shapes. This would allow additional developers to work
with the exception logic but would maintain a fairly tightly coupled
process.
Now consider the following scenario where message state is part of the exception reporting process:
There
are several BizTalk applications deployed in the environment. Some are
generating application exceptions at the messaging subsystem level,
while others are generating application exceptions at the orchestration
level. The internal IT Operations department needs to be able view a
central portal for all exception messages, their state, and application
exception trends by fiscal quarter across applications.
Today, there is no
BizTalk feature that provides this functionality, since there is no
standard and consistent way of reporting and logging all exceptions
generated in a BizTalk environment.
Clearly, you
would want to accomplish several things by developing a unified pattern
and API for business exception handling. The goals of this approach
would be
Develop
loosely coupled asynchronous orchestration processes that have the
ability to subscribe to specific business exceptions generated by either
the messaging subsystem or an orchestration.
Enable identification of common exceptions generated from either orchestration- or messaging-based applications.
Allow
the development of common business exception orchestration handlers
that function across multiple orchestration applications.
Provide
the ability to create these business exception orchestration handlers
independent of the currently deployed orchestration-based applications.
Provide
a web-based portal that displays application and business exception
trends using optional BAM observations models as well as exception
messages and application state driven from both orchestration- and
messaging-based applications.
