BizTalk Server is a middleware platform that
provides native messaging, broker, and orchestration features along
with numerous extensions. Its original purpose was to act as a
middle-tier hub for integration purposes, but it has evolved into a
comprehensive set of technologies that provide various forms of
services-based intermediary processing in support of service, service
composition, and service inventory architectures that can leverage and
build upon orchestration and enterprise service bus-style platforms.
The BizTalk technology
architecture is comprised of various parts that all, in some way,
relate to the BizTalk Server engine that lies at the core of the
BizTalk Server product. The engine drives two primary components:
messaging and orchestration (Figure 1).
Messaging
is what allows BizTalk Server to enable communication within its
boundaries and between BizTalk and external systems and services. It
relies heavily on adapters and can support a variety of protocols and
data formats.
Orchestration
is used for creating and executing business process and composition
logic. The orchestration component (also referred to as the orchestration engine) is built on top of the messaging component.
The BizTalk Server engine is augmented with several productivity tools, including:
business rules engine
health and activity tracking
enterprise single sign-on
business activity monitoring
business activity services
The business rules engine
allows you to customize the orchestration component by making it
simpler and faster to change business rules.
|
Several of the features
that pertain to and build upon BizTalk Server’s messaging framework
relate to how the Enterprise Service Bus
compound pattern can be applied to the Microsoft technology platform in
general. Similarly, BizTalk Server’s native orchestration capabilities
lay the foundation for how the Orchestration compound pattern can be applied.
|
The health and activity
tracking tool lets you monitor and manage the BizTalk engine and any
orchestrations it may be running. It is commonly used to track all sent
and received messages.
Enterprise single
sign-on is an extension of Windows Enterprise security that provides
the ability to map authentication credentials between Windows and
non-Windows systems. It is enabled by a set of processes in BizTalk
Server that provide user account and password mapping, caching single
sign-on for multiple Windows domains, and password synchronization.
Business activity
monitoring (BAM) is comprised of a set of tools and services used to
monitor transactions in real time and view key performance indicators
of a business process being automated by a given orchestration.
Finally, business activity
services are used to manage interaction with trading partners and are
focused on the utilization of BizTalk for business-to-business (B2B)
purposes.
BizTalk Server Architecture
The underlying
architecture of BizTalk Server is modular and allows for the
distribution of various processing functions to different servers. It
supports high fault tolerance and enables scaling out with increasing
load. You can dedicate some servers to exclusively process incoming or
outgoing messages, while other servers can execute orchestration logic,
transform and transmit messages, and manage process state. As shown in Figure 2, the MessageBox SQL Server database stores messages and connects the pieces of the overall architecture.
BizTalk’s architecture
recognizes that not all IT enterprises can publish Web service
endpoints for their applications. The adapter model in BizTalk Server
therefore extends orchestration capabilities beyond the use of Web
services. Applications accessed via a number of legacy protocols and
enterprise software packages can participate in orchestrations. These
adapters can bridge the transport and protocol gaps between proprietary
products, such as IBM MQSeries or SAP, and the orchestration engine.
Figure 3
provides an illustrative overview of the logical relationships and
responsibilities of the primary parts of the BizTalk architecture.
Messaging
XML
is the fundamental data representation technology that underlies the
different layers of BizTalk technology. Messages and service contracts
are expressed with XML types and transformation logic is based on the
use of XSLT. XPath expressions can be executed as part of workflow
decision points and in message assignments.
Pipelines
Figure 4
provides some insight into the message processing mechanics of BizTalk
Server. Messages are sent to receive adapters, which are available for
most communication mechanisms, including files, Web services, and HTTP.
The receive pipeline contains a pipeline of components used to convert
messages from their native format to XML documents. XML messages are
then delivered to the MessageBox hosted in SQL Server.
All messages are
converted into XML before they can be consumed by the orchestration
engine. The business rules engine augments the orchestration module by
providing a simpler way to express complex sets of rules in a business
process. Each orchestration must subscribe to the type of message it
wants to receive. The appropriate XML message is processed by the
orchestration engine, which results in a new XML message being created.
The processed XML is then sent through the send pipeline, which changes
the XML message to the native format required by the destination
system. The message is then transmitted to the destination system using
the send adapter.
Note
BizTalk publish-and-subscribe messaging and legacy adapters are covered separately in upcoming sections.
Pipeline Components
BizTalk Server engine only
works with XML documents internally, and pipelines are used to convert
a document from its native format to XML. The receive pipeline is
responsible for preparing the XML message and publishing it to the
MessageBox. A pipeline is a series of components (that exist as .NET
assemblies) that are executed in sequence. Each component provides
specific processing logic to a message, such as decryption, encryption,
parsing, or XML Schema validation, as shown in Figure 5.
Let’s take a closer look at the components from Figure 7.15, each of which represents a separate message processing stage:
1. | decode – decodes or decrypts the received message
|
2. | disassemble – converts a native message into an XML message
|
3. | validate – validates the XML using an XML schema
|
4. | resolve party – identifies the party associated with the message
|
5. | pre-assemble – performs message processing before assembling the message
|
6. | assemble – assembles the message from XML to its native format
|
7. | encode – encodes or encrypts the message before transmitting it to the send port
|
The process of converting a
message from its native format to an XML message may require a series
of transformations that will be performed on a message as it is being
sent and received. For example, a comma-delimited flat file may need to
be converted to an XML message. The disassemble component is
responsible for this transformation.
Ports and Locations
To
enable cross-service messaging you need to establish endpoints, ports,
and locations. An endpoint is a logical representation of a location
(generally expressed as a URL) that provides a physical address for
message data received or sent. There are receive and send locations and
corresponding ports, as follows:
1. | receive location – a physical location or an adapter type
|
2. | receive port – a logical grouping of receive locations—ports are used to abstract physical endpoints in BizTalk Server
|
3. | send location – a logical grouping of send locations
|
4. | send port – a physical location or an adapter type—ports are used to abstract physical endpoints in BizTalk Server
|
It is possible to completely
bypass the orchestration engine and have a message sent directly from
the receive pipeline to the send pipeline, thereby making BizTalk
Server behave like a messaging system. Messages originating from the
receive ports are deposited to the MessageBox. They are then delivered
to the send ports that subscribe to specific messages, in effect
creating a subscription-based mechanism. The send pipeline can
subscribe to messages based on the properties set on messages by the
receive pipeline.
Adapters
Through the use of
adapters the BizTalk platform can be accessed and interacted with by a
variety of products, applications, and services. Adapters are
essentially responsible for overcoming transport disparity between
incompatible systems and services. BizTalk adapters specifically
contain internal mapping logic that transforms external protocols into
native BizTalk-supported protocols and vice versa.
BizTalk Server includes
several built-in adapters, including a Web services adapter, a file
adapter, an HTTP adapter, an MSMQ adapter, an SMTP adapter, and even a
POP3 adapter. Besides built-in adapters, there is also a substantial
third-party adapter marketplace. All adapters are based on a common
BizTalk Server adapter framework that provides a set of base classes
used to create and run adapter logic.
Let’s briefly highlight some adapters relevant to upcoming topics:
1. | Web Services Adapter– The adapter used for Web service communication allows for the sending and receiving of SOAP messages over HTTP.
|
2. | WCF Adapter– Similar
to the Web services adapter, the BizTalk Server WCF adapter provides
full support for the WS-* stack, including various security, reliable
messaging, and transaction processing specifications. The WCF adapter
is essentially used to connect WCF services to BizTalk Server.
|
3. | SQL Server Service Broker Adapter–
SQL Server Service Broker (SSB) adapter for BizTalk Server allows
BizTalk Server to participate in SSB conversations as a target or as an
initiator. SSB communication is based on the dialog concept where a
dialog is comprised of a reliable stream of messages between two
services.
|
|
Although several runtime transformation-related patterns exist (such as Data Model Transformation and Data Format Transformation ), the pattern most synonymous with the use of adapters is Protocol Bridging .
|
Context Properties
When message processing
logic is defined, the details of that logic are generally not part of
the message payload data. This is why metadata is required. Message
metadata can be associated with and attached to the body content of
messages travelling through the BizTalk environment.
There are two practical ways this can be achieved:
1. | as
an envelope around the message, where the message itself becomes the
payload in an envelope that also contains a header section for the
metadata
|
2. | as BizTalk context properties
|
The envelope approach
is perfectly valid and quite common. However, it requires that a
transformation occur every time a message is received by BizTalk Server
in order to insert the message into an envelope. Most likely another
transformation prior to message delivery is required to remove the
envelope (unless the message is being delivered to a service that
understands or requires the actual envelope). This transformation logic
can impose processing overhead on every message flowing through the
system.
BizTalk context properties
are associated with a message for the duration of its lifetime inside
the BizTalk Server environment. Context properties are created and
added to messages as they are received, or at any point in their
lifecycle inside BizTalk. As they are not physically part of the
message, context properties are by definition lost when a message
leaves the BizTalk boundary (although you can demote values from
message context into message content by use of a property schema and XML Assembler pipeline component).
It is important to note that
regardless of whether you choose to work with context properties, the
downstream logic is the same. The only difference is the source of the
message metadata. Additionally, you may opt to host some of the
metadata in external stores, such as a registry.
Note
The association of metadata to messages is a concept embodied in the fundamental Messaging Metadata
pattern. The most common means of applying this pattern has
historically been through the use of SOAP messages that contain
separate body and header sections (the latter of which contains the
metadata). However, the use of BizTalk context properties also
represents a legitimate means of realizing this pattern, albeit in a
non-industry-standard manner.
Itineraries
The BizTalk architecture
allows for the decoupling of services and the use of message metadata
to route messages to services. As a result, services can be autonomous,
self-contained units that are not aware of each other. Orchestration
and service composition logic are two primary means of tying services
together for specific purposes. However, there is a third option
provided specifically by BizTalk Server and one that is especially
worth noting because it can coordinate services while still being part
of larger orchestrations.
Itineraries (a form of
microflow) are intended for simple, short-running service compositions
and message flows. Itineraries include a broker capability that allows
you to perform conditional branching from inside an itinerary. This
should only be used sparingly and when required, as it is another point
in which you can embed business logic.
Itineraries allow you to do
something like call a service, transform the response, call a second
service, and then a third service. There are no transactional or
compensation semantics in an itinerary, as these types of features are
beyond their ability and purpose.
It is important to note that
there may be circumstances that warrant the use of itineraries and
messaging only-type solutions that do not involve orchestrations. For
example, simple message flows that do not require complex runtime
processing can be based solely on itineraries and thereby avoid the
overhead of invoking an orchestration.
Unified Exception Management
Effective exception
management takes on increased importance within complex orchestration
logic as debugging failures in a loosely coupled solution can be
challenging. By using a well-planned exception management strategy, it
is possible to construct solutions that can not only record failures,
but also potentially invoke compensation processes or services in
response to exception conditions.
Exceptions fall into one of two categories:
business exceptions
system exceptions
BizTalk solutions can
span multiple technologies, including orchestrations, pipeline
components, Web services (internal and external), and custom
assemblies. They are therefore often required to cross security and
organizational boundaries. A unified exception handling mechanism can
provide exception handling capabilities for all of these disparate
participants and technologies.
BizTalk Server 2006
first introduced the notion of failed message routing which, for the
first time, allowed for the creation of message subscribers that failed
during receipt (such as when messages did not conform to an expected
XML schema). If we take that approach and apply it to exception
management, then we can create a mechanism that will allow processes
encountering failures to publish an exception message and handlers that
will subscribe to those exception messages.
This loosely coupled approach to exception management offers several benefits:
By decoupling
exception handling from the process itself, we have the opportunity to
stratify assemblies so that exception management for a given process
becomes a separate assembly that can be versioned and deployed without
affecting the currently deployed and running process.
Multiple
handlers can potentially respond to a given type of exception, as may
be the case if we want to track metrics for all exceptions that occur
within a given boundary.
A message-oriented exception management system allows for the creation of both generic and targeted handlers.
In a fully
functional message-based exception management strategy, you will also
have the ability to include, as payload in the exception message, any
messages that may have been in-flight at the time the exception
condition was encountered. With this information you will have the
capacity to potentially reconstruct those messages in an exception
handler, thereby allowing for repair and re-submission.
In a repair and
resubmit scenario, users can access InfoPath forms on a SharePoint
site, perhaps edit data in the form, or approve it, and submit it. This
technique is effective in situations where a message can be repaired by
changing a value or where a human intervention approval step may be
required (as explained earlier).
In order to
implement a unified exception management system, you first need to
create a schema that defines what an exception message looks like. A
well-defined XML schema will include enough information that both
generic and targeted handlers will be able to either generically or
selectively subscribe to.
This may include such items as:
Application – the application name (for example, Payroll)
Process – a specific process that encountered the exception condition (for example, PrintCheck)
Exception Message – human-meaningful description of the failure (for example, cannot contact printer)
Fault Code
– a standardized code used as a filter condition by subscribers
(typically, codes are governed and allocated by a centralized group in
order to ensure there is no overlap)
Severity
– allows you to set varying priority handlers (for example, a handler
may listen for critical severity exceptions and invoke a human
notification process)
