by Clemens Utschig-Utschig
Published September 2008
Over the last few years, event-driven architecture (EDA) has moved into the spotlight as a means of meeting high performance and scalability requirements. Loosely coupled architectures have been notoriously weak in meeting those requirements, mainly because of the focus on Web service technologies, but also because of XML's processing requirements. This primer aims to guide you through the basics of architecting an event-driven SOA by introducing the concepts underlying EDA, and by describing how they can be applied to SOA.
A typical, process-centric case for EDA can be found in every airport: the business process that spans check-in to departure.
Although in reality the described process has a lot of corner cases, such as fare calculation and crew assignment, this primer focuses on several key requirements:
Most of us have experienced real-life failures to meet those requirements, such as when checked baggage doesn't arrive at the destination airport, or when faulty communication of a last-minute gate change delays boarding.
Events can be described as state changes, such as the change between when you check your bag to when the baggage system detects it for the first time ( checked to in delivery).
Thinking in terms of defined state changes of data throughout its lifecycle, rather than calculating entity state based on historic information (such as the applications an entity has passed through), requires rethinking the traditional model of message-oriented application design.
Notably, events are nothing new and have been used for many years in the transportation and retail markets, as in the use of RFID sensors or the tracking the delivery of goods. However, combining them with SOA is a fairly recent concept.
Event design is a common problem area for those in the SOA world. While a message (or an entity) is focused on containing descriptive data, such as the name, an event should focus solely on describing a change.
The following example explains the difference, using a
<?xml version="1.0" encoding="windows-1252" ?> <flightRecord xmlns="urn:flightRecordSystem" > <customer> <name>Clemens Utschig</name> <dob>12-29-1981</dob> <address> <street>1535 Eddy St., Apt 517</street> <city>San Francisco</city> <zip>CA 94115</zip> <country>USA</country> </address> </customer> <freqentFlyer carrier="LH" number="1234567891"/> <flightroute segments="1"> <flight segment="1" departureDate="11-15-2005" checkinNo="123" seat="33A" specialMeal="no">LH458</flight> </flightroute> <checkedBags> <bag seq="1" weight="24" metric="pounds" guid="1234567891-1" status="checked"/> </checkedBags> </flightRecord>
flightRecord defined above, with all its information, is the data that systems will work on from the start of the business process (if not earlier) until the process is completed. As discussed earlier, an event reflects a change in the state of an entity, such as the transition of the bag (1234567891-1) as it went through security screening.
< bagEvent xmlns="urn:bagTracking"> <header> <timestamp>11-15-2005 10:23 AM PDT</timestamp> <system type="SCREENING">SFO-1</system> </header> <body> <bag guid="1234567891-1" flightCode="LH458">SECURITY OK</bag> </body> </ bagEvent>
The above event contains no additional entity data, aside from a key to uniquely identify the entity to which it belongs.
Although some people believe that an event should contain the previous state, such as "CHECKED," this should be strongly discouraged. An event should be a single, nondependent representation of change. The aggregation of events should therefore be left to the infrastructure.
Key design considerations. In service-/process-oriented design, the implementation usually follows a predetermined (designed) flow. At runtime the process is responsible for evaluating different conditions (incoming messages, conditions) that might mean that a message takes a different path through the (static) flow.
With EDA in mind, it is important to clearly distinguish between static parts of a process and conditions that are based on aggregations of events. Here the architect decides what moves into the service/process part and what events are published and aggregated outside the flow.
Let's take a concrete requirement from above and see where it fits.
From the moment a passenger checks any bags, those bags begin moving through the baggage system. They should end up on the right plane, possibly in the right container, and obviously on time. If a bag contains hazardous materials, the passenger will not be allowed to board the plane..
The following matrix of evaluation criteria will aid in deciding where and how this requirement should be modeled.
|Type of event||Process implementation||Event-based implementation|
|An event that can happen several times (e.g. seat change)||Only the final seat is of interest to the process|
|A message within the flow that takes a different path because of the event (e.g., bag contains hazardous material)||A callback from the event system into the process instance||The aggregation of several events|
|A set of (one-way) notifications without a direct impact (e.g., passenger left check-in and entered security screening)||Call to a service||Reporting in business-activity monitoring (BAM)|
|Callout to services/ orchestration||Only in the process!|
|Macro view: several events from different process instances leading to the final outcome (e.g., all passengers have passed through security screening)||A callback from the event system into the process instance||A set of atomic, single instances, with the outcome based on aggregation|
Usually it's fairly clear which direction the pendulum is swinging in -- whether it's toward the process or toward a set of events only. However, in certain cases, evaluations of events lead to new events that a process instance needs to know about.
Implementation considerations. Now that you know the decision criteria for what is modeled in the process flow and what should be modeled as events and aggregated outside, it's time to move to the implementation.
First the service-wide schema needs to be defined, reflecting the
flightRecord type. Its goal is to track the progress of the flight record throughout the process, from the beginning (passenger check-in) to the time the plane departs (including on-board luggage).
<xsd:complexType name="flightRecordType"> <xsd:annotation> <xsd:documentation> The type definition for a flight record, with a customer, an ff account, its checked bags, and the flight route </xsd:documentation> </xsd:annotation> <xsd:sequence> <!-- we can't check in a passenger whose information we don't have --> <xsd:element name=" customer" type="customerType" minOccurs="1" maxOccurs="1"/> <!-- a flight route can be credited to an ff account --> <xsd:element name=" frequentFlyer" type="frequentFlyerType" /> <!-- the collection of checked bags --> <xsd:element name=" checkedBags" type="checkedBagsType" minOccurs="1" maxOccurs="1"/> <!-- the flight route, which contains the segments --> <xsd:element name=" flightRoute" type="flightRouteType" minOccurs="1" maxOccurs="1"/> </xsd:sequence> <xsd:attribute name=" ticketNumber" type="xsd:string"/> <xsd:attribute name=" status" type="xsd:string"/> </xsd:complexType>
The next stage is to define the events the systems send out, based on the requirements discussed earlier. These events should be modeled on the event processing necessary to support the requirements. In the case of
CustomerTravelProcess, these are:
Two types of events come to mind: an event for a single bag...
<xsd:complexType name="bagEventType"> <xsd:sequence> <xsd:element name="header" type="baseEventHeaderType" minOccurs="1"/> <xsd:element name="body"> <xsd:complexType> <xsd:sequence> <xsd:element name="bagStatus" type="bagEventStatusType" minOccurs="1" maxOccurs="unbounded"/> </xsd:sequence> </xsd:complexType> </xsd:element> </xsd:sequence> </xsd:complexType>
...and an event type for a passenger (navigating through the airport or making a seat change).
Once the events are defined, the next step is to come up with a process design, centered on a "common data model" and capable of reacting to event callbacks, such as a last-minute flight cancellation by the airline.
The first steps in event-based design should focus on what is passed to the process as input. Developers often pass a lot of entity information (such as the whole flight record) to the service. However, in the event world, an event should start the process. Hence the input for the process is a
<message name="AirportCheckinToDepartureRequestMessage"> <part name="payload" element="ns1:customerEvent"/> </message> and the process is triggered by invocation of customerCheckedIn: <operation name="customerCheckedIn"> <input message="client:AirportCheckinToDepartureRequestMessage"/> </operation>
When it comes to in-flight instance events, they can be modeled in BPEL as
eventHandlers. The implementation stub for
<eventHandlers> <onMessage portType="client:AirportCheckinToDeparture" operation="flightCancelled" variable="inputEventVariable" partnerLink="client"> < .. /> </onMessage> </eventHandlers>
In these two examples, those with a savvy architect's eye will see that BPEL is centered on a push (operation-based) model, which makes things a little harder. Hence, our event-processing engine will need to call specific operations rather than just sending out events.
Because events should be transparent and not clutter the BPEL process design, the use of sensors -- sitting on variables or activities -- is recommended. From a performance point of view, sensors should only send out event notifications, rather than involving any processing logic (or issuing a synchronous service call).
<sensor sensorName="ffStatusEvent" classname="com.example.BpelActivitySensorAgent" kind="activity" target="retrieveCustomerFFRecord"> <activityConfig evalTime="completion"> <variable outputDataType="customerFFStatusEventType" outputNamespace="urn:flightRecordSystem" target="$lCustFFStatusEventVar/ns1:customerStatusFFEvent"/> </activityConfig> </sensor>
Generally speaking, the implementation of "event aggregation" inside a BPEL process is discouraged, for several reasons (although it is possible with some work). While BPEL is a programming language, it is designed for orchestrating services into a flow rather than evaluating a set of state changes, either over a long time or to handle high volume.
Another problem is the design of the evaluation algorithm itself. Whereas a static, defined flow of states can be easily modeled (for example, received, in progress), complex pattern recognition is definitely out of reach and a process is the wrong place to attempt it.
In a best-of-breed approach, these event evaluations should be done outside of the process. But once completed -- that is, if a pattern is detected -- a process can be notified through an event to continue its execution.
In the airport use case, the
checkinToDeparture process, among other systems such as security screening, will trigger events. These events are evaluated to determine whether certain passengers can make his flight or not and where their bags are. Once the last event fires (that is, the doors are armed), the process instances (of several passengers) are notified and continue processing. This does not just allow for easy, process-based reporting later, but also ensures process consistency.
Now that the distinction between what should be event-based and what shouldn't is clear, the implemented process becomes fairly lightweight. It changes from a set of tightly coupled orchestrations to an event consumer that reacts to simple events coming in.
To showcase the difference, let's look at another requirement:
If a bag contains hazardous materials, the passenger will not be allowed to board the plane.
The dozens of patterns needed to figure out if a bag contains hazardous materials -- such as weapons or other prohibited items -- are left to the complex-event-processing engine. But once a pattern match occurs, further processing, such as notifying the passenger or the authorities of the event ("Hazardous item found"), is left to the process.
The examples provided in this primer provide a brief look at how you can leverage the power of event-driven architecture and take advantage of pattern recognition, high-end loose coupling, and other important aspects of EDA in the creation of a process-centric SOA. Readers are encouraged to use this primer as a springboard to further exploration of EDA.