Adding BPEL to the Enterprise Integration Mix
by Praveen Chandran and Arun Poduval

Leverage the orchestration capability of Oracle BPEL Process Manager to enable standards-based business process integration that complements traditional EAI middleware.

Review complete BPEL Cookbook index

Downloads for this article: Oracle BPEL Process Manager and Designer

Published November 2005

Most organizations have a highly disparate application infrastructure comprising a variety of applications from multiple vendors, running on different platforms, and created with completely different technologies. Traditional enterprise application integration (EAI) products—from companies such as TIBCO, webMethods, Vitria, and SeeBeyond—have emerged in the last decade to tackle these integration challenges. Over the last few years, many organizations have made significant investments in these EAI solutions. As a result, business integrations in the EAI space tend to be locked into a single vendor and the integration components are tightly coupled.

The cost of maintaining these proprietary integration links is a significant burden. Specialized skills are required, exacerbating the cost and stability concerns. Furthermore, ripping-and-replacing existing EAI solutions is not a plausible alternative for an organization protecting massive investments in EAI.

BPEL addresses all these problems by delivering a standards-based, platform-neutral solution. The loosely coupled BPEL process eliminates vendor lock-in, reduces integration costs, and provides interoperability; it also adds a sophisticated layer of security, exception management, and logging. Most important, companies can leverage their existing infrastructure, service-enable it, and orchestrate it using BPEL.

In this installment of "The BPEL Cookbook," we will present an architectural blueprint for leveraging Oracle BPEL Process Manager to develop new integration solutions as well as interface with existing ones. It includes a case study in which orchestrated Web services must be integrated with existing heterogeneous EAI solutions based on TIBCO BusinessWorks and webMethods.

Furthermore, because any business process implementation is incomplete without a carefully designed error management, security, and logging framework, we'll explain how BPEL scopes and compensation handlers can make this process more robust and fault-tolerant, as well as how to secure the BPEL process and participating services.

Case Study Background

EAI is an excellent driver for service-enabling existing applications; existing middleware processes can be exposed as Web services, which are then orchestrated via BPEL.

The following diagram illustrates a generic approach in which Oracle BPEL Process Manager is used to orchestrate existing EAI interfaces as well as to integrate new applications. This approach assumes that the middleware can expose the business processes as Web services and that the application servers themselves have Web service interfaces.

Figure 1 Oracle BPEL Process Manager in EAI—a complete picture

An analysis of a specific case study involving two traditional EAI middleware products demonstrates how BPEL can play an important role in integrating the two products.

In many organizations, changes made in one customer "system of record" are not populated to the other systems in which customer data is maintained. Consider a scenario in which an enterprise, for various business reasons, ends up using two middleware products from different vendors, such as Siebel CRM integrated with TIBCO BusinessWorks and SAP R/3 integrated with webMethods (see Figure 2).

Figure 2 Customer Details Management Module and middleware interfaces

In this model, inconsistent customer data across the SAP and Siebel systems can adversely affect customer service levels and drive down department and organization revenues. Rather, consistency can be maintained through a common Customer Details Management Module that has multiple interfacing points with the TIBCO and webMethods integrations. For example, when Siebel receives customer data, it checks whether the customer is a new or existing one, and then either adds new data to SAP or updates existing customer data in both apps.

You could use the existing middleware tools (TIBCO and webMethods) to achieve this integration, but doing so would simply increase the footprint of proprietary integration and as well as vendor lock-in. This situation represents an excellent opportunity to service-enable the applications, thereby achieving a standards-based, vendor-agnostic solution.

The first step toward achieving a standards-based interface in EAI is to expose the process as a Web service. Most middleware platforms can talk to each other through Web services, but the scenario becomes complex when a set of interfacing services must be glued together with appropriate business logic.

You could use another middleware process or even complex Java code to orchestrate Web services. But the process would have to provide following capabilities:

• Parallel Web service invocations
• Asynchronous Web service invocations
• Loose coupling between services and portability
• Exposing the whole orchestration with standards-based interfaces
• Orchestration monitoring

Figure 3 Customer Details Management Module with Web service interfaces

Introducing BPEL into this scenario has the following potential advantages:

• BPEL supports loosely coupled Web service orchestration.
• Business logic (even parallel flows!) can be represented in simple XML tags.
• Data can be easily routed among services with simple assign ( copy rules) and invoke statements.
• The Customer Details Management Module can be invoked as an independent Web service component from another orchestration, middleware tool, or Web application.
• Processes can be managed via a simple GUI, such as with Oracle BPEL Process Manager.

Now let's consider how Oracle BPEL Process Manager can be used to achieve this synchronization of customer data across SAP and Siebel.

Implementing the Customer Details Management Module

BPEL plays an instrumental role in automating the customer data synchronization process between SAP and Siebel. The steps involved in implementing this BPEL process are:

1. Expose TIBCO and webMethods processes as Web services.
2. Orchestrate Web services using a BPEL process.
3. Add exception management capability to the BPEL process.
4. Secure communication between Oracle BPEL Process Manager, application adapters, and the EAI tool.
5. Centralize the logging and notification process.

Here are the steps you would take to expose the BusinessWorks process as a Web service:

1. Analyze functionality offered by your BusinessWorks process, to determine whether it can be an independent component in the integration scenario.
2. Determine the process input and output.
3. If input and output are complex, use W3C XML schemas (XSDs) to define them. You can use XSDs to define your custom fault schema as well.
4. Create the WSDL, using the WSDL palette, and define input and output message formats (associate them with predefined XSDs, if required). You can import existing WSDL as well.
5. Configure an HTTP Connection resource.
6. Use a SOAP Event Source as the first activity, followed by business logic, and then use a SOAP Send Reply to expose the process as a service.
7. Associate the HTTP Connection resource with the Event Source .
8. Associate WSDL and Send Reply with the Event Source .
9. Handle possible exceptions, and use SOAP Send Fault for sending exceptions to the service client.
10. If your machine name is mymachine, the port used for the HTTP Connection resource is 8000; if the process name is SOAPService , your service can be accessed using the URL http://mymachine:8000/SOAPService.
11. Get the final WSDL of the service from the WSDL tab of the Event Source activity.

1. Check whether your webMethods Flow Service can be an independent component in the integration scenario.
2. Specify "Execute ACL to Anonymous" under on the Permissions tab if you want to invoke this Web service from outside the webMethods environment without any authentication.
3. Select the Flow Service in webMethods Developer, and click on Generate WSDL from the Tools menu.
4. While generating the WSDL document specify the protocol (SOAP-RPC/SOAP-MSG/HTTP-GET/HTTP-POST) and transport mechanism (HTTP/HTTPS).
5. Define the target namespace for the WSDL document.
6. Enter the IP address or name of the machine where webMethods Integration Server is hosted in the Host field.
7. In the Port field, use the port number to connect to the current Integration Server.
8. Save the WSDL document to the local file system. You can find the service endpoint in the generated WSDL document.

Now, suppose you have the following three TIBCO Web services (implemented with TIBCO BusinessWorks processes and TIBCO Adapter for Siebel).

• Siebel Add
• Siebel Update
• Siebel Query

• SAP Add
• SAP Update

Figure 4 Solution architecture

As seen above, BPEL process brokers the conversation between the front-office call center and back-end SAP and Siebel CRM applications via EAI tools.

Here are some best practices for exposing middleware processes as Web services:

• Adhere to WS-I standards whenever possible.
• Try to expose services in "document" style with literal encoding. If that option is not available, use "rpc" style with literal encoding. Although either is recommended by WS-I, you will find document style easier, at least when you create copy rules for a BPEL assign activity—with rpc style, all the schema elements come as separate message parts, whereas with document style, the entire message is delivered in one part. You can use a single copy rule to copy the entire schema, thereby easing the development effort, and verify the style and encoding in the final WSDL document.
• Avoid the use of SOAP encoding; the SOAP Action attribute of WSDL is an empty string.
• Confirm that the middleware uses WSDL 1.1 to describe the interface of the Web service.
• Use HTTP binding of SOAP.
• Make sure that all XSDs used for describing the schema conform to the XML schema specification proposed by the W3C. For example, global element declarations should not use references to other global elements; that is, use the type attribute instead of the ref attribute.

Previously we mentioned that an important aspect of the Customer Details Management Module is that it synchronizes customer data between SAP and Siebel. This process is visually created in BPEL Designer as shown below.

Figure 5 Customer Details Management Module orchestration using Oracle BPEL Process Manager

Here's a summary of this process flow:

1. A receive activity accepts the customer details (enterprise schema).
2. Details are passed to Siebel through assign and invoke ( Siebel Query service) activities.
3. Via a pick activity, the Siebel Query result is used to determine whether the customer is existing or new.
4. If the customer is new, parallel execution is invoked, with flow activity adding the customer in both Siebel and SAP; otherwise, another parallel flow updates customer details in both applications.
5. If the customer details are not present in SAP, those fields are fetched from Siebel by Siebel Query . This and the other SAP fields to be updated are passed to SAP Update , possibly via a set of assign copy rules.
6. The final status of Customer Update/Addition is returned by use of an explicit reply activity.

This BPEL process addresses the business requirements of customer management, but it still can't handle exceptions. For example, what would happen if a customer were added successfully in Siebel but failed in SAP? To address that issue, you need to enable exception management in your business process.

Step 3: Add Exception Management Capability. Exception management allows a BPEL process to handle error messages or other exceptions returned by outside Web services and to generate error messages in response to business or runtime faults.

The following table contains exceptions that need to be handled to make the customer management BPEL process more robust.

Except for 1, 2, and 6 (see below), cases needn't be handled explicitly to keep the data consistent.

It is necessary to track the status of the Web service to catch exceptions and take appropriate actions. Before a discussion of how cases 1, 2, and 6 are handled, let's see how the status of a specific Web service is tracked.

The entire BPEL process has these reply schema attributes:

• Siebel_Add_Status
• Siebel_Update_Status
• SAP_Add_Status
• SAP_Update_Status

Now, let's see how exceptions are managed:

Case 1
If the Siebel query itself fails, the process should be terminated and the invoking client can retry.

Case 2
When customer-details insertion fails in Siebel and succeeds in SAP (they occur in parallel), data consistency will be lost. Moreover, in the case of a retry, the following problems may occur:

• If you try to invoke the BPEL process to insert customer details into Siebel, customer details may be duplicated in SAP.
• If you try to invoke the BPEL process to update customer details, update in Siebel will fail, because there is no corresponding record.

The scope and compensate activities are key BPEL development tools. A scope is a container and context for other activities; a scope activity is analogous to a {}block in a programming language. A scope simplifies the BPEL flow by grouping functional structures, and provides handlers for faults, events, and compensation as well as data variables and correlation sets.

Oracle BPEL Process Manager provides two constructs for handling compensation:

• Compensation handler—This handler is the business logic for achieving rollbacks. Handlers can be defined for processes and scopes.
• Compensate activity—This activity invokes the compensation handler on an inner scope activity that has already successfully completed, and it can be invoked only from within a fault handler or another compensation handler.

Returning to our example, let's assume that our BPEL process has two scopes: an inner one and an outer one. Invoke activities for SAP Add and Siebel Add services come under outer scope, but only SAP Add comes under the inner scope. A compensation handler can be associated with the inner scope, and the compensation handler invokes activity for the SAP Delete service.

You have to associate a catch block with the outer scope so that a SiebelAddfault sent by the BusinessWorks Web service can be caught. Whenever SiebelAddfault occurs, a compensate activity compensates the inner scope and the SAP customer record is deleted. Note that compensate will be successful only if all the activities in the inner scope have been successful.

A modified BPEL process with scopes and compensation handlers is shown in Figure 6.

Figure 6 Compensation logic to delete SAP customer record

Case 6
The transaction also fails if the Siebel update fails and the SAP update succeeds. This leads to data inconsistency; thus, you have to use compensation logic to roll back the transaction that occurred in SAP. The compensation handler is associated with the SAP Update service and will invoke the SAP Rollback service. Modification of the BPEL process will adhere to the guidelines identified above.

The ability to explicitly invoke the compensate activity is the underpinning of the error-handling framework of BPEL. Unlike traditional EAI compensation mechanisms, BPEL offers a standardized method of dealing with rollbacks.

Having created a BPEL process to orchestrate TIBCO and webMethods Web services, let's see how can we make the communication between BPEL, adapters, and EAI tools more secure.

Step 4: Secure Business Communication. Security can be enabled at two levels: Outbound (invoking secure TIBCO and webMethods services) and inbound (securing the BPEL process).

Outbound Security
TIBCO and webMethods services need to be secured to prevent unauthorized access. Oracle BPEL Process Manager supports HTTP basic and WS-Security authentication for calling external services. This example focuses on how to secure the TIBCO and webMethods services and mechanisms to call them from the BPEL process, using HTTP authentication.

Web services deployed in TIBCO BusinessWorks and webMethods Integration Server support HTTP basic authentication. While designing the TIBCO Web service, check the Use Basic Authentication checkbox on the Transport Details tab of the SOAP event source activity. While deploying the Web service by using TIBCO Administrator, you can set service access levels for different users/roles. When designing the Web service with webMethods Developer, you can set the ACL (Access Control List) for different operations.

If you use basic authentication while deploying the TIBCO and webMethods services, the services will expect them in each invocation and the credentials should be passed by the BPEL process. This task can be achieved by setting two Partner Link properties: httpUsername and httpPassword . The value of these properties can be set statically as follows.

Figure 7 Setting httpUsername and httpPassword

To pass the credentials dynamically, use a copy rule.

<copy>
  <from variable="varUsername"/>
  <to partnerLink="p1" bpelx:property="httpUsername"/>
</copy>

Inbound Security
BPEL processes can be secured from invocation by unauthorized users using HTTP authentication; it is also possible to set different credentials for different BPEL processes.

To enable this security, HTTP basic authentication has to be enabled at the application server level. Credentials can then be extracted in the BPEL process and passed to TIBCO and webMethods Web services as Partner Link properties.

For more details about BPEL security, listen to the "Securing BPEL Processes & Services" Webinar on OTN.

Step 5: Centralize Logging and Error Handling. Your business processes are now secure and robust, but it is equally important to build logging and notification capability into them. Building a centralized logging and error handling framework will make the application even more robust, increase reusability, and reduce development costs. You can build such a framework by using a Web service from the BPEL process as well as from middleware. Via Oracle BPEL Process Manager, you can add logging to this service with the file adapter and require that an e-mail notification be sent in case of error.

The following sample schema can be used for the framework:

This BPEL process can be exposed as an asynchronous one-way Web service, which makes the clients of this service continue operations without any delay. The centralized logging and notification process LogNotify is shown below.

Figure 8 Logging and error handling framework using Oracle BPEL Process Manager

As shown in Figure 8, the LogNotify process does the following:

1. Receives information to be logged from an external process
2. Invokes Log Service to write this data into a log file. Log Service leverages the file adapter for this purpose.
3. If logging is successful, the process completes. In addition, if the ROLE field contains ERROR , the service will notify the relevant person via e-mail. E-mail information is retrieved from the original message received (see the sample schema).
4. If notification fails, the process ends after the notification error has been logged in a file.

Figure 9 shows how the LogNotify process is invoked when Siebel Add fails.

Figure 9 Invoking the logging and error handling framework from the Customer Details Management Module

Conclusion

The integration market is flooded with powerful EAI products, with which lots of integration has already been done. BPEL represents a unique choice for service-enabling these existing EAI solutions. By exposing existing middleware processes as Web services and orchestrating them with Oracle BPEL Process Manager, organizations can embark on the route of SOA.

Arun Poduval also works in the EAI practice of Infosys Technologies Ltd., specializing in similar technologies.