Managing Process and Service Dependencies in a SOA Environment
by Harald Reinmüller and Stefan Kohlmann, Softlab GmbH
Published August
2007
Background
Do you know what services your BPEL processes depend on? Dependencies between BPEL processes can quickly become complex even more, if different versions of these BPEL processes are in place. The complexity of managing the dependencies rises, if we take into account the Enterprise Service Bus (ESB) services, which are invoked by the BPEL processes. This complexity makes deployment and testing time expensive, difficult, and error-prone.
Usually we end up drawing dependencies manually using a modeling tool such as Microsoft Visio—and we are kept very busy updating those dependencies with every change in the processes. This is a major drawback that impedes the agility of a service-oriented architecture (SOA) infrastructure, which is designed for agile change in business processes.
In this Technical Note, we will describe how we successfully improved our build process and, additionally, enabled it to automatically generate images with process dependencies.
|
Our challenge was to implement the pilot project for the Oracle SOA Suite for a customer consisting of many BPEL processes, and referencing many BPEL subprocesses and ESB services. We ended up in a dozen BPEL processes and ESB services—which were defined as public services and shared over the service registry—and a lot of private BPEL processes and ESB services.
First, we decided to create an Ant-based deployment for all services in our project, with the deployment of the BPEL processes, including the execution of their test cases, and the Ant-based deployment of the ESB services to different environments (test, integration, production).
Requirements
After the first version of the project was finished, some requirements and wants came up:
- We did not want to deploy all the services of the project, if a BPEL process or ESB service was changed. So we had to change the deployment from a project-centric to a public-service-centric deployment approach.
- Together with the deployment of a BPEL process, all the dependent (private) subprocesses and ESB services should be deployed automatically.
- To prevent the overwriting of a process in a specific version, which results in the loss of all the instance flow information, processes should only be deployed if they are not already deployed in that version on the server.
- A visualization of all process and service dependences should be created automatically at deployment time without extra effort in maintaining this information. This visualization should look like the following figure.
Open Questions
With all these new requirements, we had the following questions and came up with the following answers:
- Question: Where is the information from the process and service dependencies stored, and do we need to add additional information to create a complete dependency map?
Answer: All services called by a BPEL process are stored in the bpel.xml file in the partnerLinkBinding tags. For called BPEL processes, the version information is coded in the URL, too. For example:
<property name="wsdlRuntimeLocation">
${domain_url}/CustomerAccount_BES/1.3/CustomerAccount_BES?wsdl
</property>
The current version of a BPEL process, which is to be deployed, can be found in the build.properties file. For version maintenance of a BPEL process, we only have to change the "rev property inside the build.properties file.
To distinguish between public and private BPEL processes, a new "type" property was introduced to the client's partner link inside the bpel.xml file with the values "public" or "private," as shown here:
<partnerLinkBinding name="client">
<property name="wsdlLocation">CustomerAccount_BES.wsdl</property>
<property name="type">public</property>
</partnerLinkBinding>
- Question: Is there a tool or framework that can dynamically generate dependency graphs?
Answer: Yes, there is an open source tool, Graphviz (www.graphviz.org), which can generate graphs from input files in text format.
- Question: What are the options for referencing other BPEL processes?
Answer: The options are:
- Reference the default version
- Reference a concrete version
- Reference a UDDI service key (with or without an encoded version information)
- Question: How can all these new deployment requirements be implemented?
Answer: These new deployment requirements can be fulfilled best by a custom Ant task.
Starting the Work
Now that our questions were answered, we started the work by creating a custom Ant task. In this Ant task, the bpel.xml file is parsed. For all partnerLinkBindings tags found, the parsing of the corresponding bpel.xml file is to be started recursively. In addition to this parsing, the current value of the version property ("ref") from the build.properties file of each found BPEL project is extracted. One of the challenges in this recursive parsing was to skip the cyclic dependences.
Our newly introduced "type" property in the bpel.xml file and the recursive parsing fulfilled all our needs for a service-centric deployment.
To get the cherry on the top, we tackled the automatic generation of the dependency graph using Graphviz. To realize this, we had to merge all the dependencies, gathered from the recursive bpel.xml parsing, into one single XML file, as shown in the following.
<?xml version="1.0" encoding="UTF-8"?>
<BPELSuitcase>
<BPELProcess id="Resource_BAS_SetForAccount" src="Resource_BAS_SetForAccount.bpel">
<partnerLinkBindings>
<partnerLinkBinding name="client">
<property name="wsdlLocation">Resource_BAS_SetForAccount.wsdl</property>
<property name="type">public</property>
<property name="version">1.2</property>
</partnerLinkBinding>
<partnerLinkBinding name="RemoveFromAccount">
<property name="wsdlLocation">Resource_BAS_RemoveFromAccount.wsdl</property>
<property name="version">1.5</property>
</partnerLinkBinding>
...
</partnerLinkBindings>
</BPELProcess>
<BPELProcess id="Resource_BAS_RemoveFromAccount" src="Resource_BAS_RemoveFromAccount.bpel">
<partnerLinkBindings>
<partnerLinkBinding name="client">
<property name="wsdlLocation">Resource_BAS_RemoveFromAccount.wsdl</property>
<property name="type">public</property>
<property name="version">1.5</property>
</partnerLinkBinding>
<partnerLinkBinding name="CheckAvailability">
<property name="wsdlLocation">Resource_BES_CheckAvailability.wsdl</property>
<property name="version">1.1</property>
</partnerLinkBinding>
...
</partnerLinkBindings>
</BPELProcess>
<BPELProcess id="Resource_BES_CheckAvailability" src="Resource_BES_CheckAvailability.bpel">
<partnerLinkBindings>
<partnerLinkBinding name="client">
<property name="wsdlLocation">Resource_BES_CheckAvailability.wsdl</property>
<property name="type">public</property>
<property name="version">1.1</property>
</partnerLinkBinding>
...
</partnerLinkBindings>
</BPELProcess>
...
</BPELSuitcase>
This merged XML file is transformed with XSLT into the target format (.dot), which can be used as input for the Graphviz generator to generate a PNG image, as shown here:
digraph structs {
node [shape=record,fontname="Arial",fontsize="10"];
edge [fontname="Arial",fontsize="8"];
rankdir=LR;
labeljust=l;
"Resource_BAS_SetForAccount_1_2" [shape=record,label="{Resource_BAS_SetForAccount}|{1.2|public}",
fillcolor=yellowgreen,style=filled];
"Resource_BAS_RemoveFromAccount_1_5" [shape=record,label="{Resource_BAS_RemoveFromAccount}|{1.5|public}",
fillcolor=yellowgreen,style=filled];
"Resource_BES_CheckAvailability_1_1" [shape=record,label="{Resource_BES_CheckAvailability}|{1.1|public}",
fillcolor=yellowgreen,style=filled];
"Resource_BAS_SetForAccount_1_2" -> "Resource_BAS_RemoveFromAccount_1_5" [decorate=true,label=""];
"Resource_BAS_RemoveFromAccount_1_5" -> "Resource_BES_CheckAvailability_1_1" [decorate=true,label=""];
}
The dependency graph generated from the above DOT file is shown in the following figure:
In the boxes, the name, version, and type of the services are shown. The color of the boxes indicates the type of service. In this graph, we have three public (green) services. In the graph at the beginning of this article, ESB services (white) and private BPEL services (orange) are also shown.
After generating a fancy graph for each public BPEL process, we started to implement the automatic deployment of dependent processes. For this service-centric deployment, the previously generated list of processes is used. The standard deploy target is to be called for each process found in this list. To prevent the overwriting of existing process versions on the server, the deploy target is only called if the process is not already available on the server in the current version.
The availability of a specific process version is checked by opening a HTTP connection to the process WSDL URL and checking the return status code (HTTP status 200 – already deployed, HTTP status 404 – not yet deployed).
The output of our custom Ant task with the recursive parsing, PNG image generation, and service-centric deployment looks like the following example.
...
[mkdir] Created dir: /MyProject/Resource_BAS_SetForAccount/doc
[bpeltask] ** STARTING DeployWithDependencesTask **
[bpeltask] Resource_BAS_SetForAccount-1.2
[bpeltask] ** PARAMETER **
[bpeltask] dotfilename: doc/bpel-recursiv-all.dot
[bpeltask] dotfilenamepublic: doc/bpel-recursiv-public.dot
[bpeltask] deploytarget: deploy
[bpeltask] deployment: true
[bpeltask] overwriting: false
[bpeltask] stoponalreadydeployed: false
[bpeltask] ** RESOVLING RECURSIVLY ALL bpel.xml FILES **
[bpeltask] [Resource_BAS_SetForAccount-1.2]
[bpeltask] [Resource_BAS_SetForAccount-1.2, Resource_BAS_RemoveFromAccount-1.5]
[bpeltask] [Resource_BAS_SetForAccount-1.2, Resource_BAS_RemoveFromAccount-1.5, Resource_BES_CheckAvailability-1.1]
[bpeltask] ** WRITING CENTRAL bpel.xml FILE **
[bpeltask] ** TRANSFORMING DOT FILE **
[bpeltask] ** SERVICE DEPLOYMENT **
[bpeltask] ========================================================================
[bpeltask] SERVICE ALREADY DEPLOYED (HTTP/1.1 200 OK) 'Resource_BES_CheckAvailability'
in version '1.1' http://localhost:8888/orabpel/default/Resource_BES_CheckAvailability/1.1/
Resource_BES_CheckAvailability?wsdl
[bpeltask] ========================================================================
[bpeltask] ========================================================================
[bpeltask] SERVICE ALREADY DEPLOYED (HTTP/1.1 200 OK) 'Resource_BAS_RemoveFromAccount'
in version '1.5' http://localhost:8888/orabpel/default/Resource_BAS_RemoveFromAccount/1.5/
Resource_BAS_RemoveFromAccount?wsdl
[bpeltask] ========================================================================
[bpeltask] ========================================================================
[bpeltask] SERVICE NOT YET DEPLOYED (HTTP/1.1 404 Not Found)
http://localhost:8888/orabpel/default/Resource_BAS_SetForAccount/1.2/ Resource_BAS_SetForAccount?wsdl
[bpeltask] STARTING SERVICE DEPLOYMENT of Resource_BAS_SetForAccount in version 1.2
[bpeltask] ========================================================================
...
Conclusion
This service-centric deployment with the automated generation of dependency graphs fulfilled all our needs regarding transparency of our SOA infrastructure. Because the deployment is completely Ant based, we are able to use it for our nightly and production builds with Luntbuild—our continuous build environment—too.
To support the customer's SOA governance, we are publishing our public processes together with their documentation and the generated dependency graphs in the customer's wiki, as shown here:
This wiki page is generated dynamically by a lookup in the service registry for retrieving the versions of the currently registered public services. The graphs in this wiki page represent links to our source repository (Subversion), which contains the documentation and the full-size graphs of the registered version of our public services. A click on the thumbnails retrieves the corresponding graph directly from Subversion via WebDAV.
Discuss this technical note in the SOA
Suite Discussion Forum or Integration
B2B Discussion Forum.
Harald Reinmüller is a SOA architect and IT consultant at Softlab GmbH in Germany focusing primarily on SOA, BPM, and Java projects. He has been working in the fields of integration, middleware, and many other software technologies for 10 years now.
Stefan Kohlmann is a SOA architect and IT engineer at Softlab GmbH focusing primarily on SOA, EAI, BPM, and Java projects. He has been working in the fields of business process modeling, integration, middleware, and many other software technologies for 17 years now.
|
Bookmark
