Building a SOA Practice from the Ground Up:
An Oracle SOA Suite Case Study

by Brian Jimerson

Published January 2011

Downloads:
Oracle WebLogic Server
Oracle SOA Suite
Oracle Database
Oracle WebCenter Suite
Oracle Content Server
Oracle Healthcare B2B Adapters
Oracle Data Integrator

Introduction

Service-oriented architecture (SOA) is an architectural style that promotes integration of applications through a set of design principles that focus on applications being exposed through well-defined interfaces. Most organizations that adopt a SOA strategy do so through a systematic and incremental approach. They start by exposing business functions through stateless interfaces such as Web Services or asynchronous messaging. They then implement middleware to stitch these services together, and then they plan out a maturity model to gradually bring in more and more SOA capabilities, such as service registries, business process execution language (BPEL) engines, and messaging systems. This is a very pragmatic approach to adopting SOA, and typically results in improved agility, return on investment (ROI), and capabilities.

Every once in a while, however, an organization has a vision to invest heavily in building a SOA infrastructure from the ground up, and is also in the position to benefit greatly from doing so. I recently had the opportunity to work with a health care system that was poised to do just that. This article describes the process involved, building a SOA platform and development practice from scratch.

The Business Need

The client had grown through a series of acquisitions of hospitals and clinics. Typical of health care, each acquisition introduced new health care management solutions that each acquired company had in place for its needs: clinical systems, emergency room management solutions, patient medical records, discharge applications, and so on. Each hospital and clinic had purchased commercial, off the shelf (COTS) applications to meet each need. An inventory revealed a total of nearly 550 applications in use at the various hospitals and clinics.

As these hospitals and clinics were acquired, the client recognized the need to integrate all of those systems into a single, cohesive set of IT solutions. Physicians and clinicians often rotated between hospitals, each with its own set of applications. IT personnel had to support multiple systems that served the same purpose. But the client didn't want to impose a single set of applications on all of the hospitals, as they had all built expertise around the existing applications. Instead, the client wanted to leave the decision of the point systems to the individual hospitals, but still provide a seamless interface between hospitals.

The client's vision included providing first-class web applications to the physicians and patients that comprised their customer base. These applications needed to provide customer functions such as electronic medical records (EMR) and patient orders through a web interface, without regard to the system being accessed. Collectively, these applications were named portals, with specific requirements for a Clinical Portal and a Patient Portal.

The Solution

Product Selection

The client went through an extensive analysis of SOA software vendors, and ultimately decided that Oracle SOA Suite and Oracle Fusion Middleware products provided the best foundation for their needs. Oracle SOA Suite, coupled with the other Oracle products, including Oracle WebCenter and Oracle Database, provided a full complement of products that enabled the client to implement a strong SOA infrastructure, reduce vendor glut, and leverage an industry leader in enterprise middleware.

The product stack chosen for the SOA practice included:

• Oracle WebLogic Server
• Oracle SOA Suite
• Oracle Database
• Oracle WebCenter Suite
• Oracle Content Server
• Oracle Healthcare B2B Adapters
• Oracle Data Integrator

Development Practice

While the product selection process was underway, I began to define the client's new development practice. Since the client had almost exclusively used COTS applications, they had never been involved in the process of developing custom solutions, and therefore had no process or tools to ensure successful delivery of custom solutions.

The definition of the development practice focused on three major areas:

• Methodology, process, and governance
• Tooling (IDEs, source control, etc.)
• Staffing roles and skill sets

I'll briefly discuss these three areas to provide a full picture of the implementation.

Methodology and Process

The first step in building the development practice was to define standard processes and methodologies. Since custom solution development was a new endeavor for the client, it was important to provide an organizational framework that would help them build a successful practice, but at the same time would not require a substantial amount of training or rigorous structure. The goal was to provide a structure that would provide a solid foundation on which they could build as their SOA and development practices matured, yet would provide immediate value.

Oracle Unified Method (OUM) is Oracle's methodology for enterprise software. It is a full lifecycle methodology, including development, operations, and retirement of IT assets. OUM methods fully encompass SOA paradigms such as SOA governance, operations and networking. However, it seemed like a large undertaking to get stakeholders to learn and adopt OUM for a burgeoning practice, and there were concerns that OUM might introduce more complexity than was necessary at this stage.

OUM is rooted in the Unified Process (UP), a well-known incremental delivery process, of which there are several implementations, including the Rational Unified Process (RUP) and the Open Unified Process (OpenUP). OpenUP is interesting process in that it was designed to be low-ceremony, technology-agnostic, and extensible to any IT need. Being low-ceremony made it a good fit for the client, and being a UP and extensible provided a clear upgrade path when the client was ready to adopt OUM.

Tooling

Once the methodology was chosen, stakeholders bought into the process to start creating artifacts based on OpenUP. These initial artifacts were mostly vision and requirement documents that gave structure to the strategy. It was now time to put tooling in place to support the development operations of the practice.

Specifically, the initial needs for tools were:

• Development environments, including IDEs, workstation specifications, and other local developer tools.
• Source and version control.
• Build and configuration tools.
• Document and knowledge repositories.
• Issue and task tracking.
• Design and architecture tools.

The following tools were chosen:

• Oracle JDeveloper for any Composite, BPM or BAM applications (as well as WebCenter / ADF Faces). JDeveloper is the best platform for developing these types of applications, as it has graphical editors and deep integration for BPEL, Mediator, SCA adapters and Human Workflow.
• Eclipse with Oracle Enterprise Pack for Eclipse (OEPE) for standard Java EE applications, such as Web Service applications. Eclipse with OEPE was deemed the best IDE for developing more traditional Java EE applications, given its Maven and Spring support, and the fact that most Java EE developers are very familiar with Eclipse.
• Subversion for source and version control. This is currently the de-facto open-source source control software.
• Apache Maven for building, deploying and dependency resolution for Java EE applications.
• Apache Ant for building and deploying Composite and JDeveloper / ADF applications. Maven support for these is still a work in process, but Ant works well with these applications.
• Trac for issue and task management, document repository, and knowledge tracking. Trac is an open-source issue tracking system as well as a full-fledged wiki. The wiki system was used to provide easy access to knowledge items (lessons learned and how-tos) as well as project artifacts, which were mostly documents.

Roles and Skill Sets

A discussion of the roles and necessary skill sets for this practice could be an article on its own, but in the end there were several technical roles that were deemed necessary for success. These roles are all required for a SOA practice:

• XML / XSD / XSLT specialist
• Networking specialist
• Middleware administration specialist
• Solution architect
• Java EE developer
• Business architect
• Technical documentation specialist

Note that these are the roles that were identified; multiple roles could be satisfied by a single person, assuming the appropriate skill set.

Reference Architecture

Once the products were chosen, and the framework of the development center had been laid, it was time to define the over-arching principles, guidelines, and constraints for participants in the SOA infrastructure. This body of knowledge was collectively called the reference architecture, as it was intended to provide a point of reference for anyone who needed to know the boundaries and guidelines for services, collaborations and solutions.

The reference architecture covered all aspects of the SOA infrastructure, and took form in many documents and diagrams; describing it in detail would beyond the scope of this article. I will, however, cover the salient points of the reference architecture and their justification and importance.

Security

One of the most important concerns for the client was security. Being a health care provider, the client owned a substantial amount of private data, and was subject to regulatory compliance specifications, such as the Health Insurance Portability and Accountability Act (HIPAA).

The client had previously addressed authentication and authorization of end users, such as physicians and clinicians. User information was stored in Microsoft Active Directory (AD) and each point solution used AD at some point to authenticate and authorize users. However, introducing concepts such as service composition and integration created the problem of how to pass credentials to these point solutions and leverage their permission sets.

The decision was made to use Security Assertion Markup Language (SAML) for passing credentials through layers of a SOA-based application. SAML is an industry standard for passing credentials between domains in a loosely-coupled fashion. Oracle WebLogic Server fully supports SAML through the use of policies, and can act as both an Identity Provider (IdP) and Service Provider (SP).

The point at which SAML breaks down, however, is when legacy applications need to provide authentication and authorization. Each legacy application needs to provide security mappings in a proprietary way; using SAML allowed the SOA infrastructure to pass SAML tickets all the way to each individual service.

At the point of authentication and authorization, each application's service interface needed to decide whether to use the SAML ticket, or to use a trust-based authentication scheme, such as a token ID. In essence, each service interface was given the opportunity to use the SAML ticket, but if it couldn't digest the SAML ticket due to technical limitations, the interface could choose to use its own ID and trust that the request was valid.

Figure 1 depicts an example of authentication and authorization using this approach. In this simplified case, the Client (normally a browser) sends user credentials to the User Interface (a web application hosted on WebLogic). The User Interface passes a SAML ticket created by the IdP to the Composite (an SCA composite); the Composite can propagate this ticket to services that know how to use it (External System 1), or the Composite can use a configured token ID for external systems that don't support SAML (External System 2).

Figure 1: SAML Authentication

Enterprise Service Bus

The Enterprise Service Bus (ESB) is a core architectural concept in SOA. The ESB provides many of the core functions necessary to integrate and orchestrate enterprise services, such as:

• Service registry and discovery
• Service mediation
• Data transformation
• Transport protocol abstraction
• Transaction support
• Process choreography

The interaction of these functions is shown in Figure 2.

Figure 2: ESB Connectivity

The ESB is an architectural concept; as such it is comprised of many components. These components may be applications, custom development, or logical entities. Oracle SOA Suite provides all of these concepts through various products, such as Oracle Service Bus, mediators, and SCA adapters.

It is common practice in organizations with large numbers of enterprise services and organizational needs to implement departmental ESBs, and aggregate these ESBs into an enterprise-wide ESB, as depicted in Figure 3. ESBs are logical and therefore need not be implemented on the same hardware and software; they are logically partitioned to provide departmental and enterprise services. Providing multiple departmental ESBs allows departments and lines of business to manage their domain's business services; however it also adds overhead for governance, compliance, and security.

Figure 3: Departmental ESB

Messaging Integration

One of the main purposes of an ESB is to enable system integration through messaging integration. Message integration is a style of integration that allows disparate systems to remain decoupled by communicating through packets of data, called messages. Messages are agnostic in terms of transport and communication protocols, as well as in terms of sending and receiving systems. The services that comprise the SOA infrastructure are encapsulations of participating systems, along with components that allow these systems to communicate via messages.

The ESB provides the facilities to reliably address responsibilities and issues pertaining to messaging. Some of these concerns are:

• Enabling transfer of messages between systems, independent of the systems' underlying technologies.
• Knowing how to route messages from source to destination.
• Enabling transformation of data into formats required by communicating systems.
• Providing endpoints and connectors to 'plug in' new systems.

Service Taxonomy

The basic tenet of SOA is that an organization's capabilities are brought together through the extensive use and composition of atomic services. These services cover the spectrum of business, technical, and operational facilities.

In order to help business units understand the capabilities available in enterprise services, and to aid in management and government of these services, a taxonomy was created to describe the purpose and relationship of enterprise services. This taxonomy allowed clients and business units to discover, query, and describe capabilities and business functions of services. The service taxonomy was aligned with many facets of the SOA infrastructure, including service registries, Oracle Service Bus projects, source control, and even WebLogic managed server naming conventions. This ensured a consistent structure across all facets of the SOA infrastructure.

A detailed description of the taxonomy classifications is beyond the scope of this article. However, the initial structure is illustrated in Figure 4.

Figure 4: Initial Service Taxonomy

This catalog encompassed the initial inventory. The taxonomy will grow as the organization's SOA practice matures.

The Implementation

Once the reference architecture was complete, it was time to start building the infrastructure and SOA solutions.

Infrastructure

The logical infrastructure design that was created followed a traditional 3-tiered model, as shown in Figure 5.

Figure 5: Logical Infrastructure Design

The three infrastructure tiers are:

Data / Integration Tier

This tier includes all databases, legacy systems, and third-party applications that provide data or integration points. Communication with this tier could be through a variety of protocols, such as JDBC, JMS, SOAP, file systems, or even raw sockets. Custom and composite applications are responsible for interacting with this tier.

Middleware Tier

The middleware tier provides all of the infrastructure to expose business and operational services to third-party and presentation tier clients. These services include custom service applications (i.e. Java EE applications), BPEL composites, Human Workflow applications, and BPM applications. This tier also includes service registries and service buses.

All enterprise services that are available for client consumption are proxied through Oracle Service Bus (OSB). OSB was used to provide a number of functions, such as version compatibility, Service Level Agreement (SLA) monitoring, dynamic routing, and security policies. No direct communication with composite or other business services is allowed. Projects in OSB were modeled after the service taxonomy described above.

Oracle Service Registry (OSR) is used for a number of functions, including:

• Dynamic Endpoint Resolution
• Service Discovery and Query
• Service Taxonomy Governance
• Service Versioning
• Service Ownership Documentation

Presentation Tier

This tier includes all systems that provide user interface, content, and presentation logic to clients, including WebCenter for portal, intranet, and extranet solutions, and Oracle Content Server for enterprise content management.

In order to ensure proper management of services, this tier does not consume any third-party services directly. All enterprise and third-party services are proxied through OSB: all communications outside the presentation tier went through OSB. This provides a central point of management for SLAs, performance monitoring, health checks, security, and the like. WebCenter applications also use OSR for endpoint resolution and versioning.

It should be noted that some presentation is physically located in the middleware tier. For example, human workflow queues are served by a SOA Suite-managed WebLogic server. These exceptions were deemed acceptable since they are used only on the organization's internal network. If external access is ever needed, these workflow queues could be exposed by WebCenter through Web Clipping portlets or WSRP, or by Oracle HTTP Server (OHS) through reverse proxies.

Proof of Concept

Once the infrastructure was designed, built, and configured, a proof of concept (POC) was designed to ensure that all of the components functioned as intended. The POC needed to prove the following features:

• SOA Composite
  • SCA adapters
  • BPEL processes
  • Mediator components
• Custom Java EE Web Service
  • WebLogic Server fundamentals
  • JSR-109 Web Services support
  • JTA / JPA support
  • Spring Framework support
  • JMS / MDB support
• OSB Project
  • Business service consumption
  • Proxy service support
  • Fault handling / notification
• OSR Business Service
  • Binding template queries
  • Endpoint resolution

An application was designed to prove that the SOA framework provided all of these functions. The application was contrived to ensure that it used all of these features, but wasn't overly complex to implement. The chosen POC was a fictitious employee management solution. This simple solution had the following components:

1. A Java EE Web Service that provided access to an underlying employee database. This Web Service would allow clients to query employee data, get a specific employee, and save an employee.
2. A SOA Composite that orchestrated interaction with the Employee Service and a JMS Queue to log transactions for auditing support.
3. An OSB project that proxied the Composite, monitored performance, and sent email notifications for service faults.
4. A simple JSF Web Application that interacted with the OSB project for querying, fetching, and editing employees.

The application's design is shown in Figure 6:

Figure 6: POC Application Design

The proof of concept was successful, and was demonstrated to key stakeholders. This was considered a significant milestone, as it proved that all of the software acquisition, planning, designing, and building up to this point functioned as intended, and full featured composite applications could be built rapidly and successfully.

Performance Testing

Performance testing was conducted to provide a benchmark for services, composites, and solutions. The goal of the performance testing was three-fold:

1. Tune the core infrastructure without the overhead of heavy applications running on it.
2. Determine benchmark metrics against which to measure individual services and composites.
3. Determine reasonable and acceptable performance SLAs for solutions.

Since the Employee Management POC application was very lightweight and exercised all of the various SOA components, it was used in performance testing as well. This application was a good candidate as it added very little overhead to the performance of SOA components, each individual component was easily isolated, and the full solution could also be tested.

The approach for performance testing:

1. Test each component individually (Web Service, Composite, etc.), by mocking external communications at each level. For example, the Web Service was refactored to use Spring JDBC unit testing support instead of communications with the database to mimic database interaction. This ensured that only the performance of the individual component was tested.
2. Wire up all of the components into the full solution and test end-to-end performance. These tests took into account the overhead of resources such as networks, serialization and deserialization of data, and other intangibles.

Varying loads were simulated during testing using Apache JMeter, which was also used to record results.

Very high performance was achieved on the Oracle Middleware infrastructure with minimal tuning. For example, the Employee Management service had average response times of less than 0.1 seconds with 100 users invoking an employee fetch operation. This was achieved through some basic JVM memory tuning of the default settings, such as maximum heap space and permanent generational heap space. The service was running on WebLogic Server 10.3.2 on commodity hardware (8 GB RAM, 4 core Intel processors, SATA drives).

Pilot Solution

The final step in the implementation process was to determine a good candidate for a pilot application built on the SOA infrastructure. This application needed the following qualities:

1. It had to be a real application that would be deployed to production, not a contrived one like the proof of concept.
2. It had to exercise many of the functions of the SOA platforms, to prove their function and performance in a production environment.
3. It had to be fairly small in scope, to reduce some of the risk associated with new technologies.
4. It had to have significant visibility, to prove value to key stakeholders.

The chosen pilot solution was collectively known as Physicians Directory. This solution aggregated physician data from multiple sources, such as spreadsheets and databases, and allowed clients to query this data through a single interface. It met all of the defined criteria for a pilot application:

1. Physicians Directory was a real solution that would be consumed by public web sites, as well as internal business applications.
2. It exercised many parts of the SOA infrastructure; a custom Java Web Service to support querying of the aggregated data and a SOA composite with file and database adapters to process the legacy data, transform it to a canonical format, and insert the data into a database.
3. It was fairly small in scope; it was intended as a module to be included in many sites and applications.
4. It had significant visibility in the organization. Aggregated physician data hadn't been available up to this point, and it would be used across the organization and presented to the public.

The pilot application was successfully built and deployed on the SOA infrastructure with minimal issues or problems. The success proved many things to key stakeholders:

• The Oracle SOA infrastructure enabled the organization to quickly integrate legacy systems.
• New solutions can be easily built with existing services and data sources.
• Project and technology risk were greatly reduced by leveraging a mature SOA framework, industry best practices, and best-of-breed tools.
• SOA put the organization in a position to quickly adapt to changes in business needs and provide competitive solutions.

Conclusion

This article describes a real-world example of building a SOA practice from the ground up using the Oracle Middleware, SOA, WebCenter, and database products. It covers the entire process, from the business justification and vision, through planning and design, up to the actual implementation of the SOA framework. The client is now delivering many solutions based on this infrastructure, such as portals, directory aggregations, and integrated business applications.

This was a great project to work on, as it was a chance to build a practice from scratch, and proved the viability of SOA and Oracle Middleware as business enablers for integration and rapid solution delivery. Hopefully you will be able to use information in this article in your own SOA projects.