SOA, Cloud, and Service Technologies

by Bob Rhubart

SOA experts Thomas Erl, Tim Hall, and Demed L'Her explore the evolution of IT architecture and the emerging role of the IT architect.

Published August 2012

The entire four-part conversation from which this article is drawn is available as an OTN ArchBeat Podcast.

In September 2012, with the dust from the Games of the XXX Olympiad barely settled, IT professionals will gather in London for the fifth SOA, Cloud, and Service Technology Symposium. In this interview event organizer and best-selling SOA author Thomas Erl talks about that event, about the evolving nature of IT architecture, and about the emerging role of the IT architect.

Joining in the conversation is Tim Hall , Senior Director of product management for the Oracle Enterprise Repository and Oracle’s Application Integration Architecture products, and Demed L'Her ,Senior Director of Product Management within the Oracle Engineering team responsible for Oracle SOA Suite, Oracle Service Bus, Oracle WebLogic Integration, and other products. Hall and L'Her are among several members of the Oracle community scheduled to present at the SOA, Cloud, and Service Technology Symposium (see the speakers list).


The Conversation

B. Rhubart Thomas, the title of this event has changed over the years. It started off as SOA Symposium. Then it was the SOA and Cloud Symposium and for 2012 it's the SOA Cloud and Service Technology Symposium, so what's driving that evolution and how does it reflect what's happening in enterprise IT?
T. Erl

That's a great question. It really is a representation of how broad the whole field of service technology has grown since we first began focusing exclusively on service oriented architecture. Over the past number of years, we've seen the emergence of a number of different areas, not just cloud computing or cloud-based, but also areas that are more specialized in terms of what can be considered part of service technology or service sciences or services based computing.

In this year's event, there were just too many different areas that needed to be captured in order to limit the event to its previous scope of SOA and cloud computing. So the theme and title of the event grew to what it is now. At some point it might just become the service technology symposium just to provide a general umbrella term for all these different fields that encompass cloud computing, SOA, semantic web technology, big data, and so on. So it's really just a representation of how the field has grown, how this area of practice has become a much larger part of IT mainstream and the variety and range of topics and synergies that can be explored across those different areas.

B. Rhubart

But despite all of the attention on cloud computing of late, and there's certainly been a lot of that, SOA still gets top billing at the symposium. Is there any significance to that?

T. Hall

No, not really. SOA is still a fundamental part of a lot of these sessions. We open this event up to anything that falls within any of these categories, and this year we received more submissions than ever before. We've had to reject more submissions than we've ever had to and we've had to make room for more. We have nearly 100 speakers now scheduled for the London symposium and it's still very, very interesting and impressive to see how SOA is part of a wide range of top areas.

What we thought originally, in the first three years of this symposium, was that there were more focused sessions that talked just about cloud computing, that talked just about platform as a service, that talked just about service orientation, and so on. We're now seeing many more sessions that just combine or make various references to cloud, to SOA, to different types of service technologies, just because it's become such an integrated environment where all of these technologies can and rightfully should be explored in combination and relation to each other.

When you look at the different trends and the different segments that have emerged in this space, SOA is still the only one that provides a formal set of models and paradigms and principles that are exclusively focused on service technology and services-based practices. The rest of these segments that are explored in this event are all technology centered, so we have cloud computing. We have semantic wave. We have big data. We have other types of services-based areas like good computing and so on. Each of these is primarily based on a technology centric innovation set that we can utilize in any way we choose. We can combine it with other types of methodologies, other types of traditional models. We can build silo-based applications in the cloud just as we have previously in our in premise IT enterprise environment.

So what SOA provides is a means of formalization for these technologies centric service areas. That combination therefore is very intriguing to explore, how can we formalize a cloud environment and make it truly service oriented and position our software as a service offerings to be reusable IT assets, as opposed to repeating traditional approaches that have proved unsuccessful from many organizations and build single purpose silo-based applications that we call services, but put those up on the cloud and expect the fact that they're on the cloud to give us greater business benefits, but then still live the fact we'll have to integrate them and overcome disparity in how they relate to other systems in the future.

So it's something that is more of a foundational type of topic area that acts as a point of reference for a lot of these newer emerging technology centric topic areas.

B. Rhubart

Demed, you're deeply connected with SOA programs and products at Oracle. What's your take on what Thomas just said?

D. L'Her

I totally agree with Thomas. I think SOA is maturing and at the same time SOA adoption is accelerating, so we might not be talking as much about SOA these days as we were maybe two or three years ago, but SOA is maturing. It's not a matter of if I am going to do SOA. It's more a matter of how am I going to do it.

At the same time we've got all these trends and buzzwords in the industry. Three years ago it was cloud. This year it's probably big data, and this is one of the things I want to address. The question we get from many people is, "I'm working on my SOA project, so how is cloud going to impact me. How is big data going to impact me? Are there any touch points between these two trends?"

That's kind of what I'm going to be discussing in my session. I'll talk a little bit about big data, and then talk a little bit about the touch points between service oriented architecture and big data, because there are touch points. It's similar to the way cloud impacted service oriented architecture. Big data is going to have an impact on service oriented architecture as well.

B. Rhubart

Tim Hall, you're another Oracle presenter at the Symposium. What are you going to be talking about?

T. Hall

So around this trend of SOA and cloud, there has emerged a new category called API management. Part of it deals with the fact that we have mobile devices trying to connect to all these capabilities that are being exposed as services either from enterprises or now from software service providers offering them from cloud-based solutions.

I would like us to move beyond the tactics here. I think API management is "cloud washing" for something that's been around for much longer than that, which is SOA governance. So the fact that we're using REST-based APIs now to expose these capabilities does not fundamentally alter the goals and motivations that we had originally from the concept of governance, particularly around the establishment of what those capabilities are from concept through delivery, and the consumption of those capabilities by an audience.

Now whether those audiences are all internal to your enterprise or business partners or now all the way out to mobile developers, I don't think that that changes. What I'd like to have people focus on is not so much SOA governance and API management being sort of two sides of the same coin, but I'd like to change the name. For a long time people have argued that SOA governance is sort of an awkward name, no one wanted to be audited. There's 50% of the world that think, yes, we're going to have to tops down initiative to address this and there's 50% of the world that says that it feels like a heavy weight process that I want no part of.

So what I think we should do is change the name and look at what the motivation here is and at the end of the day, it's really about community. So if we take a page out of the B2B book, trading partner communities, what we're really talking about is community and management. How do we connect all the people that are going to use these capabilities exposed from the cloud? Are they using traditional WS*-based SOA or now REST-based SOA approaches? How are we going to engage that community in the life cycle of the delivery of these capabilities for mobile consumption, integration and other purposes?

B. Rhubart

So let me swing this back around to the Symposium for a moment. Thomas, you mentioned that you had an extraordinary number of submissions for presentations and that, in fact, you had to turn several away. I think you mentioned there were something like 100 presentations this year. What else sets this year's event apart?

T. Erl

There are a lot of high profile organizations that have reached out to the committee and have asked to speak at this event. We're actually feeling quite privileged and when we have members from NASA, the U.S. Department of Defense, the European Space Agency, the Federal Aviation Administration, the Department of Health and Human Sciences, and Veteran Affairs, and there are a lot of public sector organizations that are part of it. That nicely balances private sector and industry contributions that have also been growing, but have been sort of the primary contributors traditionally. So, of course, Oracle and other major vendors in the service technology space, as well as a lot of organizations that represent the actual practitioner community. FedEx, for example, and a number of telecommunication firms and banks—Credit Suisse, and so on.

So if you read the list of participating organizations, it feels like a good three quarters of that list represents the highest ranking public sector organizations and Fortune 100 companies. It really does feel to me, more than ever, like a truly international event. Even though this year it's in London, I think only a fraction of the speakers and contributors are actually from the U.K. I think over 90% are flying in from locations all around the world just to be part of this event. It seems more international than ever before to me. That, plus the fact that it has an increased scope in terms of topic areas and the fact that we have the largest quantity of speakers that we've ever had on the agenda. It seems quite special.

B. Rhubart

Thomas, in addition to your involvement with the Symposium, you're also the CEO of Arcitura, which is an IT training and certification organization that focuses on IT architecture. From that perspective, what forces are shaping the evolution of the role of the IT architect?

T. Erl

Well, the emergence of these different modern contemporary areas of practice is very important because these are no longer niche areas of specialization. They're part of IT mainstream. So any IT architect or practitioner needs to stay current and have a level of proven proficiency with regards to whatever facets of these technology areas or areas of practice they're involved with. That's the whole purpose behind what we do, and that's also one of the reasons that we're involved with these events. Every SOA and cloud computing certification program carried out by SOASchool and CloudSchool is project role-based. They range from architects to analysts to governance and security specialists. With cloud we're getting more into technology-centered areas like virtualization specialists, also governance security.

They're purely vendor neutral. They're overseen by independent education committees, and those same committees are comprised of members from the same type of community that forms the pool of speakers, authors, and subject matter experts—and also the attendees—that participate in these events. So there's a lot of synergy among the different groups and organizations we work with and that take part in these events.

But going back to your original question, service technology is very rapidly moving space. As it grows in its presence within mainstream IT, I think that just amplifies its importance to IT architects and other IT practitioners, especially those who want to stay current and marketable within that profession.

B. Rhubart

Tim, Demed, in your respective roles at Oracle you're both deeply involved in architecture, and I suspect with architects as well. What's your take on the forces are shaping that role?

T. Hall

Honestly there's really nothing new under the sun here. Integration has been a problem that we've faced in IT for 40-plus years. But I think there are new ways to tackle the problem. So the role is being shaped by the things Thomas mentioned at the top, everything from service oriented architecture to the capabilities being moved to a place in the cloud where you may have less control over them. And d big data is now coming in, too, as we're able to capture more and more information we have to figure out what to do with it, either from a historical processing perspective or from a data mining and potentially real time analytical perspective. How do we deal with these large volumes of data?

We had big data on the mainframe. It's just that now it's even bigger and more pervasive and there are more people that potentially have an appreciation of the problem. Not everybody had a mainframe. Those that did and were processing large volumes of data at the time have lived this experience before. But now with the cost of compute power coming down, and the increasing ability to capture these massive volumes of data on a more regular basis, obviously the problem is more pervasive.

So in some ways I think it's a about exploring some of the ways that we've tackled these problems in the past combined with the new technologies that are available to us to maybe solve these problems more rapidly, making sure that we don't forget some of the lessons learned, and then trying to rapidly deliver these systems in a consistent, cost-effective, and repeatable fashion customers, whether it's through a consulting organization or whether you're part of the company that's trying to do these things and respond to business faster.

D. L'Her

Yes, exactly. The integration needs are here to stay, and I think a lot of the new trends we see, like cloud, are actually just adding more entropy to the picture. It's just new end points to integrate with. The idea of a single ERP, for instance, which people were pursuing in past years, has suddenly imploded with the arrival of cloud. Now people have CRM on the cloud and they've got with them applications internally. So the need for integration and service oriented architecture is greater than ever, but I think what is changing, as Tim mentioned, is that requirements are exploding on all fronts. We've got more volume. People have more expectations now that they've been exposed to cloud and other trends. They've got more expectations in terms of time to market. They've got expectations in terms of elasticity and analytics, and everything has to be real time.

So I think the needs remain the same. The expectations from the users are changing are becoming more and more demanding, so the products have to follow.

B. Rhubart

Let's step away from the technological end of this for just a moment and look at the social and cultural end. We've talked about the evolving role of the IT architect. Yet that role remains, arguably, one of the most misunderstood and frequently maligned roles in IT. But there still appears to be a great deal of interest among IT professionals in acquiring the skills and qualifications to function in that role. So what's happening? Why is there that dichotomy?

T. Erl

It's a good question. I don't know why that persists in some cultures or some perceptions from other areas of the IT community. Look at what we can now build and deploy and make interoperate. Look at the scope at which systems, in terms of data processing volumes and functional complexity, are being put together. Look at the evolutionary concerns and responsibilities that come with owning and augmenting those systems in response to business change. How those systems and their underlying environments are designed logically, physically, and how the underlying architecture is defined in relation to the systems, in relation to its supporting infrastructure, are absolutely critical parts of making those systems provide true, ongoing business value.

For there not to be an acknowledgment, or even respect, in relation to the design aspects of that is baffling. Because as with anything you build, the blueprint stage is crucial. What we do see is various methods being used to rapidly get the applications out, tactically focused bottom-up approaches that fully accept the fact that what we're putting out there will be disposable within a period of time. That may be suitable for certain business requirements and for certain situations where circumstances and resources and funding may not warrant designing these types of IT assets for long-term use. But it really does, for many organizations, end up causing much more of burden. It results in much more of an impact on how that organization can move ahead within its business space with an IT enterprise supporting its automation that is comprised of single-purpose applications that were not designed to work with each other, but now are required to when business requirements change. That organization must then resort to the whole integration nightmare, and results in the organization becoming less competitive because IT is more of a burden than an enabler.

That whole experience is something that can be traced right back to the lack of attention to the underlying technology architecture of the enterprise and of the systems that were built within the enterprise. And that can be traced back to a lack of acknowledgment of the importance of the IT architect as a legitimate professional, as a legitimate part of not just project team membership, but also a legitimate part of IT enterprise level staffing requirements.

That oversight, the negligence of that is something that ends up costing organizations big time, especially in the long run. I think culturally it just takes some organizations a longer period of time to recognize that to track that back to what was missing when these systems were first introduced and to realize that it's not just a matter of technology. It's a matter of practice. It's a matter of customizing technology architecture to support that organization's specific business requirements and their business goals.

T. Hall

I think what Thomas is getting at is that in many cases organizations have never learned the lesson, and lesson number one is that there is no single approach that you can use in a repeatable fashion to solve every integration challenge you're going to come across. On the flip side, that should not mean that there are no criteria that you can establish to determine if there is a subset of patterns and approaches that should be used to evaluate your integration projects, so that you can actually establish a coherent architecture that's made up of choices.

The problem is that either the choices are left on the floor, so anybody can pick them up and run with any choice that they want because they don't have that discipline and structure, and that's not being provided by the architects within that organization. Or they've tried to oversimplify the problem and say every integration problem is going to be solved in this one way, which then leads to people saying that your enterprise architecture team is a bunch of ivory tower academics and you're never going to accomplish what you need to.

From our perspective we've certainly seen customers struggle with this and we've tried to provide some examples of decision criteria, looking at when to apply the approaches and principles of service orientation, and trying to do real-time integration versus when to apply batch data-centric integration, because you can deal with both data sets. Frankly, a lot of times those integrations are simpler operationally to build and maintain and to satisfy the business requirements. But it's about understanding those decision criteria and establishing them within the organization, so that they can be repeatable, following architectural principles and guidelines.

What skews the conversation is all the trends that appear. SOA is a trend. Cloud is a trend. So people are trying to jump on those things as the next new answer. Maybe it's human nature. One of the challenges we see with customers doing more and more real time integration is that they end up with architectures that are loosely coupled, systems that can be fault-tolerant and sustain business processes. But then they struggle with, "What I do when I need to upgrade one of the participating applications? Do I have enough architectural isolation that I can do this easily? What other approaches can I take for things like zero downtime migration of those participating applications."

I guess from Oracle's perspective, we have some very interesting technology that allow you to deliver those real time integrations using a sustainable architecture, and also allow you to upgrade and roll those participating apps easily and effectively.

D. L'Her

What strikes me is that the if you're a CIO or CTO, finding the right architect is always your challenge, finding somebody who has all the skills, somebody who is a scientist. Somebody who is very technically savvy. Somebody who is also a pragmatist. Somebody who can navigate the intricacy of your organization and come up with an architecture that is going to fly at the end of the day, and is not going to alienate the development team. This is a very tall order. So at the end of the day the architect's job is a very difficult one. It's probably one of the most precious ones for an IT department for all the reasons that Tim and Thomas have framed.

T. Erl

I think another challenge may be the criteria organizations use to evaluate project team performance. There's still a lot of tactical emphasis on determining what was and was not a successful project. By that I mean, did it come in under budget? Did it come in on time? Did it meet functional requirements? Those tend to be the focal points when projects are assessed, and it's easy to lose sight of the importance of what you're designing and the long term implications of what you're building when you're focusing more on the time line, the budget, and meeting the functional requirements within the parameters of that budget and time line.

It is easier to meet those goals, as you're being assessed, by bypassing proper technology architecture and just getting developers to build something that works as per the immediate functional requirements. If you as a project manager are not being assessed based on the quality of the technology architecture, its compatibility with its surrounding IT enterprise architecture, its compliance to predefined standards, if there's no means of assessing that or regulating that, then there's no motivation to really emphasize that or make that part of the project cycle. Because it'll just use up more time and resources and will go against the other points on which you are being evaluated.

So I think assessment from a strategic perspective is also locking in some IT environments, and that is to their detriment in the long term. Because it's not part of how they assess projects, application delivery projects, then those projects do become very much silo based, single purpose, and then consequently impact the subsequent governance and ownership burden that the IT enterprise assumes. It's that shortsightedness that I think also leads to situations like that, where IT architects are not given the level of responsibility that they should.

B. Rhubart

Thomas, you're the editor of the Prentice Hall Service Technology Series, which now includes something over a dozen titles. What new titles are available?

T. Hall

Every year we try to launch at least one book at the Symposium. This year we're launching SOA with REST, which took about three years to come together. It's myself, Raj Balasubramanian, Benjamin Carlyle, and Cesare Pautasso. It's a deep technical exploration of building service oriented solutions that are REST-based and adhere to various levels with REST constraints. It looks at how REST constraints relate to service orientation principles and SOA design patterns, and at how and where we can position Web centric technology architecture as part of enterprise service oriented solution design. I'm happy it's finally put together because I think it provides a level of detail that is needed for people to understand that convergence, because many still associate SOA exclusively with SOAP based Web services, which of course is a foundational and completely legitimate medium for building services and services on solutions, but it's not the only one. We have to separate the formal models and the paradigm from the technology mediums we can use to build in accordance with those models and principles, so it's good to have documentation on how REST can be used as an alternative medium in addition to whatever else may come out of the industry, regardless of whether it's cloud based.

The other book is Cloud Computing: Concepts and Technology. We're not sure yet, but it looks like Prentice Hall may not have the books at the event in time. But we'll have gallies if we don't of the complete manuscript. It's the first book in the series focused exclusively on cloud computing. It's not about SOA. It's about cloud computing only, just to establish the building blocks, the mechanisms, the models that comprise the cloud computing environment. It's documented using the same conventions, writing style, diagram, notations, symbols and so on as the SOA books have been so far.

So it's certainly a continuation of the library, but before we look into the convergence of SOA and cloud we wanted a title that purely focused on cloud computing. Following that title next year will be another cloud computing title that will be dedicated to cloud computing design patterns.

The third book that we are also hoping to launch is Next Generation SOA: A Real-World Guide to Modern Service-Oriented Computing. That book has contributions from a number of authors, including members of the Oracle community. The book documents service oriented architecture and service technology more at a high level. It discusses what's termed "next generation SOA" in relation to all these recent service-based technology trends, and discusses the implications and where opportunities lie to formalize them using SOA. Conversely, it also addresses where opportunities lie to further the application of service orientation by utilizing these technology resources. That's a very important part of looking at SOA and cloud computing. It's not just about formalizing a cloud environment using SOA based models. It's also about the fact that if we look at the goals of the paradigm itself and the requirements to apply some of these principles, having elasticity and resiliency and having a theoretically infinite amount of resources we can leverage and scale out into, all that provides us with a much greater potential of realizing those strategic goals that come with service orientation.

So that book is more of a high level documentation of that, meant for just about any IT professional. Many of the authors will present at the Symposium, including Ann Thomas Manes, who was one of the co-authors of the SOA governance book that we released at last year's event. Anne will talk a lot about what Tim mentioned, the open API market and how that's emerging into its own segment of IT encompassing a number of these individual areas that we already discussed.

An important distinction there, of course, is also management versus governance and regulation. But just in terms of the publishing that's what's happening right now. There are those three titles that will be rolled out in the short term.

Disclaimer: The views expressed in this interview are those of the participants and do not necessarily reflect those of Oracle.

Bob Rhubart, manager of the OTN Architect Community, is the host of the OTN ArchBeat Podcast, the author of the OTN ArchBeat Blog, and a columnist for Oracle Magazine.