Write for OTN
Earn money and promote your technical skills by writing a technical article for Oracle Technology Network.
Learn more
Stay Connected
OTN Architect Community
OTN ArchBeat Blog Facebook Twitter YouTube Podcast Icon

Speedy Perception versus Speedy Reception: Smart Asynchronous Interactions

by Lucas Jellema Oracle ACE Director

Leveraging opportunities for asynchronous processing to improve system response and scalability.

March 2014

Downloads
download-icon13-1 Oracle SOA Suite

Picture the following scene at your favorite restaurant. The dinner party is seated at one of the tables and the people sitting around the table are conducting a lively conversation with sparkling interaction. Then, conversation at the table dies when the waiter is taking an order. Even when the waiter has written down all requests and takes his leave to go and fulfill the request, everyone continues to be quiet until the waiter returns with the orders. There's no interaction at all while the wait goes on.

jellema-async-processing-fig01
Figure 1: Synchronous Restaurant Interaction

The waiter meanwhile goes to the kitchen to deliver the order-and he keeps hanging around, waiting for the order to be prepared, instead of waiting on another dinner party. Only when the waiter has delivered the order from the kitchen to the guests at the table is he released to start waiting on another table. And only when the order arrives at the table will conversation among the guests resume.

That would be a somewhat strange restaurant, soon to be no longer your favorite. From an economical perspective, it is also horrible: tying waiters up with one table until they have served the request. I do not know of any but perhaps the fanciest restaurants that work like this.

Yet in many parts of today's application architectures, this highly synchronous model is very much in use for all questions, even those that are really more one-way messages than two-way questions. It applies, for example, to most HTTP requests from browser to server, to Java web tier to Java business tier and, of course, to Java Database Connectivity (JDBC) requests from the middle tier (Figure 2).

jellema-async-processing-fig02
Figure 2: The Synchronous Model

As a consequence of the synchronous approach, many threads and associated resources are frozen while some other tier is processing a message. This approach of "ask a question then hold your breath until the response comes in" is neither smart nor necessary. Not smart because of the resource waste and reduced responsiveness of the application, and not necessary because of some recent technological developments around off-loading workloads, asynchronous processing requests and push-style communication. This article describes how, by leveraging opportunities for asynchronous processing, at least the perceived responsiveness of our systems is likely to improve. Frequently, the actual scalability is also enhanced by making much better use of the parallel processing power of today's CPUs and virtual machines and through throttling peak loads by deferring some of the work.

In real life, when you tell something to a person, you don't mind waiting for answer (beyond some form of acknowledgement that your message was heard and understood) if and when:

  • You asked a question that you want an answer to
  • You cannot meaningfully continue with your life until you receive the response
  • You do not have to wait too long (a little subjective)
  • You will lose the opportunity to receive the answer at all if you do not wait for it
  • You can afford to wait (because of the time or cost of resources involved with waiting)

However, if these considerations do not apply, you want to continue with your life as soon as possible after saying your piece or asking your question. In the world of multi-tier applications, very similar considerations apply. Replace "you" in these bullets with "the browser" (talking to the middle tier) or the middle tier (talking to the database), or an application component (talking to a web service), and you have a checklist that will help you decide when interactions must be synchronous and when perhaps they could be processed asynchronously.

One-Way Conversations

In the case of fire-and-forget interaction (a one-way communication) in which the party that started the conversation is not interested in an answer-and don't we all have family members that fit that bill-the request is made synchronously and, ideally, the resulting work is taken off-line or handed to another thread as soon as possible. It is therefore useful to take a look at the mechanisms we have at our disposal for handing off jobs in each of the tiers our Fusion Middleware applications typically make use of.

Client Tier

Looking at the browser-the world of JavaScript and HTTP requests-we have the situation illustrated in the figure below. We discern between the foreground thread and the background threads. Until recently, JavaScript was strictly single-threaded. However, this single thread can distribute its attention between the user interface-events caused by the end user by navigating, typing and clicking-and operations performed in the background. A background operation can be initiated through a timer, a combination of a function expression and a time-out period; the time-out period can be zero (meaning that the background task should start forthwith).

jellema-async-processing-fig03
Figure 3: Client Tier

Recent browsers have introduced a true multi-threaded model for JavaScript, as part of what is commonly referred to as HTML5. This multi-threaded model allows the initiation of WebWorkers in a JavaScript program. These workers are typically associated by the browser (aka, the JavaScript VM) with a real operating system thread, one that runs in parallel to the main browser thread. The worker threads execute JavaScript code. They can send messages-with intermediate results-to the main thread, which can update the user interface through the Document Object Model (DOM APIs). The workers cannot access the DOM directly.

Using a WebWorker, a JavaScript function can truly offload its work to a background process that executed the task asynchronously. The main thread is freed to continue interaction with the end user, offering at least the perception of incredibly speedy processing.

Both the main thread (through an HTML form submit, a document reload or in a JavaScript function) as well as parallel WebWorker threads can make HTTP requests to the server. When the request is made using the XMLHTTPRequest object, the call is asynchronous (AJAX). This means that the thread that issued the call can continue processing while the HTTP response has not yet been received. When the response arrives at the browser, the requesting thread is interrupted and the designated call back function is invoked by the browser. When no call back function is specified, an arriving response goes ignored. We have thus created a one-way conversation, not by eliminating the answer (which would have been the more elegant approach but is not in line with how HTTP works) but because we did not listen to the answer.

Middle Tier

We assume a Java middle tier; after all, this column is on Fusion Middleware. Requests to the Java middle tier will usually arrive over HTTP, although of course Remote Method Invocation (RMI) and more indirect and inherently asynchronous channels (such as email or file/FTP) are also an option.

Figure 4 illustrates many mechanisms available on the Java, and especially the Java EE, platform for offloading work from one thread to another, thereby releasing the initial thread to immediately return a response to the invoking party. That gives the impression of speed: very fast processing seems to have taken place or at least the wait time for the consumer minimized.

jellema-async-processing-fig04
Figure 4: Middle Tier

All work performed by a thread is done synchronously; the initiator of the call waits for the thread to complete. For example, in Figure 2 the HTTP client caused a Java thread to start executing a servlet to handle the request. Until this thread completes the HTTP response, the thread is locked and the HTTP client is waiting. Completion of the thread can be accelerated in several ways:

Executor: Using the concurrency facilities in Java SE (and, starting with Java EE 7, also in Java Enterprise Edition), a task can be handed to an executor that runs independently of the thread that executed it. Various executor implementations may make use of the multiple cores available to the JVM and can be Java EE container-managed.

(Quartz) Scheduler: A scheduler can be used to take on responsibility for executing a task; the task execution is done in the background, on a different thread from the requesting one. Several scheduler libraries are available, of which Quartz is probably the best known.

EJB Asynchronous Methods: Introduced in EJB 3.1 (Java EE 6), Enterprise JavaBeans can have methods that process invocations asynchronously. This means, in practice, that when the method is invoked, the container steps in and invokes the method in a different thread based on use of the @Asynchronous annotation. The interaction is fire-and-forget as far as the calling thread is concerned.

EJB Timers: In EJB 3.0 (Java EE 5) the Timer Service was introduced. In an approach similar to the timers we discussed above that work in JavaScript as a mechanism to offload processing to a background job, the EJB Timer Service could be used for that. Create a timer that expires immediately and have it perform some task. The Java EE container ensures that the execution of the timer's task is performed on a background thread.

Java Batch: Very recently, Java EE 7 introduced the Java Batch API, which the latest generation of Java EE containers should implement. Through this facility, a standardized framework to schedule and run predefined jobs is guaranteed to be available in every Java EE 7 container.

One-Way Web Service Call: When the work a thread has to perform can be invoked through a one-way Web Service call-one that immediately returns an HTTP-200 response without any payload-then the thread can trigger the job execution by calling the Web Service and quickly returning.

Work Manager: Before the release of the Java EE Concurrency Utilities in Java EE 7 under JSR-236, several Java EE containers offered a Work Manager API, an abstraction for dispatching and monitoring asynchronous work.

JMS Queue or Topic: A very robust mechanism to decouple threads on the Java EE platform is JMS (the Java Message Service). Java threads running inside the Java EE container can acquire access to a JMS Queue (single consumer) or Topic (potentially multiple consumers) and publish a message. This operation is fire-and-forget. Once published, the message is the responsibility of the Java EE container. The container routes the message to the subscribed listeners, typically Message-Driven Beans that will be invoked in a dedicated processing thread.

Grid: Grids come in various shapes, including compute grids that can be engaged for processing potentially huge jobs. The help of a grid can be enlisted in several ways, including JMX, REST and other remote communication protocols. One approach to be mentioned is the Coherence Write-Behind scenario, in which data changes are captured fail-safe in the in-memory grid and pushed to the persistent data store (typically a relational database management system) at a later point in time.

Database Tier

The third tier in our discussion of getting rid of the work that has to be done is the database tier. Figure 5 shows several entry points into the database, some synchronous and some asynchronous. It also visualizes some mechanisms inside the database that allow one session to communicate to another (background) session and have that background session execute the work.

jellema-async-processing-fig05
Figure 5: Database Tier

The usual channel from Java middle tier threads into the database is through JDBC. From a connection pool, the middle tier thread has been allocated a dedicated database connection with a dedicated database session-the database equivalent of a Java thread. Any statement issued over this JDBC Database Connection is executed synchronously, blocking the connection and the issuing Java thread. If we manage to transfer the work load from the database session associated with the JDBC connection to some other session, then we have achieved our goal of asynchronously processing the work.

The database contains a job scheduler. Through either dbms_job or dbms_scheduler, PL/SQL units can create and schedule jobs, typically PL/SQL programs. These jobs are executed by background processes. Initiating a job is a prominent way of offloading work from a database session.

Pipes are non-transactional, in-memory communication channels that can also be used to transfer messages (and units of work) from one session to another. Using the supplied package dbms_pipe, text messages can be written to a pipe. Any database session can read from that pipe and start processing whatever message was read.

A global application context is somewhat similar to the pipe, in that it is an in-memory area that can be written to from PL/SQL not bound by the scope of transactions. Unlike a pipe, whose contents are removed upon read, the application context will retain its value until it gets explicitly removed. Note that no events or triggers can be associated with an application context and the update of its contents. That means a form of background polling would be required to actually start processing a message.

Another asynchronous interaction channel is formed by Advanced Queues (AQ). One session can publish a message that represents a unit of work on a queue; this completes the session's involvement. Asynchronously, a message handler can be invoked in a background job to process the message. Note: a WebLogic JMS Queue or Topic can be backed by an AQ. This means that publishing a message on a JMS Queue results in a message being published on the underlying AQ queue. And that means that a PL/SQL message handler associated with that queue could start asynchronously processing the message from the JMS publisher, opening up yet another asynchronous Java => (JMS => AQ =>) PL/SQL channel.

Of course, inserting a record to a regular database table is another way of transferring information from one session to another. Note, however, that an insert trigger will fire in the same session that executed the DML statement, which means that any work done by the trigger is done synchronously, blocking the thread that issued the JDBC call. A polling job could keep an eye on the table, register new records and act upon them.

As an aside: the Oracle Database supports something called "asynchronous commit"; this is a type of commit (introduced in release 10gR2) that does not wait for redo to have been completely written to disk (to the online redo log). A normal commit forces a disk I/O and does not return until the commit is complete. You will save some time if you instruct the database to do an asynchronous commit and not wait until the commit is guaranteed. You do run a risk, of course: if the database crashes before some buffered redo is written to the logs, or a file I/O problem prevents the log writer from writing buffered redo to a non-multiplexed log, then the commit is lost.

The Oracle Database is capable of accepting resources through FTP and WebDAV (in the XML DB Repository). Such a transfer of a resource ends with the creation of the resource in the database. As a result of the resource creation, events are published. In reaction to these events, background jobs can start some action, such as processing the resource.

Another asynchronous communication channel into the database is through email. Even though the database does not have native mechanisms to consume email, it is fairly simple to create a Java Stored Procedure that will do exactly that. Such a procedure could be scheduled to periodically check a mailbox and, for example, put every email's contents in a table or a queue.

One special type of job carried out by the database scheduler is file watching. The scheduler can be instructed to watch specific directories for new files. Whenever a file arrives in the watched directory, a PL/SQL program is started in background job.

Oracle SOA Suite: BPEL and Service Bus

Web Services published from the Service Bus are invoked in a synchronous manner. The Service Bus itself does not do asynchronous interactions. It can do one-way communication, where after accepting the request message, it will issue an HTTP 200 response code and conclude the conversation. If we want the service to return a proper SOAP/XML message based on some initial processing and validation, we can use a common approach to have part of a synchronous interaction turned into a deferred, asynchronous process. Figure 6, below, illustrates this approach.

A Proxy Service is exposed on the Service Bus. This service has a synchronous interaction style, accepting and returning a SOAP message. This proxy service does very limited processing-typically, some validation of the request message to ensure the asynchronous processing is likely to be successful. After successful validation, the message is published to a JMS Queue. This completes the work of the synchronous section of the chain. A SOAP response is returned to the invoker.

jellema-async-processing-fig06
Figure 6: Deferred Asynchronous Process

A second, internal proxy service uses an inbound JMS Adapter to listen to the JMS Queue. When a message arrives, this second proxy service picks up where the first one left of, and starts doing the actual processing-such as synchronously invoking one (or even more) business services.

When using a Service Component Architecture Composite, the options increase, largely because of the capabilities of the Business Process Execution Language (BPEL) engine. For example: a BPEL process can do some work after receiving the request, then send a response and continue processing.

jellema-async-processing-fig07
Figure 7: BPEL Process

The processing performed by the BPEL process after the initial response was sent to the client can be long running. It can involve calls to many business services-for example, using adapters to integrate with database or third-party applications. Some human tasks can be part of the process. BPEL is well equipped to handle asynchronous interactions, such as with humans through the human workflow service, and also with external asynchronous services. The BPEL engine handles the life cycle of the process instance (the BPEL equivalent of a Java HTTP session or a database session). The instance is dormant when no work can be done and responses or timers are awaited, and is revived when a callback delivering an asynchronous response arrives or a timer fires. As a final step, when the result of the work should somehow be made known to the erstwhile client, the BPEL process can invoke a component that is capable of routing the results in a way that the client can find them.

Delayed, Asynchronous Response

Back in our restaurant, the staff has found a way to break through the stifling procedures used before. Now the guests at a table can attract the attention of a waiter, tell him their wishes and then continue with their conversation as the waiter—asynchronously—makes his way to the kitchen. The waiter in turn informs the kitchen staff about the order and then goes his merry way, probably to some other table where they require his assistance. The cooks are asynchronously dealing with the order. The restaurant is more lively and more efficient as a result.

Now, however, we have to deal with the second stage: get the prepared orders from the kitchen to the guests who ordered them. Some simple rules apply (just like in a multi-tier architecture where the lower tier does not communicate to the higher level tiers):

  • Kitchen staff is not allowed in the restaurant
  • Waiters can go to a table only when requested by guests

The challenges for the restaurant staff include finding a way for the kitchen staff to draw the attention of the waiters, helping the waiters find out for which table an order delivered by the kitchen crew is destined and, finally, ensuring that the waiter is summoned to a table when he has the food for that table.

jellema-async-processing-fig08
Figure 8: Asynchronous Restaurant Interaction

Two mechanisms are in use between kitchen crew and waiting staff. The kitchen will ring a bell whenever they have completed an order. The sound of the bell does not, of course, call out to any specific waiter, nor does it indicate for which table the kitchen has prepared an order. The kitchen staff also places all prepared orders in a heated or chilled holding area that the waiters can check whenever they are idling nearby. On each tray, the cooks place a numbered token, corresponding with the order jotted down by the waiter, that bears the table number.

The rule that waiters can approach a table only when invited by the guests to do so proves cumbersome when it comes to speedily delivering food to the tables. Waiters have invented little tricks, such trying to regularly seek eye contact with the guests at a table, or even suggesting to the guests that they should as that the waiter come to the table "as soon as the food is available." Based on that standing invitation, the waiters could go to the table at any time.

Lately, the restaurant started to employ messenger boys-very cheap labor-who constantly move around between waiters and guests to deliver messages back and forth. Whenever a waiter has a dish for a table, he sends one of the messengers to inform the table and have them formally request his presence with the food. This also means that any waiter can serve any table, instead of only the waiter originally attending the table.

When we turn our attention back to our multi-tier architecture, where we have jumped through several hoops in order to have the work performed asynchronously, we now face similar challenges when it comes to getting results back to the requesting party. By offloading the work, we severed the direct, synchronous link that makes it easy (through blocking) to return a response. It turns out that we can use solutions similar to those employed by the restaurant squad for communicating back from the processing tiers and threads to the calling party.

The challenge is two-fold: how to alert a higher level tier in the first place, and how to access the correct party [instance, session, thread, client].

From Database Tier to Middle Tier

The database is typically the end of the line in interactions. So how can we make it the starting point of the asynchronous response?

From within the database we can take some initiatives, similar to ringing a bell from the kitchen to draw the attention of the waiters in the restaurant. Using the Query Result Change Notification feature, which involves Java Objects registering themselves as listener with the Oracle JDBC Driver, we can have the database notify the middle tier, which results in a thread running based on that signal. Using the utl_http package or a Java Stored Procedure, we can initiate an HTTP call from within the database; this call can be handled by a servlet in the Java middle tier. The utl_tcp package allows lower-level communication with external TCP/IP-based servers. Java applications can easily be made to act as partner in a TCP/IP communication, opening up a two-way channel between database and Java middle tier.

jellema-async-processing-fig09
Figure 9: Database Tier to Middle Tier Interactiion

As discussed before, a JMS Queue on the WebLogic platform can use a database Advanced Queue as its underlying store. As a result, messages published to the AQ at the database level are effectively published to the JMS Queue or Topic at middle tier level as well. Alternatively, Java Stored Procedures can be used in the database, along with an impressive collection of Java libraries, to publish to a JMS Queue. Listeners to such a queue are triggered on an independent Java thread by the JVM.

The outcomes of the background jobs in the database can be reported in emails, files written to the file system, or resources stored in the XML DB repository. However, it is up to the Java middle tier to come along and collect these results. This and all other approaches can be equated with the cold and hot holding areas where the cooks put all completed dishes, ready for shipment. From within the database, the results from the asynchronously processed workloads can be made available after having been prepared, but just like the waiters have to come and check out the holding area, a polling thread in the middle tier has to check with one of the holding areas used by the database. These can range from the above mentioned mail box, file system directory and XML DB repository folder, to a plain database table or any encapsulated structure exposed through a stored procedure.

The middle tier can be apprised of completed work in several other ways, such as an asynchronous web service response (which is really just an incoming web service call) or a WebSocket message received by a WebSocket endpoint. One way of dealing with the web socket message is to forward it to a JMS queue.

From Middle Tier to Client Tier

The waiters in the restaurant need to be able to associate orders prepared by the kitchen with the table that requested them. Similarly, the middle tier threads need to be able to determine from the results received or retrieved from the database what original requesting client, thread or session they are associated with. The middle tier also needs to have hooks into the client tier.

Until recently, the communication between client (browser) and middle tier (Java EE application server) was entirely was on HTTP. Since HTTP is essentially a stateless request/response model with the client in control, the middle tier cannot start a conversation to inform the client of anything, very much like the waiters who could approach a table only when explicitly invited to do so. Working around this limitation, the long poll was invented; it's essentially a request from the browser that the middle tier holds under consideration until it actually has information to send to the client. This long-running request ties up resources on both the server and the client, but it does give the impression of the server pushing information to the client. Instead of a long (or a streaming) poll request, the client can also periodically (e.g., every five seconds) submit a request to the middle tier that will typically result in an empty response but sometimes may contain a useful payload.

Today's browsers, as part of the HTML5 avalanche of developments, have support for WebSocket interaction. WebSockets offer a light-weight bi-directional communication channel on top of TCP. Support for WebSockets is available through JavaScript in modern browsers and in Java middle tiers through special WebSocket Server products as well as in several Java EE containers, such as GlassFish 4 and WebLogic 12c. Typically, the web client initiates the WebSocket by contacting the WebSocket endpoint. The client typically identifies itself with a token that is also known in the HTTP session in the server. The WebSocket endpoint keeps track of all open channels to the various web clients. Note that the WebSocket endpoint lives outside the context of HTTP sessions. In Java EE 7, the endpoint can listen to CDI events that can be published, for example, in reaction to messages on a JMS queue or topic, or happenings in an HTTP session. The WebSocket endpoint can publish CDI events when it receives messages, and these events can be turned into JMS messages or listened to in HTTP sessions.

The WebSockets could be said to be the HTML5 equivalent of the messenger boys that were introduced in the restaurant to run around between the stately, expensive waiters and the dinner parties.

jellema-async-processing-fig10
Figure 10: WebSocket Interaction

Another HTML5 mechanism relevant to our story is Server Sent Events (SSE). A listener can be specified through JavaScript in the web client, associating a JavaScript function with an HTTP connection to a URL from which messages of mime type text/event-stream may arrive. Each such message results in a call to the JavaScript event handler. A simple servlet can act as event provider, informing the client through simple text messages of relevant news from the server.

Before the middle tier can start feeding information to the client using the mechanisms described here, the results from asynchronous processing need to be communicated from the Java thread that possesses the information to the WebSocket endpoint as well as the session-specific listener objects. An application-scoped event handler could be set up that is reached from any thread with processing results as a singleton, using CDI Events or through a JMS Queue. This end point holds a list of all channels and for each channel an indication of the associated client-similar to the list of table numbers used in the restaurant.

The event handler can also reference to listener objects in HTTP sessions. When these objects are alive (e.g., because the session is involved in a long poll or in server-side event emitting), they can be notified immediately and subsequently handle the information passed in. Otherwise, the event handler can keep the information for the session until such time as it comes along inquiring after news.

Conclusion

Running a restaurant efficiently turns out to be challenge that bears a lot of similarity to architecting a multi-tier application. Having others do the work for you-in order to asynchronously do the heavy lifting without blocking the requesting party-is one of the premises of restaurants as well as applications. As a restaurant guest, you have the waiter work on your behalf and he in turn has a whole kitchen crew to do the work while he takes care of other guests.

Handing over work-either in the browser, the middle tier or the database-requires an asynchronous model with multiple parallel threads and a way to communicate from one thread to another. This article demonstrated many such mechanisms that help break the synchronous chain by enlisting the help of such another thread. The client-side HTML5 Web Worker in addition to the traditional (AJAX) XMLHTTPRequest is an example of such a mechanism; so is the Java EE Concurrency Utilities on the middle tier along with the Asynchronous EJB methods and JMS for off-loading from a thread. In the Oracle database tier we have identified jobs as well as Advanced Queues and pipes, in addition to files and emails.

Note that when, eventually, a response is still required-with a report on the result of the processing that was performed in the back-some form of identification of the original requestor should travel with the request and eventually with the asynchronous response.

Sending the asynchronous response back up the tiers is not trivial. Just like the kitchen crew may not enter the restaurant proper, the database cannot step up to the Java middle tier to make itself known. Through polling strategies, the middle tier can find out what results the database has to offer. Additionally, several mechanisms allow the database to alert the database, including the AQ-JMS bridge, Query Result Change Notification and HTTP communication. In turn, the Java middle tier can leverage the HTML5 mechanism's Server Side Events and WebSockets to push results to the browser.

Fault handling is one aspect not discussed in this article that changes with an asynchronous model. With completely synchronous interactions, faults occur in real time with respect to the client. They need to be handled or they blow up in the face of the end consumer. (Which, if not gracious, is at least clear.) With asynchronous processing, exceptions take place in the background and will typically not be visible to the requesting party. Some form of fault handling is required, such as administrator alerts or automated recovery, in order to not let the request simply vanish into the void.

Using an appropriate selection of the mechanisms introduced in this article, we can quickly take off-line the work associated with a request, returning a very quick (but largely meaningless) response to the client. When the processing is complete and a more meaningful response has been produced, we can use push-channels to asynchronously deliver the response to the client.

Resources

About the Author

Lucas Jellema has been active in IT (and with Oracle) since 1994. An Oracle ACE Director specializing in Oracle Fusion Middleware, Lucas is a consultant, trainer, and instructor in diverse areas including Oracle Database (SQL & PLSQL), Service Oriented Architecture, ADF, and Java. He is the author of the Oracle SOA Suite 11g Handbook, and a frequent presenter at JavaOne, Oracle Open World, ODTUG Kaleidoscope, Devoxx, OBUG and other conferences.
Oracle ACE Director