High Performance Message Processing with BMT Message-Driven Beans and Spring
Pages: 1, 2, 3, 4, 5

Disadvantages of the Traditional Model

Although robust and successful, this transactional message consumption model has some serious drawbacks. First of all, distributed transactions add a significant performance impact to the processing (it is not unusual to see a 50 percent performance degradation when switching from local transaction to XA).

A second disadvantage is that because of CMT, the actual transaction commit happens outside of application code, after the return from the onMessage() call. This may not seem like a big deal (after all, the whole idea of CMT is to relieve the application from handling transactions), but there are some unpleasant implications—some error conditions won't be detected until the transactions commit. For example, in BEA WebLogic Server, by default, all DML operations resulting from manipulations with CMP beans (create, update, and so on) are deferred until transaction commit time. This means that an application can think it successfully updated an instance of a CMP bean, while in fact the actual SQL update would fail because of some constraint violation in the database. The worst part is that application code won't be able to react to this or just log it properly because it would never see that exception.

Proposed Optimization

Although there is a workaround for delayed DML operations—for example, in BEA WebLogic Server it can be disabled in the deployment descriptor—it comes with a performance penalty. The J2EE server will no longer be able to aggregate and/or batch SQL updates for more efficient execution, or skip them altogether if the transaction is later marked for rollback later.

This article proposes that embracing a bean-managed transaction (BMT) approach can provide the same quality of service, with far more control over transaction life cycle. Application code would have the opportunity to recover and/or report errors much more clearly, while avoiding all disadvantages of the CMT model described above. Moreover, we expect a significant performance gain as a result of removing message retrieval from the transaction scope.

Before we look at the BMT approach, we need to analyze what would happen with message consumption from the queue in this case. If we deployed the MDB with BMT demarcation, the J2EE server would no longer enlist the JMS destination (queue or topic) that the MDB is listening on into the transaction (the transaction will be started after the message is picked from the queue). In this case, the BMT MDB should be configured with a non-XA connection factory in the deployment descriptor; otherwise the J2EE server will fail to deploy it.

According to the JMS specification (JMS 1.1 section 4.5.2), if a message listener is deployed non-transitionally with AUTO_ACKNOWLEDGE or DUPS_OK_ACKNOWLEDGE modes and RuntimeException or any subclass of it is thrown from the onMessage() method, then the message will be redelivered. In other words, it is possible to redesign our use case to use BMT, and if something goes wrong during message processing, the application code can throw a RuntimeException, and the message would be redelivered (retried). This approach works pretty well, because it's just natural to use RuntimeException to indicate unrecoverable errors (for example the Spring Framework's exception hierarchy is almost entirely based on subclasses of RuntimeException). The message will be redelivered up to a certain number of times (configurable at the MOM software level) after which it's usually discarded or moved into a dead message queue, or alternatively the application code could count the number of times the message was redelivered and decide when it's time to stop trying to process and either consume it without processing (generating an error message if appropriate) or move it into the separate queue.

As you can see, the behavior described above guarantees that we have control on message redelivery, and it's possible for the application to retry in case of repeated processing failures. We are going to show how we can guarantee once-and-only-once behavior in the BMT case.

Non-transactional Message Consumption Model

Let's now look at the sequence of events in a non-transaction message consumption model:

1. A message is consumed from the queue/topic and passed to the onMessage() method of the MDB.
2. The application starts a BMT transaction.
3. The MDB processes the message. Any transactional resource that the MDB will use while processing the message (such as database, other queues or topics, and JCA adapters) will be enlisted in the BMT transaction started in Step 2.
4. The application has full control of the BMT transaction. It can commit or roll back the transaction based on any business logic necessary. The simplest case would be to roll back the transaction if there are any exceptions thrown from the business logic message processing code, and commit it otherwise.
5. If the onMessage() call returns successfully (without a RuntimeException being thrown), then the message will be acknowledged and removed from the queue.

You probably notice that logically, the sequence is somewhat simpler, although the implementation could be a little bit more complicated than the plain CMT case. The payback here is significant flexibility in handling message processing. For example it's up to the application code to either commit or roll back the transaction, and it's independent from message acknowledgment. Note that the message is not acknowledged until the successful return from the onMessage() method call.

Bean-managed vs. Container-managed Transactions and Required QoS

So far so good, but let's see if we can guarantee the same QoS (once and only once) as when using the transactional message consumption model. We need to analyze two things:

1. If the message could ever be lost as a result of our message processing.
2. If the message could be processed more than once.

The first item in the list above is very simple. As evident from our discussion, since a message is not acknowledged until after a successful return from the onMessage() we just need to make sure that our application code is not swallowing exceptions where it shouldn't and that we're rolling back transactions in the case of any exception thrown. This is a pretty natural approach.

The second item in the list above is just a bit trickier. As we have seen, if an exception is thrown during business processing, the message will be redelivered. It's not a problem if we are following the paradigm described above: If there is any exception, roll back the transaction and re-throw the exception further, to the J2EE server. Of course, action should be taken to prevent infinite message delivery if the error is truly unrecoverable. This could be done at the MOM level (for example, dead message queue, redelivery limit) or, as we are going to show, this could be handled on application level.

In the case of a previous processing attempt terminated with a transaction rollback, message redelivery is not a problem. From the application perspective, the next delivery attempt should be treated as a new message (the result of the previous processing attempt of the same message is not visible because the transaction was rolled back), so we are fine in this case without any special processing rules.

It's important to analyze as well what would happen in case of catastrophic failures of different components—for example if the J2EE server crashed—is the system going to be in consistent state, from our required QoS point of view, after restart? Any failure before message acknowledgment (which happens on the very last step) is going to result in message redelivery. Therefore, we need to take care of only one failure scenario, namely when the J2EE server, or MOM, or connection between them fail after our code successfully finished processing and committed the BMT transaction but before the message was acknowledged by the J2EE server. This would leave the system in an inconsistent state: From the application point of view the message was successfully processed, but from the MOM point of view the message hasn't been successfully delivered (it was not acknowledged because of the failure). This could result in duplicate message delivery and processing, violating our QoS contract. Arguably, since the time interval between transaction commit and acknowledgment should be very short, this failure scenario should not happen very often. Nevertheless if your system is under heavy load, then there could potentially be dozens if not hundreds of messages processed simultaneously, and we should take this into account when building our recovery mechanism.

It's also important to emphasize once again that we are concentrating on the most strict QoS level (once and only once), and therefore recovery from duplicates in such rare cases are needed. If your use case could be developed with more relaxed QoS levels (for example, once or more), this recovery mechanism we are about to describe could be skipped.

To summarize, in order to ensure a once-and-only-once QoS while using a non-transactional message retrieval with BMT, we need to have control over the message redelivery behavior (which is achieved by throwing a RuntimeException from the onMessage() method as per the EJB specification) and some mechanism to prevent processing duplicate messages in the event of failures during certain stages of message processing (between the return from the onMessage() call and the message acknowledgment). We're going to discuss prevention of processing duplicate messages next.