Using Asynchronous Servlets to Deal with Hung Threads
by Francesco Marchioni
12/04/2007
An example of using the Abstract Asynchronous Servlet
Imagine an application that outputs the stock exchange quotations. We suppose that stock values are obtained using a JCA interface, which, in turn, accesses a banking system. The customer needs feedback within a few seconds. However, if thousands of people are connected within the same time period, it's likely that the response will be delayed, and, in the worst scenario, this will lead to a HTTP connection timeout.
But the damage can go even further: If customers notice that the site is not responsive, they will go on clicking the refresh button, flooding the application server with new requests.
It's time to put a brake on this. Well, the bad news is that if the backend resource is stuck, we cannot actually satisfy the customer's request. However, if we cache the latest stock ticker along with the time it was gathered, we could produce this information inside our timeout callback, pointing out the time at which this data was collected. In our example, we set the timeout to 2500 milliseconds. If the response is not produced by that time,
doTimeout() will kick in. Figure 3 shows our architecture.
Figure 3. The
doTimeout() callback is triggered when the servlet is not notified in time
Let's look at the servlet in more detail:
public class AsyncServlet extends AbstractAsyncServlet {
// Need JDK 1.5
Hashtable <String,Long> hashStocks = new Hashtable<String,Long>();
Date dateLastUpdate = new Date();
public void init() {
// The default request timeout period in milliseconds.
super.setTimeout(2500);
// The default interval period in milliseconds at which
// requests will be checked for timeouts.
super.setScavangeInterval(1);
// Populate hashtable with Stock initial value
hashStocks = readStocksValue();
}
public boolean doRequest(final RequestResponseKey rrk)
throws ServletException, IOException {
HttpServletRequest req = rrk.getRequest();
HttpServletResponse res = rrk.getResponse();
// Read Stock quotation from Back end
hashStocks = readStocksValue();
TimerManagerFactory.getTimerManagerFactory().getDefaultTimerManager()
.schedule(new TimerListener() {
public void timerExpired(weblogic.timers.Timer arg0) {
try {
// This will trigger delivery of the response.
AbstractAsyncServlet
.notify(rrk, null);
} catch (Exception e) {
e.printStackTrace();
}
}
}, 1000);
return true;
}
public void doResponse(RequestResponseKey rrk, Object context)
throws ServletException, IOException {
HttpServletRequest req = rrk.getRequest();
HttpServletResponse res = rrk.getResponse();
// Display fresh quotations
displayStockQuotations(res);
}
public void doTimeout(RequestResponseKey rrk) throws ServletException,
IOException {
HttpServletRequest req = rrk.getRequest();
HttpServletResponse res = rrk.getResponse();
// Hung thread. Recover quotations from the cache
displayStockQuotations(res);
}
private Hashtable<String,Long> readStocksValue() {
Hashtable<String,Long> hash = new Hashtable<String,Long>();
hash.put("XOM", new Long(75));
hash.put("GE", new Long(1005));
hash.put("JPM", new Long(1008));
dateLastUpdate = new Date();
return hash;
}
private void displayStockQuotations(HttpServletResponse res) {
PrintWriter pw = null;
try {
pw = res.getWriter();
} catch (IOException e) {
e.printStackTrace();
}
pw.println("Quotations at :" +dateLastUpdate);
Enumeration<Long> enumStocks = hashStocks.elements();
while (enumStocks.hasMoreElements()) {
pw.println(enumStocks.nextElement());
pw.flush();
}
pw.close();
}
}
The main benefit of this approach is that we provide an answer without keeping the execute thread queue too busy. This will contribute to a more scalable application and improve the customer experience.
This approach has been kept intentionally simplistic; after all, the real purpose of this article is to illustrate a feature with a trivial example.
In a real stock system we could add a bit of complexity by customizing the answer based on the type of customer. For example, stock brokers or users who have subscribed a contract with our stock system might have longer timeouts, or no timeout at all. For all other users, it could be acceptable to deliver quotations that are some seconds stale.
Javascript Hacking
The advent of Ajax applications has deeply changed the style of Web applications. Applications built using Ajax produce richer and more dynamic Web sites, like Google Maps, MySpace, Gmail, and the Netflix site. The sites do a lot of work behind the scenes so they are less about users filling out one form after another, and more about the application automatically giving the user the information he or she needs.
Unfortunately, common Ajax best practices have not been developed, which leaves plenty of room to get things wrong.
The problem is that Web 2.0 sites are vulnerable in a way that Web 1.0 sites aren't: Every keystroke or mouse click can be transformed into an execute thread that is consumed. So, given the nature of Ajax applications, this could lead to a nasty mechanism that can exhaust WebLogic Server with many short-lived HTTP requests. No surprise that several pieces of malware already exploit this problem, with just a couple of lines of code. In the trenches of sys admins, they have even coined the term Javascript hacking for this kind of attack.
Future Servlet to the Rescue
The future servlet has been around already in WebLogic Server 6.1, but this functionality has been “hidden” inside WebLogic Server, until the release of WebLogic Server 9.1, where BEA has finally made this model public. So let's examine our future servlet model to see how it can help us. To make use of this servlet, implement the weblogic.servlet.FutureResponseServlet class.
Instead of implementing doRequest(), doResponse(), and doTimeout() methods, all we have to do is define a service method that handles servlet requests:
public void
service(HttpServletRequest req, FutureServletResponse rsp)
If we decide to provide an immediate answer, all we have to do is call the send method on the FutureServletResponse instance. However, if we want to delay the response in the future, we must provide an extension of the TimerTask class. Here's an example:
Timer timer = new Timer();
ScheduledTask mt = new ScheduledTask(rsp, timer);
myTimer.schedule(mt, 100);
......
public class ScheduledTask extends TimerTask {
private FutureServletResponse rsp;
Timer timer;
ScheduledTask(FutureServletResponse rsp, Timer timer){
this.rsp = rsp;
this.timer = timer;
}
public void run() {
try {
PrintWriter out = rsp.getWriter();
out.println("This is a delayed answer");
rsp.send();
timer.cancel();
} catch(IOException e) {
e.printStackTrace();
}
}
But how can one leverage this class in a real use case? Let's go back to our Ajax paradigm. When a new request arrives, if we follow our traditional model, we should provide an immediate response to the client and sit back waiting for a new request. But here is the catch: If we push the client's request in a buffer, returning immediately to the client, we could then process the request asynchronously in a different thread and send the response to the client only when the backend process has terminated.
In other words, instead of client-server polling, we process the request on the server and we simply notify the client when processing has terminated. Decoupling the request from the response can prevent the execute from being too busy, leading to a more scalable application. It can be also used as a defense against Ajax flooding requests.
But how will the client know when server-side processing has terminated? When we deal with event-driven programming, the Observer Pattern kicks in (Figure 4).
For those who still aren't familiar with this pattern, here is a short introduction: This pattern relies on three actors: the Subject , the Observer, and the Concrete Observer.
Figure 4. The Observer pattern
- The Subject provides an interface for attaching and detaching observers.
- The Observer defines an updating interface for all observers, to receive update notification from the subject.
- The Concrete Observer maintains a reference with the Subject, in order to receive the state of the subject when a notification is received. It contains this function that will be executed eventually.
Here's a practical example of this approach:
public class FutureServlet extends FutureResponseServlet {
long time = 100;
public void service(HttpServletRequest req, FutureServletResponse rsp)
throws IOException, ServletException {
PrintWriter pw=null;
try {
pw = rsp.getWriter();
} catch (IOException e) {
e.printStackTrace();
}
pw.println("Request arrived");
pw.flush();
new Subject(req,rsp);
}
}
class Subject extends Observable {
private static Stack <FutureServletResponse>
stackResponses = new Stack <FutureServletResponse>();
public Subject (HttpServletRequest req, FutureServletResponse rsp){
PrintWriter pw=null;
try {
pw = rsp.getWriter();
} catch (IOException e) {
e.printStackTrace();
}
pw.println("Request sent to server");
pw.flush();
addObserver(new ConcreteObserver());
stackResponses.push(rsp);
startListening();
}
public void startListening() {
Thread t = new Thread() {
public void run() {
while (true) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
if (stackResponses.isEmpty()) {
continue;
}
FutureServletResponse rsp = stackResponses.pop();
setChanged();
// Notify event to Observer attached
notifyObservers(rsp);
}
}
};
t.start();
}
class ConcreteObserver implements Observer {
public ConcreteObserver() {}
public void update(Observable o, Object arg) {
FutureServletResponse rsp = (FutureServletResponse) arg;
PrintWriter pw=null;
try {
pw = rsp.getWriter();
pw.println("Response sent at :" + new Date() + "
");
pw.flush();
rsp.send();
pw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
As you can see, when a new request kicks in, its request and response are stored in a stack. Then the Subject class registers as observer the ConcreteObserver class, which holds the business functions to be performed when notification occurs.
Using this simple but effective approach we could scale Ajax applications, reducing the impact on the execute thread, which can now serve even more requests at the same time.
Note that BEA says that this model "gives you full control over how the response is handled and allows more control over thread handling. However, using this class to avoid hung threads requires you to provide most of the code." They recommend the Abstract Asynchronous Servlet approach in most cases.
Conclusion
Under ideal testing conditions, network timeouts don't occur and the amount of effort that has to be expended to handle timeouts makes it tempting to ignore them. However, when requests arriving at the server start growing, poorly written Web applications may give rise to overloaded servers and eventually stall, causing a network client to block indefinitely.
For these reasons, it is necessary to prevent hung-thread proliferation. The Abstract Asynchronous Servlet provides hooks to decouple response from incoming requests and timing out unresponsive requests.
However, if you need to schedule your response at a certain time in the future and you need full control over thread handling, you can adopt the Future Response Servlet interface.
References
- Ajax Security Basics from Security Focus
Francesco Marchioni joined the Java community in 1997 and is certified as a Sun Enterprise Architect. He is an employee of Pride SpA and has designed and developed many J2EE applications on the Weblogic Platform for BEA Customers.