SPECjAppServer2004: The New Way to Evaluate J2EE Performance

by Samuel Kounev
09/05/2005

Abstract

This article presents SPECjAppServer2004—the new industry-standard benchmark for measuring the performance and scalability of J2EE hardware and software platforms. SPECjAppServer2004 is a completely new benchmark and not comparable to the SPEC J2EE benchmarks released in late 2002. This article discusses the business domains and workload modeled by the benchmark, as well as the benchmark design and architecture. The author also explains the meaning of the benchmark metrics, discusses the different purposes the benchmark can be used, and provides some links to additional information.

Introduction

SPECjAppServer2004 is a new industry-standard benchmark for measuring the performance and scalability of Java 2 Enterprise Edition (J2EE) technology-based application servers. SPECjAppServer2004 was developed by SPEC's Java subcommittee, which includes BEA, Borland, Darmstadt University of Technology, Hewlett-Packard, IBM, Intel, Oracle, Pramati, Sun Microsystems, and Sybase. It is important to note that although some parts of SPECjAppServer2004 look similar to SPECjAppServer2002, SPECjAppServer2004 is much more complex and substantially different from previous versions of SPECjAppServer. It implements a new enhanced workload that exercises all major services of the J2EE platform, including:

  • The Web container, including servlets and JavaServer Pages
  • The EJB container
  • EJB 2.0 container-managed persistence
  • JMS and message-driven beans
  • Transaction management
  • Database connectivity

Moreover, SPECjAppServer2004 heavily exercises all parts of the underlying infrastructure that make up the application environment, including hardware, JVM software, database software, JDBC drivers, and the system network. Server vendors can use SPECjAppServer2004 to measure, optimize, and showcase their products' performance and scalability. Their customers can use it to gain a better understanding of the tuning and optimization issues surrounding the development of modern J2EE applications.

SPECjAppServer2004 Business Model

The SPECjAppServer2004 workload is based on a distributed application claimed to be large enough and complex enough to represent a real-world e-business system. The benchmark designers have chosen manufacturing, supply chain management, and order/inventory as the "storyline" of the business problem to be modeled. This is an industrial-strength distributed problem that is heavyweight, mission-critical, and requires the use of a powerful and scalable infrastructure. Most important, it requires the use of middleware services, including object persistence, caching, distributed transactions, clustering, load-balancing, resource pooling, asynchronous messaging, and dynamic Web page generation, among others. It is those services of application servers that are stressed and measured by the SPECjAppServer2004 benchmark.

The SPECjAppServer2004 workload has been specifically modeled after an automobile manufacturer whose main customers are automobile dealers. Dealers use a Web-based user interface to browse the automobile catalogue, purchase automobiles, sell automobiles, and track dealership inventory.

As Figure 1 depicts, SPECjAppServer2004's business model comprises five domains:

  • Customer domain dealing with customer orders and interactions.
  • Dealer domain offering a Web-based interface to the services in the customer domain.
  • Manufacturing domain performing "just-in-time" manufacturing operations.
  • Supplier domain handling interactions with external suppliers.
  • Corporate domain managing all customer, product, and supplier information.

SPECjAppServer2004 Business Model
Figure 1. SPECjAppServer2004 business model

Figure 2 shows some examples of typical transactions run in these domains (the dealer domain is omitted, since it does not provide any new services on itself).

Business domains
Figure 2. SPECjAppServer2004 business domains

We include a brief overview of the four domains as they are described in the benchmark documentation:

The customer domain

The customer domain hosts an order entry application that provides some typical online ordering functionality. The latter includes placing new orders, retrieving the status of a particular order or all orders of a given customer, canceling orders, and so on. When orders arrive from customers, a credit check is performed on the customer by sending a request to the corporate domain. In addition, the proper discount is calculated. Orders for more than 100 automobiles are called large orders.

The dealer domain

The dealer domain hosts a Web application (called dealer application) that provides a Web-based interface to the services in the customer domain. It allows customers, in our case automobile dealers, to keep track of their accounts, keep track of dealership inventory, manage a shopping cart, and purchase and sell automobiles.

The manufacturing domain

The manufacturing domain hosts a manufacturing application that models the activity of production lines in an automobile manufacturing plant. There are two types of production lines, namely planned lines and large order lines. Planned lines run on schedule and produce a predefined number of automobiles. Large order lines run only when a large order is received in the customer domain. The unit of work in the manufacturing domain is a work order. Each work order is for a specific number of automobiles of a certain model. When a work order is created, the bill of materials for the corresponding type of automobile is retrieved and the required parts are taken out of inventory. As automobiles move through the assembly line, the work order status is updated to reflect progress. Once the work order is complete, it is marked as completed and inventory is updated. When inventory of parts gets depleted, suppliers need to be located and purchase orders (POs) need to be sent out. This is done by contacting the supplier domain, which is responsible for interactions with external suppliers.

The supplier domain

The supplier domain is responsible for interactions with external suppliers. It decides which supplier to choose based on the parts that need to be ordered, the time in which they are required, and the price quoted by suppliers. The company sends a purchase order (PO) to the selected supplier. The purchase order will include the quantity of various parts being purchased, the site it must be delivered to, and the date by which delivery must happen. When parts are received from the supplier, the supplier domain sends a message to the manufacturing domain to update inventory.

The corporate domain

This domain manages the global list of customers, parts, and suppliers. Credit information, including credit limits, about all customers is kept solely in a database in the corporate domain. This is to provide maximal security and privacy.

Pages: 1, 2

Next Page »