As Published In
Oracle Magazine
May/June 2009


Test for Success

By David Baum

Prevent errors, reduce downtime, and save money with Oracle Application Testing Suite.

When IT professionals absolutely, positively have to get it right—whether they’re helping an educational software company improve its application lifecycle or they’re running the Web site of a charitable organization that raises funds online—they test. Making sure that all applications run smoothly in all kinds of conditions is mission critical, and the way to ensure that applications perform as needed is to test them—rigorously and repeatedly—under real-world conditions.


Meridian Knowledge Solutions

(a subsidiary of Visionary Integration Professionals)

 Location: Chantilly, Virginia
 Industry: Software and services
 Revenue: US$13 million
 Employees: 80
 Oracle products: Oracle Application Testing Suite, Oracle Database

Comic Relief

 Location: London, England
 Industry: Charity
 Revenue: US$136 million
 Employees: 120
 Oracle products: Oracle Load Testing for Web Applications, Oracle Database, Oracle Application Server, Oracle Data Guard, Oracle Real Application Clusters

Application testing is becoming increasingly important as customers seek to deploy applications faster, at lower cost, and with higher quality of service. Quality assurance (QA) professionals want application testing solutions that prevent costly application performance problems, avoid unplanned outages of business-critical applications, and automate the manual steps involved in application testing.

These QA goals were key for Meridian Knowledge Solutions, which wanted to establish processes that would make its entire application testing lifecycle more efficient. The company, a subsidiary of Visionary Integration Professionals, provides learning management system (LMS) and learning content management system software. Meridian serves 4.5 million users at more than 200 public and private sector employers. For this educational software company, testing isn’t something that their end users do after they’ve mastered the material. Testing starts at home.

“We didn’t want to simply develop our testing processes around an application,” says James Downs, product QA manager at Meridian. “We wanted to define processes that would make us more efficient so we could better manage our application testing lifecycle from beginning to end.”

As part of defining their processes—as well as managing application testing lifecycles—QA professionals at Meridian asked themselves a simple question: how can we become more efficient in the QA department while driving our testing processes forward?

To better serve its customers and keep the LMS performing optimally, Downs and his team follow proven methods for control processes, testing, documentation, and review procedures, as well as creating templates and guidelines for QA deliverables. Oracle Test Manager for Web Applications, one component of Oracle Application Testing Suite, helps them bring all these pieces together while enabling collaborative communication through a common portal.

The biggest advantage of the Oracle software, according to Downs, is its simplicity: all fundamental areas of the software lifecycle are available in a simple and intuitive interface.

“Oracle offers three important testing modules integrated into one interface,” says Downs. (For a description of the testing modules, see the sidebar, “Oracle Application Testing Suite: Putting It All Together”.) “Requirements, test management, and defect tracking are all bundled together, which is fantastic for traceability purposes. When you’re trying to itemize things, it is easier to have them all in one place.”

One of the most time-consuming QA tasks for Meridian involves creating traceability between requirements, tests, and defects. “By allowing users to associate requirements to tests, tests to issues, and issues to requirements, Oracle Test Manager for Web Applications provides traceability and mapping that aids in our CCB [change control board] process, issue resolution, and testing preparation and execution,” Downs says. That functionality has big payoffs for Meridian. 

Oracle Application Testing Suite: Putting It All Together

The Oracle Application Testing Suite, part of Oracle Enterprise Manager’s suite of application quality management products, provides an integrated solution for load testing, functional testing, and test management. It enables customers to test packaged, Web, and service-oriented architecture-based applications and their underlying infrastructure, and it helps ensure optimum quality, scalability, and availability. Oracle Application Testing Suite includes three components.

Load testing for Web applications simulates thousands of virtual users accessing the application simultaneously and measures the effect of the load on application performance, thus validating the performance and scalability of applications and Web services.

Functional testing for Web applications provides for comprehensive functional testing of applications and Web services by simplifying automated test script generation and enabling the automation of complex Web applications and associated technologies out of the box.

Test manager for Web applications helps manage Web application testing by allowing users to define testing requirements, specify and execute manual or automated tests to validate requirements, and manage the defects that those tests uncover.

Oracle Application Testing Suite also provides a common scripting environment between functional testing and load testing to simplify test processes and accelerate test completion by as much as 50 percent.

Melinda-Carol Ballou, program director of IDC’s Application Life Cycle Management Service and a principal analyst at the independent research firm, says, “Oracle’s acquisition of the Oracle Application Testing Suite functionality [acquired with Emperix’ e-TEST suite] is an important first step toward developing an overall lifecycle strategy for Oracle Applications.”

Indeed, Oracle is in the process of integrating and enhancing Oracle Application Testing Suite functionality in several of its applications lines, including Oracle E-Business Suite and Siebel Customer Relationship Management (CRM). “We are leveraging Oracle’s extensive knowledge of these applications to generate test scripts more efficiently and to provide best-of-breed solutions for testing Oracle Applications,” says Leng Leng Tan, vice president of applications and systems management at Oracle. For example, the Oracle Application Testing Suite accelerators for Siebel let customers jump-start their Siebel application testing by providing automated load and functional testing for Siebel customer relationship management applications. “Oracle Application Testing Suite integrates with Siebel Test Automation Interfaces to natively recognize Siebel UI elements and correlate application requests when creating automated test scripts, greatly simplifying the testing process,” says Tan. “The framework allows customers to focus on testing their Siebel applications rather than creating test scripts and greatly reduces the overall cost of testing.”

“Oracle’s traceability capabilities alone save our team days, if not weeks, on every release,” says Downs. “These capabilities systematically cover our functionality and prevent a lot of upkeep and overhead.”

Meridian’s application testing procedures are built both for today’s requirements and with an eye to the future.

“We focus on making sound decisions for all areas that we want to fix, from requirements management to development to QA to the change process,” explains Downs. “It’s all about repeatability. You want to devise procedures that are cyclical, consistent, and predictable.”

No Laughing Matter

Consistent and predictable application performance under extreme conditions is paramount for Comic Relief, a U.K.-based charity whose vision is “a just world free from poverty” and whose mission is “positive change through the power of entertainment.” Established in 1985 by comedians with a passion for helping others, each year Comic Relief runs a major fundraising campaign—either Red Nose Day or Sport Relief, which are held in alternate years.

During these campaigns, Web site performance is critical. Comic Relief uses Oracle Load Testing for Web Applications, one of the three primary Oracle Application Testing Suite modules, to define and manage a comprehensive testing process that includes validating the functionality of its Web applications and ensuring that those applications can perform well even when the organization experiences huge spikes in traffic.

Comic Relief’s Web application testing processes ensure that people visiting the organization’s fundraising sites can get information and make donations quickly. “Oracle Load Testing for Web Applications includes infrastructure performance monitors that record metrics on Web servers, application servers, databases, and other infrastructure components,” says Charlotte Melén, Web technology manager at Comic Relief. “That’s been really useful.”

Comic Relief’s test campaigns follow a set process that begins with performance testing and tuning and then moves on to load and stress testing. Oracle Application Testing Suite helps the QA team define and manage these testing processes, validate application functionality, and ensure that Comic Relief’s Web properties perform under the anticipated demand. Comic Relief’s testing procedures include simulating 30,000 simultaneous users and measuring the effect of the load on application performance.

During extensive load testing leading up to Red Nose Day 2007, Oracle Load Testing for Web Applications was instrumental in ensuring that the online donation application would be able to handle traffic processed by 9,800 call center agents. “Thanks to our systematic testing efforts, we were confident that our infrastructure would be able to handle the tremendous demands it would experience during the event,” Melén says.

The site performed as planned. Comic Relief processed more than 450,000 transactions during a seven-hour period. During peak loads, the site processed an average of 54 transactions per second. Comic Relief is planning for three times that amount of traffic for the 2009 Red Nose Day campaign. “We’ll be testing for peaks of 150 transactions per second to ensure we can handle an expected 750,000 transactions in six hours,” Melén says. “This will allow us to walk into Red Nose Day 2009 with confidence that the Web site will perform under pressure.” 

Next Steps

 DOWNLOAD Oracle Application Testing Suite

Melén believes that testing thoroughly is the key to application success. “I would advise anyone putting in place a business-critical Web application to test thoroughly,” she says. “It’s a key quality assurance measure to show whether or not an application will perform under traffic. We certainly could not run our campaigns without such rigorous testing.”

James Downs from Meridian Knowledge Solutions acknowledges the importance of application testing and the value of the Oracle testing tools to his team: “Oracle has provided a relatively low-cost solution that supports almost all aspects of our product development lifecycle.”

David Baum
( is a freelance business writer based in Santa Barbara, California.

Send us your comments