As Published In Special Edition for Linux 2006

Running the Grid on Linux
By David Baum

At its Austin Data Center, the winning combination of Oracle and Linux brings an enterprise computing grid to life.

For years, when you needed more computing capacity, you bought more-expensive computers. Today, with the advent of grid computing, capacity is added on demand, one inexpensive PC server at a time—and off-the-shelf Linux blades are frequently the servers of choice, explains Monica Kumar, Oracle's director of Linux and open source product marketing. "The combination of Oracle and Linux allows users to consolidate on standardized servers and storage, significantly lowering costs and forming the foundation of a grid-computing infrastructure. As grid computing increases performance and offers higher quality of service, Linux further reduces costs, making it an ideal platform for low-cost, high-performance computing at world-class enterprise data centers," says Kumar. Take Oracle's Austin Data Center (ADC), for example.

At its ADC, Oracle and key partners have brought an enterprise-computing grid to life, and it runs on Linux. In fact, according to Kumar, it's the world's largest Dell/Linux installation, "meaning it provides the benefits of grid—high performance, availability, and reliability, while quickly adapting to changing business needs—all of this with the low cost of Linux," says Kumar.

ADC Stats More than 17,500 operational servers 355 servers on average installed every month World's largest Dell/Linux installation Certified by the EPA as a Green Power Partner, with 30 percent of the energy used coming from renewable sources such as wind and solar More than an acre of raised floor space Hundreds of mission-critical applications hosted for customers on the premises

As you approach Oracle's ADC through the open hills and plains of central Texas, you might mistake the nondescript stucco-clad facility for a traditional office building or warehouse. As you get closer and pass through the perimeter fencing, however, you see that this is no ordinary corporate structure. Embassy-grade gates monitor vehicles entering and leaving the facility. Armed guards patrol the premises, and dozens of cameras constantly record activity. Biometric hand and iris scanners restrict entry to authorized personnel, who must pass one by one through metal detectors and weight-sensitive traps.

Once inside, you gaze through protective security glass at more than an acre of raised floor space upon which rows of computers, storage devices, and networking gear have been carefully configured. You've reached the heart of the Oracle On Demand Grid—an advanced computing infrastructure consisting of more than 17,500 servers—all running the Linux operating system—and 3.8 petabytes of storage capacity. The ADC has two primary functions: Hundreds of Oracle customers house their applications and data here as part of the Oracle On Demand program, and the ADC also hosts many of Oracle's internal production applications, such as global single-instance enterprise resource planning (ERP), new development activities, global education services, and demo software. There are multiple grids within the ADC that serve these different groups.

"The Austin Data Center is what a power station of the future will look like," explains Benny Souder, vice president of distributed database development for Oracle. "Not only is it massive but it's also incredibly precise. Every computing resource is cabled, labeled, stacked, configured, and deployed in exactly the same way, over and over again."

This standardization allows systems administrators to quickly isolate and troubleshoot problems, eliminating downtime so that Oracle can provide its customers with the most-reliable computing services available. Because of primary, backup, and tertiary systems within its electrical, mechanical, and power infrastructure, the ADC is fully redundant.

R.L. Polk Launches Enterprise Grid Computing on Linux Kevin Vasconi, CIO of R.L. Polk, a worldwide provider of automotive industry intelligence and marketing solutions, talks about implementing an enterprise grid on Linux. Oracle Magazine: What savings have you found with grid computing? Kevin Vasconi: We're conservatively estimating 30 percent savings from moving a clustered UNIX platform onto an Intel- and Linux-based grid using Oracle Database 10g. When we migrate other parts of the business off mainframes, our savings will be even higher. We predict at least a 40 percent return on investment. Oracle Magazine: And other benefits? Kevin Vasconi: We're also picking up improvements in our service-level agreements. Grid computing is more reliable, because there aren't as many single points of failure. Another benefit is in capacity planning. With grid computing, we can approach it with less risk to the business, because when we need more capacity we aren't buying another US$300,000 mainframe processor or throwing another Sun StorEdge 8600 at it. Oracle Magazine: How should a CIO sell the management team on the idea of grid computing? Kevin Vasconi: Stress the business-ROI side. Talk about the business impact of grid technology and what it can mean to the total cost of ownership—to your company's bottom line.

"One of our requirements is one-hour time to recovery [TTR], 90 percent of the time," explains Chris Pohto, senior director of the ADC. "Utilizing the grid, we always have excess capacity provisioned, with at least 100 systems and several terabytes' worth of storage just for this purpose. If a server fails, our first task is service restoration—fixing the box is secondary. So we simply move the affected instance to a free server, and the corresponding systems are back up in 15 or 20 minutes."

Continued uptime is promoted in other ways as well. Oracle is a U.S. Environmental Protection Agency (EPA) Green Power Partner and Climate Leader, which means the company not only uses renewable energy sources such as a large wind farm in West Texas, but also works to reduce greenhouse gases. With its intensive focus on energy usage and mechanical efficiency, the ADC has not experienced any electrical or mechanical infrastructure outages since it opened, in 2002.

The ADC is rated by an independent engineering firm as equivalent to a Tier IV site, which means it has multiple active power and cooling distribution paths, redundant components, fault-tolerant computing devices, and 99.995 percent availability. According to the Uptime Institute, this level of availability exceeds industry best practices. It is achieved by a focus on continual process improvement through the application of methodologies such as Six Sigma and Capability Maturity Model integration. In the unlikely event of a primary site failure, a backup site can pick up the load in under two hours, with all data and applications synchronized.

Oracle Promotes Grid Standards

Oracle has a long-standing commitment to the advancement of open, interoperable standards, for use across all products, and grid computing is no exception. Oracle is an active participant in several associations focused specifically on grid technology, including the Enterprise Grid Alliance (EGA) and Global Grid Forum (GGF).

EGA was formed to identify obstacles to the adoption of grid technologies in enterprise data centers and to promote open, interoperable solutions to these problems. Joining Oracle in EGA are more than 30 member organizations, including EMC, Fujitsu-Siemens Computers, HP, Intel, NEC, Network Appliance, and Sun Microsystems, as well as large end users such as UBS and Micron.

"EGA has helped enterprises understand how they can build and manage a computing grid in their own data centers today," explains Donald Deutsch, president of EGA and vice president of standards strategy and architecture at Oracle. "The alliance-developed Enterprise Grid Reference Model and specifications for data provisioning and grid security, freely available to the IT community, are helping vendors, end users, and others as they work to develop standards and implement grid infrastructures to service production applications."

Oracle is also a member of the GGF, a community of users, developers, and vendors leading the global standardization effort for grid computing. The GGF consists of thousands of individuals in industry and research, representing more than 400 organizations worldwide. As with the EGA, the work of GGF is carried out through community-initiated working groups, which develop best practices and specifications in cooperation with other leading standards organizations, software vendors, and users.

"Oracle's participation in the GGF—as with other standards bodies such as the W3C, OASIS, SNIA, and DMTF—is critical to our long-term strategy to build products on the foundation of industry standards," says Deutsch. "Oracle's success has been based largely on our commitment to and compliance with industry standards."

Get a Piece of the Grid

Pohto's team has responsibility for Oracle On Demand operations, including provisioning new customers and expanding their environments. When a customer initiates the Oracle On Demand service, the team provisions a "slice" of the grid for that customer's needs. As the implementation progresses and the applications are moved into production mode, systems administrators establish Grid Control rules that allow the environment to dynamically respond to the need for additional processors or storage.

"Grid Control is all about managing a large number of systems as if they were a single system," explains Pohto. "When you have a whole lot of computers, the ability to manage on an exception basis—to look across all of them and immediately see if there's a problem—is incredibly powerful."

Just as the local power company doesn't allocate electrical capacity for an individual house or business but, rather, creates distribution plans for a large set of consumers, Oracle sizes the grid infrastructure to service the aggregate demand. "When you walk into a room and throw the light switch, you don't think about who manufactured the switch or where the power came from," says Pohto. "All you care about is one thing—that the light goes on. That's the kind of turnkey capacity and reliability we supply at the ADC."

Brains of the Grid

Grid computing is built around the concept of virtual software environments. In its entirety, the grid appears as a single computing resource. But look a little closer, and you will find a pool of shared resources that is both self-correcting and self-managing. With its Grid Control features, Oracle Enterprise Manager 10g enables IT professionals to group multiple hardware nodes, databases, application servers, and other targets into single logical entities—making it easy to support distinct segments of the grid for individual customers. "By executing jobs, enforcing standard policies, monitoring performance, and automating tasks across a group of targets instead of on many systems individually, Oracle Grid Control enables us to scale smoothly," says Oracle's Souder. "Thanks to these advanced management features, the existence of many small computers in a grid infrastructure does not increase complexity."

Additionally, Oracle Application Server 10g and Oracle Database 10g are designed to run very well in clusters. Oracle Application Server 10g takes advantage of grid technology features such as policy-based resource management, metric-based workload management, and centralized user provisioning, dramatically simplifying maintenance in multinode environments. Oracle Database 10g offers data provisioning capabilities such as Oracle Streams and transportable tablespaces, which allow database administrators to detach part of a database and attach it to another database without unloading and reloading it. And Oracle Real Application Clusters (Oracle RAC) technology enables a single database to be partitioned across multiple clustered nodes, pooling the processing resources of several standard machines.

Snapshot National Retail Properties, Inc. www.nnnreit.com Location: Orlando, Florida Industry: Professional services Total assets: US$1.3 billion Oracle products and services:Oracle E-Business Suite On Demand, including Assets, Cash Management, Financial Analyzer, Financials, Internet Expenses, Payables, Procurement, Projects, Property Manager, Purchasing, Receivables, and Sales Analyzer; and Oracle Consulting services

"Oracle 10g software can be scaled simply by addition of more servers to the [Oracle] RAC configuration," Pohto notes. "The whole data center possesses the same kind of architecture. Eventually there will be no big servers anywhere but, rather, lots of little boxes, dynamically allocated as needed."

In general, servers consist of a combination of two- and four-processor Dell PowerEdge servers running Linux. For data storage, the ADC uses clustered storage systems from Network Appliance. This infrastructure supports hundreds of Oracle On Demand customers running 1,600 independent database environments and application stacks.

"Preprovisioned equipment is always ready to go in certified configurations," Pohto notes. "From the time a customer signs an Oracle On Demand contract, we have 21 days to turn over the entire environment. Right now we're averaging 11 to 14 days."

Comparing Costs

When customers contract with Oracle to place their applications in the ADC, they get to take advantage of Oracle's rigorous management and administrative practices. This is partly what attracted National Retail Properties, Inc., to adopt Oracle E-Business Suite On Demand. "We like being able to leverage Oracle's internal consulting team for implementation, upgrades, and services, and we enjoy having real-time support available if problems occur," reports Dan Tervo, vice president of information systems at the Orlando, Florida-based firm.

As a real estate investment trust, National Retail Properties, Inc., acquires, owns, manages, and indirectly develops net-leased, single-tenant properties nationwide. According to Tervo, Oracle's On Demand environment offers significant savings and faster resolution for potential technical problems, allowing his team to focus on internal support issues and on bolstering the business processes that give his company a competitive edge.

Next Steps LEARN more about Oracle Grid Computing Oracle On Demand Oracle on Linux

Before adopting Oracle E-Business Suite On Demand, National Retail Properties, Inc., conducted a comprehensive cost analysis that included hardware, software, professional services, and upgrades. The company figured that it would spend about US$3.5 million over a three-year period to deploy, maintain, and upgrade the infrastructure itself, or slightly more than US$2 million if it let Oracle do the job.

Always Online

As Oracle pushes the boundaries of grid computing, the Austin Data Center remains a unique test bed for the industry as a whole. But the real beneficiaries are Oracle customers, who end up with better products as a result of Oracle's experience. "You can see some features in Oracle 10g that came about as a result of our experiences with grid computing in Austin," says Souder. "Moving forward, I think you'll see more and more of the knowledge we learn and the experience we gain in this grid environment coming back into the products."