As Published In May/June 2004

Data Warehouses Go Big
By Caroline Kvitka

New features from Oracle raise performance of massive warehouses.

The world's largest commercial database holds nearly 30TB of data—three times the size of all the printed material at the United States Library of Congress. According to the 2003 Winter Corp. Top Ten Survey, which identifies the world's leading database implementations based on database size, rows/records, and workload, France Telecom is No. 1 in database size with 29.2TB of raw data managed by Oracle.

France Telecom isn't alone on Oracle: hundreds of Oracle customers run data warehouses that exceed 1TB of data (think of a thousand copies of the Encyclopedia Britannica). According to an IDC survey (January 2004), Oracle dominates the large data warehouse market at the 500GB+, 1TB+, and 4TB+ levels, and is rapidly becoming the vendor of choice at the 6TB+ level.

Pushing Scalability

One of the biggest technology issues facing enterprises today is the explosion in data volumes that is expected to occur over the next several years. According to Giga Information Group's Quarterly Technology Survey (August 2003), 43 percent of data warehouses will be larger than 1TB by January 2005.

Many data warehousing projects are simply not designed for growth. For example, according to Richard Winter, president of the Winter Corp., algorithms that work efficiently on 100GB data warehouses don't necessarily succeed on 10TB data warehouses.

"If you plan to implement a data warehouse that will grow into the tens of terabytes, you want to use a product that has proven it can operate successfully in the tens of terabytes or more," he says.

Next Steps LEARN more about Oracle scalability READ about Oracle BI features

Increasingly, companies such as Acxiom Corp., of Little Rock, Arkansas, turn to Oracle for these mega installations. The company—which helps businesses in financial services, automotive, insurance, retail, and other industries to use their customer data to grow their business, reduce fraud, and manage risk—has data warehouses that range in size from 2TB to 6TB of raw data. Acxiom's largest Oracle data warehouse, for a large financial services customer, stores a total of 30TB, six of which are raw data. The client relies on this system to manage new credit-card customer prospects.

"We're seeing some of our data warehouses grow about 20 percent a year," says Tim Donar, enterprise architect with Acxiom. "Our clients are finding better, smarter ways to market to customers, requiring feeds from other data sources they may not have stored in the past."

For two years, Acxiom has worked with HP and Oracle to plan for successful growth by testing the scalability of very large data warehouses. "HP and Oracle put together a test system that today has a total of 150TB of storage and a 45TB database running on Oracle Real Application Clusters," says Donar. "It's possible that we could have a client that needed to quickly scale up to 120TB tomorrow, and the good news is that we've been through the validation and certification to know that the hardware, the operating system, and Oracle are capable of scaling."

Acxiom uses Oracle's unique data warehousing features, such as Compression and Partitioning, to efficiently manage terabytes of data. "We've seen 30- to 40-percent disk space savings using Compression, with minimal impact on query performance," Donar says.

As its client data management needs continue to grow, Acxiom sees Oracle as a strategic partner. "Oracle has a proven track record with very large data warehouses and has proven many times over its ability to scale and meet our customer requirements. Oracle takes advantage of each new generation of hardware to scale efficiently and maximize high performance," Donar says. "In addition, many of our clients have Oracle in-house, so that means they are more willing to use and trust our data management services."