Every organization that deploys database archiving needs to establish two data retention policies: how long (or how much) data will be kept in the active database, and how long archived data will be retained before it is deleted altogether.
American Tire Distributors (ATD) initially established different lengths of time for active data in each of its Oracle E-Business Suite modules, but this proved impractical. “A lot of our users have to go between modules when they do their analyses,” explains Angelic Gibson, ATD’s director of IT operations. “They would find two years’ worth of data in one module but only nine months in another. They were getting really confused.”
So ATD established that a minimum of 13 months of historical data would be kept active for all modules. But because archiving is done annually rather than on an ongoing basis, as much as 24 months of data is allowed to collect in the production database.
It works like this: Archiving of data from two years prior is done at the end of January, after the close of the company’s fiscal year. (For example, data from calendar year 2008 was archived beginning in February 2010.) The archiving and purging process, which happens during evening and weekend maintenance windows, takes about eight weeks and removes a terabyte of data; when it is complete, 15 months worth of data remains in the active production database.
The next annual archive, in February 2011, will remove data from calendar year 2009. Gibson notes that, because of upgraded hardware, the archiving process will probably take four weeks instead of eight.
Data will spend seven years in ATD’s archive and then be removed to a disk-based backup system. While it’s in the archive, that data remains available to business users who need it. “That’s one of the reasons we went with Solix,” Gibson says. “Solix allows us to view the data through one single system, as opposed to our users having to pull it up somewhere else.”
Raghu Kodali, vice president of product management and strategy at Solix, says ATD’s three-tier arrangement is typical for Solix users. “As the data ages out of active archive, it can be moved into a highly compressed and immutable mode,” he explains. “Even then, it can still be searched, reported, and audited, and can only be deleted when the retention period expires. This helps customers mitigate risk from a compliance point of view.”
Some companies will need to keep their archived data much longer than ATD does, says Jack Olson, author of Database Archiving (Morgan Kaufmann, 2008). “It varies by application, by industry, and by country,” he explains. “You need to find all [that apply to you], and then pick the longest line.”