Legal | Privacy
Implementing Information Lifecycle Management Using Oracle Database 10g

Implementing Information Lifecycle Management Using Oracle Database 10g

The goal of this tutorial is to show you how to manage data durings its lifetime using the Information Lifecycle Management strategy with Oracle Database 10g.

Approximately 1 hour

This tutorial covers the following topics:

Place the cursor over this icon to load and view all the screenshots for this tutorial. (Caution: Because this action loads all screenshots simultaneously, response time may be slow depending on your Internet connection.)

Note: Alternatively, you can place the cursor over each individual icon in the following steps to load and view only the screenshot associated with that step.

Information Lifecycle Management (ILM) is concerned with everything that happens to data during its lifetime.

Although most organizations have long regarded their stores of data as one of their most valuable corporate assets, how this data was managed and maintained varies enormously. Originally, data was used to help achieve operational goals, run the business and help identify the future direction and success of the company. However new government regulations and guidelines, such as Sarbanes-Oxley, HIPAA, DOD5015.2-STD in the US and the European Data Privacy Directive in the European Union, are a key driving force in how and why data is being retained, as they are now requiring organizations to retain and control information for very long periods of time.

Information today comes in a wide variety of types, for example it could be an email message, a photograph or an order in an Online Transaction Processing System. The challenge now before all organizations, is to understand how their data evolves, determine how it grows, monitor how its usage changes over time, and decide how long it should survive

Today there are two additional objectives IT managers are trying to satisfy: to store vast quantities of data, for the lowest possible cost; and to meet the new regulatory requirements for data retention and protection.

Oracle Database 10g is Ready for Business ILM

Oracle Database 10g is ready today for Business ILM. It is capable of storing many different types of data, and storing all of your data in an Oracle database, means that it is much easier to manage, because the data is all in one place, instead of being stored using many different formats. Thus the Oracle database is the ideal platform to implement an Information Lifecycle Management policy, because it has a number of unique features, which makes it very easy to implement an ILM solution:

Low-Cost: uses low cost storage to reduce costs

Implement ILM in 3 Easy Steps

ILM can be implemented using Oracle Database 10g by following these three simple steps:

1.
2.
3. Manage Access and Migration of Data by Class

Data can be classified in a variety of ways, such as on frequency of usage, and then storage tiers can be created for the data classes that have been identified. During the lifetime of the data, it will then migrate between the data classes and access to it will be controlled. Eventually the data may be archived or it could remain inside the database forever.

Oracle Database 10g is ideal for implementing ILM because it’s simple to use since there are no specialized data stores to manage and it operates independent of any hardware. It has proven fast performance, therefore all information can be quickly retrieved. The security features in the database ensure that data is secure from unauthorized access and data is always transactionally consistent. The Oracle Database provides total flexibility, therefore it can rapidly adapt to any change in requirements, which is extremely important due to the continuing evolution of the emerging regulations. Using Oracle Database 10g, a comprehensive ILM solution can be implemented for the lowest possible cost.

This tutorial shows how a number of the features in the Oracle Database can be used to implement an Information Lifecycle Management strategy. During this tutorial, you migrate data between storage tiers, protect it from changes and control the data which users may see.

In this tutorial, you retain all sales orders placed since 1995. Users are only allowed to see data from the year 2000 onwards and only a special category of user is allowed to view the historical data.

Before starting this tutorial, you should have:

1.

Completed the Installing Oracle Database 10g on Windows Using Real Application Clusters (RAC) and Automated Storage Management (ASM) tutorial.

OR

Completed the Installing Oracle Database 10g on Linux tutorial.

 

2.

Download and unzip ilm.zip into your working directory (i.e./home/oracle/wkdir).

 

Back to Topic List

Identifying the Data Classes requires analyzing the data to determine the most suitable classification. Perform the following steps to see how the data is currently classified:

1.

Open a browser window and enter the following URL:

http://<hostname>:5500/em

Login as system/<password> and click Login.

Move your mouse over this icon to see the image

 

2.

Click Administration tab.

Move your mouse over this icon to see the image

 

3.

Under Schema, click Tables.

Move your mouse over this icon to see the image

 

4.

Enter SH for the schema and SALES for the Object Name and click Go.

Move your mouse over this icon to see the image

 

6.

Make sure SALES is selected and click Edit.

Move your mouse over this icon to see the image

 

7.

Select Partitions tab.

Move your mouse over this icon to see the image

 

8.

The data has been classified by Date using the column TIME_ID and the data has been grouped by years or quarters depending on how old it is. All the data for each date range is held in a partition. Classifying by time is a very common method and in this case is the most suitable method. Click Database breadcrumb.

Move your mouse over this icon to see the image

 

You next need to identify where all of your data is to be stored. Typically this would involve using different types of storage devices such as high performance disks for frequenctly accessed data and low-cost devices for infrequently used and historical data. In this tutorial, you create 3 storage tiers: High Performance, Low Cost and Historical. Since you don't have three types of storage devices available you create tablespaces that map to the tiers instead. In this section, you perform the following tasks:

Create tablespaces for each storage tier
Assign partitions to storage tier tablespaces

Create Tablespace for Each Storage Tier

1.

Under Storage, select Tablespaces.

Move your mouse over this icon to see the image

 

2.

Click Create.

Move your mouse over this icon to see the image

 

3.

Enter HIGH_PERFORMANCE as the Name. In the Datafiles section, click Add. Note that for the High_Performance tier the data files would typically be placed on a high performance storage devices or using ASM. Since those devices are not unavailable, the datafiles will have to be placed on a single device in order to complete this step.

Move your mouse over this icon to see the image

 

4.

Enter high_perf for the File Name and specify 3 MB for the File Size. Then click Continue.

Move your mouse over this icon to see the image

 

5.

Click OK.

Move your mouse over this icon to see the image

 

6.

Now you can create the next Tablespace. Click Create. The process is now repeated to create the Low Cost Storage Tier.

Move your mouse over this icon to see the image

 

7.

Enter LOW_COST as the Name. In the Datafiles section, click Add. Note that for the LOW_COST tier the data files would typically be placed on a low costs ATA storage devices. Since those devices are not unavailable, the datafiles will have to be placed on a single device in order to complete this step.

Move your mouse over this icon to see the image

 

8.

Enter low_cost for the File Name and specify 10 MB for the File Size. Under Storage, click Automatically extend datafile when full (AUTOEXTEND) and enter 10KB for the increment. Then click Continue.

Move your mouse over this icon to see the image

 

9.

Click OK.

Move your mouse over this icon to see the image

 

10.

Click Create. The final tier is the Historical tier where all of the orders prior to 2000 are held.

Move your mouse over this icon to see the image

 

11.

Enter HISTORICAL as the Name. In the Datafiles section, click Add. Note that for the Historical tier the data files would typically be placed on a low costs ATA storage devices. Since those devices are not unavailable, the datafiles will have to be placed on a single device in order to complete this step.

Move your mouse over this icon to see the image

 

12.

Enter historical for the File Name and specify 10 MB for the File Size. Then click Continue.

Move your mouse over this icon to see the image

 

13.

Click OK.

Move your mouse over this icon to see the image

 

14.

Scroll down to see the list of Tablespaces you just created.

Move your mouse over this icon to see the image

 

15.

You see the three tablespaces which correspond to each Storage Tier. Notice that tablespaces are currently empty. Note that in a real ILM implementation, the tablespaces would not have to be named to correspond to the required storage tiers. The tablespace approach has been used in this tutorial to illustrate the process because three different types of actual storage devices are unavailable.

Move your mouse over this icon to see the image

 

16.

Scroll up to the top. Click Database breadcrumb.

Move your mouse over this icon to see the image

 

   

Assign Partitions to Storage Tier Tablespaces

Once the Storage Tiers have been created, the next step is to assign the partitions holding the data to the appropriate storage tiers.

1.

Under Schema, select Tables.

Move your mouse over this icon to see the image

 

2.

Enter SH for Schema and SALES for Object Name. Then click Go.

Move your mouse over this icon to see the image

 

3.

Select SALES.

Move your mouse over this icon to see the image

 

4.

Select Partitions tab.

Move your mouse over this icon to see the image

 

5.

Assign the HIGH_PERFORMANCE partitions first. Select the last set of pages in the drop down list box. The High Performance storage tier should only contain orders for Q3 to Q4 for 2003.

Move your mouse over this icon to see the image

 

6.

Select the SALES_Q3_2003 partition, select Move for the Action and click Go.

Move your mouse over this icon to see the image

 

7.

Select the search light for the Tablespaces field.

Move your mouse over this icon to see the image

 

8.

Select the HIGH_PERFORMANCE tablespace and click Select.

Move your mouse over this icon to see the image

 

9.

Click OK to assign the partition to the HIGH_PERFORMANCE tablespace.

Move your mouse over this icon to see the image

 

10.

Select the SALES_Q4_2003 partition, select Move for the Action and click Go.

Move your mouse over this icon to see the image

 

11.

Enter HIGH_PERFORMANCE in the tablespace field and click OK.

Move your mouse over this icon to see the image

 

12.

Click Apply to commit your changes.

Move your mouse over this icon to see the image

 

13.

Now you can assign the LOW_COST tablespace to the partitions from Q4 2000 to Q2 2003. Select any of the partitions within this timeframe (i.e. SALES Q1 2002), select the Action Move and click Go.

Move your mouse over this icon to see the image

 

14.

Enter LOW_COST in the tablespace field and click OK.

Move your mouse over this icon to see the image

 

17.

Perform the prior steps for the following partitions and assign the LOW COST tablespace. When done, click Apply.

SALES_Q1_2000
SALES_Q2_2000
SALES_Q3_2000
SALES_Q4_2000
SALES_Q1_2001
SALES_Q2_2001
SALES_Q3_2001
SALES_Q4_2001
SALES_Q1_2002
SALES_Q2_2002
SALES_Q3_2002
SALES_Q4_2002
SALES_Q1_2003
SALES_Q2_2003

Move your mouse over this icon to see the image

 

18.

Perform the prior steps for the following partitions and assign the HISTORICAL tablespace. When done, click Apply.

SALES_1995
SALES_1996
SALES_H1_1997
SALES_H2_1997
SALES_Q1_1998
SALES_Q2_1998
SALES_Q3_1998
SALES_Q4_1998
SALES_Q1_1999
SALES_Q2_1999
SALES_Q3_1999
SALES_Q4_1999

Move your mouse over this icon to see the image

 

19.

All your partitions have been assigned to the various storage tiers. Click Database breadcrumb.

Move your mouse over this icon to see the image

 

20.

Click Tablespaces.

Move your mouse over this icon to see the image

 

21.

Notice that your tablespaces are now full with data.

Move your mouse over this icon to see the image

 

22.

Scroll up to the top and click Database breadcrumb.

Move your mouse over this icon to see the image

 

The final step in implementing your ILM strategy is to decide who can access your data and what they can do. There are a variety of different techniques that can be used to restrict access to the historical data so that it is not always visible in routine reports. In addition, you can prevent changes to the historical data and illustrate how to store all of this data using the minimum of storage requirements. Perform the following tasks:

Control access to Historical Data

Query Historical Data

Specify a VPD policy on Sales
Restrict Update Access to the Sales Data

Try to Update the Historical Data

Make Historical Data Read Only

Control Access to Historical Data

Since all the data is now being held in the database, it is necessary to ensure that it is only viewed when it is really required. Thus saving query execution time and system resources by only accessing the data that is really needed.

There are 2 techniques that can be used to restrict access to the Historical data:

Using Views
Using a Virtual Private Database (VPD) policy

In this section, you create two views, one that allows standard users to access data from 1st Jan 2000 and another view, that allows access to the historical data. Perform the following steps:

1.

From the Administration tab, under Schema, click Views.

Move your mouse over this icon to see the image

 

2.

Create a view called SALES_HISTORICAL which only allows access to the historical data, that is up to 31st December 1999. Click Create.

Move your mouse over this icon to see the image

 

3.

Enter SALES_HISTORICAL as the Name, SH as the Schema and enter the following Query Text, then click OK.

SELECT * FROM sales WHERE time_id < '1-Jan-2000'

Move your mouse over this icon to see the image

 

4.

Now create a second view SALES_CURRENT which allows access to all the current data which is from the 1st Jan 2000. Click Create.

Move your mouse over this icon to see the image

 

5.

Enter SALES_CURRENT as the Name, SH as the Schema and enter the following Query Text, then click OK.

SELECT * from sales where time_id > '31-Dec-1999'

Move your mouse over this icon to see the image

 

6.

Notice the Views you just created. You can see that you have 2 views created, SALES_CURRENT for the recent data and SALES_HISTORICAL for all of the old orders. Now you need to create the user SH_USER that will use them. Scroll up and click Database breadcrumb.

Move your mouse over this icon to see the image

 

Query Historical Data

Now you can query the historical data. Perform the following steps:

1.

To illustrate the view SALES_HISTORICAL is only showing data prior to the year 2000, query the SALES_HISTORICAL view to report the most recent data it holds. This can be found by querying the time_id column. Open a terminal window and enter the following commands:

cd /home/oracle/wkdir
sqlplus sh/sh
@ilm_query01 

The @ilm_query01.sql contains the following:

select max(time_id)from sh.sales_historical;

Move your mouse over this icon to see the image

 

2.

Since this view only contains data up to the end of 1999, you can now report which Customers purchased the most goods up to the year 2000. Execute the following command:

@ilm_query02 

The @ilm_query02.sql contains the following:

SELECT cust_last_name, sum(amount_sold) 
  FROM sh.sales_historical h, sh.customers c 
  WHERE c.cust_id=h.cust_id 
  GROUP BY cust_last_name 
  ORDER BY sum(amount_sold) desc;

Move your mouse over this icon to see the image

 

Specify a VPD policy on Sales

With so much data being retained in the database it is important to ensure that users only view the data they are allowed to see. Although views restrict what a user can see in the database, the user can bypass the view by querying the table directly. To avoid this potential breach of security which can occur when someone does not used the designated application to query the data and instead uses a tool like sql*plus. A security policy can be defined using Virtual Private Database (VPD) which defines exactly what data is visible. In this section, you see how VPD can be used to prevent access to the historical data. Perform the following steps:

1.

First as user SH count the number of rows visible in the Sales table. Make a note of this number . At the moment you have access to all the data in the Sales table. From your sqlplus session, run the following SQL script:

@ilm_count_sales

The @ilm_count_sales.sql contains the following:

connect sh/sh
select count(*) from sales;

Move your mouse over this icon to see the image

 

2.

Next as user SYSTEM add the following VPD Policy which states that a user only see data from 1-Jan-2000. From your sqlplus session, run the following SQL script:

@ilm_add_policy

The @ilm_add_policy.sql contains the following:

connect system/oracle
CREATE OR REPLACE FUNCTION ilm_pf1 (oowner IN VARCHAR2, ojname IN VARCHAR2) RETURN VARCHAR2 AS
con VARCHAR2(200);
BEGIN
con :='time_id > ''31-DEC-1999''';
RETURN (con);
END ilm_pf1;
/
BEGIN
DBMS_RLS.ADD_POLICY (object_schema=>'SH',object_name=>'Sales',
policy_name=>'ilm_current_data',function_schema=>'system',
policy_function=>'ilm_pf1',
sec_relevant_cols=>'time_id');
END;
/

Move your mouse over this icon to see the image

 

3.

Now repeat the query of the Sales table as user SH and fewer rows should be visible because only data from the 1st Jan 2000 is available to this user. From your sqlplus session, run the following SQL script:

@ilm_count_sales

The @ilm_count_sales.sql contains the following:

connect sh/sh
select count(*) from sales;

Move your mouse over this icon to see the image

 

Restrict Update Access to the Sales Data

In order to comply with some new regulations, you need to show that only authorized users can update the sales data. Therefore, you need to create a new user called SH_USER who is only permitted to read the Sales Data. Perform the following steps:

1.

From the Administration tab, under Security, click Users.

Move your mouse over this icon to see the image

 

2.

The first step is to create a new user SH_USER. Click Create.

Move your mouse over this icon to see the image

 

3.

Enter SH_USER for the Name and Password. Enter a Default Tablespace of EXAMPLE and a Temporary Tablespace of TEMP, then click Role tab.

Move your mouse over this icon to see the image

 

4.

Since this is not a privileged user, only limited resources are granted to this user. Click Modify.

Move your mouse over this icon to see the image

 

5.

Select RESOURCE from the list of Available Roles and click Move then click OK.

Move your mouse over this icon to see the image

 

6.

To ensure that this user can only read the sales and customer data, they must be explictly granted the SELECT privilege. Click Object Privileges tab.

Move your mouse over this icon to see the image

 

7.

Select the Object Type Table and click Add.

Move your mouse over this icon to see the image

 

8.

In the Select Table Objects field, enter SH.SALES, SH.SALES_HISTORICAL, SH.CUSTOMERS. Select Select from the list of Available Privileges and click Move. Then click OK.

Move your mouse over this icon to see the image

 

9.

Click OK to create the user.

Move your mouse over this icon to see the image

 

10.

Select the Database breadcrumb.

Move your mouse over this icon to see the image

 

Try to Update the Historical Data

Some new regulations require that data cannot be changed or there may be a business requirement where records may not be altered after a certain date. Using the security features within the Oracle Database, user SH_USER does not have sufficient privilege to update the historical data. Perform the following steps:

1.

Execute the following SQL commands:

connect sh_user/sh_user
@ilm_update01

The @ilm_update01.sql contains the following:

update sh.sales set prod_id=12 where prod_id=13 and time_id='10-Jan-98';

Move your mouse over this icon to see the image

 

Make Historical Data Read Only

There will come a point during the life time of the data when updates to the data will no longer be allowed. In the prior section, you saw how database security can be used to prevent changes, but an even better approach is to set the tablespace to read only. Once a tablespace is read-only absolutely no one can update the data. Perform the following steps:

1.

Switch back to Enterprise Manager Database Control. From the Administration tab, under Storage, click Tablespaces.

Move your mouse over this icon to see the image

 

2.

Select the tablespace HISTORICAL, select the Action Make Readonly and click Go.

Move your mouse over this icon to see the image

 

3.

Notice that the tablespace is now set to read only.

Move your mouse over this icon to see the image

 

In this tutorial, you've learned how to:

Identify Data Classes
Assign Data Classes to Storage Tiers
Manage Access and Migration

Back to Topic List

Back to Topic List

Place your cursor on this icon to hide all screenshots.

 

 

 

E-mail this page
Printer View Printer View
Oracle Is The Information Company About Oracle | Oracle RSS Feeds | Careers | Contact Us | Site Maps | Legal Notices | Terms of Use | Privacy