Gaining Flexibility and Performance with External Tables

Gaining Flexibility and Performance with External Tables

External tables are tables that represent data from flat files in a relational format. When you create and define an external table, the metadata for the external table is saved in the Warehouse Builder repository. You can use these external table definitions in mappings to design how you want to move and transform data from flat file sources to your target.

Topics

This lesson will discuss the following:

Overview
Prerequisites
Flat File Sampling

Creating the External Tables

Generating and Deploying CUSTOMERS_EXT external table
Summary

This lesson will take approximately 45 minutes to complete.


Move your mouse over this icon to show all screenshots. You can also move your mouse over each individual icon to see only the screenshot associated with it.


Overview

In this lesson you will learn how to define external tables using Oracle Warehouse Builder. To get started you will sample the customer.txt flat file definition.

Back to Topic List

Back to Topic List

In order for this lesson to work successfully, you will need to have performed the following:

1.

Completed Overview lesson.

2.

Completed Logging in to OWB lesson.

3.

Completed Importing Source Metadata lesson.

4.

Completed Defining Target Module lesson.

5.

Completed Dimensional Design using Oracle Warehouse Builder lesson.

6 .

Completed Designing ETL Data Flow Mappinglesson.

7 .

Completed Deploying Targets lesson.

 

Flat File Sampling

Back to Topic List

In this topic you will begin with sampling the customers_txt flat file.

1.

In the OWB10G_DEMO project, right click the Files node and select Create Flat File Module.

Move your mouse over this icon to see the image

 

 

2.

Click Next on the New Module Wizard: Welcome page. On the New Module Wizard: Name page type FF_MODULE as the module name and click Next.

Move your mouse over this icon to see the image

 

 

3.

On the New Module Wizard: Connection Information page, you can either type a valid directory path or use Browse to select the directory.

Click Browse.

Move your mouse over this icon to see the image

 

 

4.

In the Flat File Directory Chooser dialog select C:\OWB-OBE\sourcefiles or select the path where you saved your setup files as instructed in Lesson 1 and click OK.

Move your mouse over this icon to see the image

Click Next in the New Module Wizard: Connection Information page.

Move your mouse over this icon to see the image

 

5.

The New Module Wizard: Location page appears. You will create a new location.

To create a new location, click New. In the New Location Dialog type FF_MODULE_LOC as the name for the location. Accept the default type File System and click OK.

Move your mouse over this icon to see the image

Move your mouse over this icon to see the image

 

6.

The newly created location will automatically be selected click Next.

Move your mouse over this icon to see the image

 

 

7.

The New Module Wizard: Finish page appears, review the module summary information. Check the box - Proceed to the Import Metadata Wizard and click Finish.

Move your mouse over this icon to see the image

 

 

8.

Click Next on the the Import Metadata Wizard: Welcome page. The Import Metadata Wizard: Filter Information page appears. Accept the Default selection All Data Files and click Next.

Move your mouse over this icon to see the image

 

 

9.

The Import Metadata Wizard: Object Selection page appears. Expand C:\OWB-OBE\sourcefiles or the path that you selected in step number 4 will be displayed. Select CUSTOMERS.TXT from the Available Objects list and move it to the Selected Objects list on the right-hand side using the > (single greater than) and click Next.

Move your mouse over this icon to see the image

Move your mouse over this icon to see the image

 

10.

The Import Metadata Wizard: Summary and Import page appears. Select the customers_txt record, if it is not already selected and click Sample.

Move your mouse over this icon to see the image

On this page you see file definition (customers_txt) prefixed with a red ball indicator. The red ball indicator indicates that there is no metadata available for these files. Before importing, Warehouse Builder needs to know the structure of these files. Through the sampling process, information about the structure is gathered.

You will learn how to sample the customers.txt file.

 

11.

The Flat File Sample Wizard: Name page appears. On this page you specify the file's name and the number of rows to sample. Warehouse Builder offers a nearly endless set of character sets you could use to read the data.

Rename customer_txt the default sampled file's name to CUSTOMERS. This is the name Warehouse Builder will refer to the file after the import. Accept the default number of rows to sample as 200 and click Next.

Move your mouse over this icon to see the image

 

12.

The Flat File Sample Wizard: Record Organization page appears. You use Record Organization page to indicate how records are organized in the file that you are sampling.

Accept the default option of Records delimited by carriage return (<CR>) and click Next.

Move your mouse over this icon to see the image

 

13.

In the Flat file Sample Wizard: File Layout page, accept the default values of 0 for Skip rows. This file is an example of a single record type file, select Single for Record Type and click Next.

Move your mouse over this icon to see the image

On this page you can specify the number of rows to skip. Warehouse Builder also supports multiple file record types.

 

14.

The Flat File Sample Wizard: File Format page appears. On this page you specify the formatting for the file structure. Fixed length files as well as character-separated files are supported.

Accept the default selection, Delimited for File Format, comma (,) is the delimiter character enclosed within double quotes (" ").

Move your mouse over this icon to see the image

 

15.

The Flat File Sample Wizard: Field Properties page appears.

You can choose to use the values in the first record for the field names, when the flat files first record is a header. Check the box Use the first record as the field names.

Move your mouse over this icon to see the image

Notice that some of the data types now have changed. Without the column names, some values are actually numeric. Click Next.

Move your mouse over this icon to see the image

 

16.

Review the summary specifications after sampling the file customer.txt. Click Finish on the Flat File Sample Wizard: Summary page.

Move your mouse over this icon to see the image

 

17.

The Import Metadata Wizard: Summary and Import page reappears. The wizard has now completed the sampling process (identifying the structure) for the CUSTOMERS file, which is indicated by the green ball indicator.

Click Finish in the Import Metadata Wizard: Summary and Import to complete the import.

Move your mouse over this icon to see the image

 

18.

The Importing Progress Dialog indicates the progress of the import. You will see that metadata from the flat files has been imported to the FF_MODULE flat file module.

Move your mouse over this icon to see the image

Move your mouse over this icon to see the image

 

Creating the external table

Back to Topic List

After performing the following steps you will have created the CUSTOMERS_EXT external table in the STG target module.

1.

Collapse the Files node, expand the Databases node, expand the Oracle node and expand the STG node. You will create your external table in the STG module.

Right click the External Tables node, and select Create External Table.

Move your mouse over this icon to see the image

 

2.

Click Next on the New External Table Wizard: Welcome page. On the New External Table Wizard: Name page type CUSTOMERS_EXT as the name for the external table and click Next.

Move your mouse over this icon to see the image

 

3.

In the New External Table Wizard: File Selection page select CUSTOMERS flat file definition from the FF_MODULE. Click Next.

Move your mouse over this icon to see the image

 

4.

On this page you will specify the deployment location of the flat file datafile that you want to use at runtime. In the New External Table Wizard: Locations page select FF_MODULE_LOC. Click Next.

Move your mouse over this icon to see the image

 

5.

The New External Table Wizard: Finish page shows the result of the external table wizard. Click Finish. You will see the CUSTOMERS_EXT external table in the OWB client.

Move your mouse over this icon to see the image

Move your mouse over this icon to see the image

 

Generating and Deploying CUSTOMERS_EXT external table

Back to Topic List

In this topic you will Validate, Generate and Deploy the External table CUSTOMERS_EXT that you created.

1.

To validate the external table definition.

Expand the STG target module and expand the External Tables. Right click the CUSTOMERS_EXT external table and select Validate.

Move your mouse over this icon to see the image

 

2.

The CUSTOMERS_EXT external table is generated and the Validation Progress window displays the progress of the validation and the Validation Results window displays the validation results. Click Close to close the Validation Results window.

Move your mouse over this icon to see the image

Move your mouse over this icon to see the image

Note: Make sure the validation is successful without any errors.

 

3.

At runtime in the database an external table definition has to have a directory definition in order to get access to the flat file. This Directory object is a materialization of a connector that is defined in the context of a location. OWB will conveniently populate the connector once you validate the External table definition.

Move your mouse over this icon to see the image

 

4.

You can configure properties of the external table.

Right click the CUSTOMERS_EXT external table and select Configure. The Configuration Properties pops up.

Move your mouse over this icon to see the image

 


5.

In the Configuration Properties, the Access Specification property allows you to specify a bad file name, the discard file and log file name. This is used for debugging purposes.

Move your mouse over this icon to see the image

In the Configuration Properties, Data Files property, expand DATAFILE1 you will view the name of the data file, you can change the name too. For this example you will use the default. Close the Configuration Properties window.

By default OWB will add a data file to the External table referring to the flat file (customers.txt) name you sampled.

Move your mouse over this icon to see the image

 


6.

In the OWB client select Project > Deployment Manager

Move your mouse over this icon to see the image

 


7.

Select OBE_RUNTIME_CONNECTION Runtime Repository Connection from the connection drop-down list and click OK or use the runtime repository connection that you created in Lesson 7.

Move your mouse over this icon to see the image

 

8.

The Connection Information window opens. Provide the connection Information. You will have to enter only the password, type connect_user or use the same password that you provided in Lesson 7.

Move your mouse over this icon to see the image

 

9.

The OWB Deployment Manager opens. Expand Target_LOC, expand Connectors and select TARGET_LOC_FF_MODULE_LOC.

Move your mouse over this icon to see the image

 

10.

Expand target module STG, expand the External Tables node. Hold CTRL and select external table definition CUSTOMERS_EXT. Click Default Action.

Move your mouse over this icon to see the image

Clicking the Default Action button changes the Deploy Action from None to Create.

Move your mouse over this icon to see the image

 

11.

Click the Generate/Deploy button on the toolbar.

Move your mouse over this icon to see the image

 

12.

The Location Registration for location FF_MODULE_LOC appears. In this case Username and Password are optional. For hostname enter localhost or <name of your computer>. In the Root Path type C:\OWB-OBE\sourcefiles\ or type the path where you saved you setup files (include the backslash at the end) and click OK.

Move your mouse over this icon to see the image

 

13.

The Confirm Commit dialog pops up. Click Commit to commit your changes before deploying.

Move your mouse over this icon to see the image

 

14.

After generating the deployment specification the Pre Deployment generation results window opens.

Move your mouse over this icon to see the image

Move your mouse over this icon to see the image

 

15.

On the Pre Deployment Generation Results window select CUSTOMERS_EXT external table definition. Select Script tab at the bottom, select the CUSTOMERS_EXT record in the bottom and click View Code. The script shows a fully defined external table.

Move your mouse over this icon to see the image

Move your mouse over this icon to see the image

 

16.

In the Pre Deployment generation results window click Deploy.

Move your mouse over this icon to see the image

The Deploy results window displays and the deployment is successful. Click OK to close the Deployment results window.

Move your mouse over this icon to see the image

 

17.

After successful deployment the OWB Deployment Manager Deploy Status column displays Success. Close the OWB Deployment Manager.

Move your mouse over this icon to see the image

 

 

18.

To check whether the external table definition works. You will log into SQL Plus.

Click Start > Programs > Oracle [Oracle_Home] > Application Development > SQL Plus. In the Log on window provide connection information. For username and password type TARGET_SCHEMA, or <your target schema name and password that you created in Lesson 7>. For Host String type obeowb or <your database service name>.

Move your mouse over this icon to see the image

 

 

19.

Describe the structure of the table from the SQL prompt using DESC CUSTOMERS_EXT.

Move your mouse over this icon to see the image

 

 

20.

Run a select statement at the SQL prompt SELECT statement SELECT * from CUSTOMERS_EXT where rownum < 5; (use the semi colon to terminate the SQL statement)

Move your mouse over this icon to see the image

To close SQL Plus type exit at the SQL prompt.

 

Summary

Back to Topic List

External Tables provide a huge benefit to the Oracle Warehouse builder user. By using external tables, Warehouse Builder developers can eliminate the need for flat-file staging tables. External Tables thus reduce the processing time to load flat-file data and the need for extra storage space that flat-files staging table requires. External Tables can be joined with relational tables and other external tables. Heterogeneous joins are thus enabled between flat files and relational tables. Also the flexibility in PL/SQL transformations versus SQL loader transformations will in most cases lead to tremendous performance gain, simply because of elimination of an additional step.

 

Move your mouse over this icon to hide all screenshots

 

 

 

 

E-mail this page
Printer View Printer View
Oracle Is The Information Company About Oracle | Oracle RSS Feeds | Careers | Contact Us | Site Maps | Legal Notices | Terms of Use | Privacy