Gaining Flexibility and Performance with External Tables
Gaining Flexibility and Performance with External Tables
Purpose
External tables are tables that represent data from flat files
in a relational format. When you create and define an external table, the metadata
for the external table is saved in the Warehouse Builder repository. You can
use these external table definitions in mappings to design how you want to move
and transform data from flat file sources to your target.
This lesson will take approximately 45 minutes to complete.
Viewing Screenshots
Move your mouse over this icon to show
all screenshots. You can also move your mouse over each individual icon to see
only the screenshot associated with it.
Overview
In this lesson you will learn how to define external
tables using Oracle Warehouse Builder. To get started you will sample the customer.txt
flat file definition.
In this topic you will begin with sampling the customers_txt
flat file.
1.
In the OWB10G_DEMO project, right click the Files
node and select Create Flat File Module.
2.
Click Next on the New
Module Wizard: Welcome page. On the New Module Wizard:
Name page type FF_MODULE as the module name
and click Next.
3.
On the New Module Wizard: Connection
Information page, you can either type a valid directory path
or use Browse to select the directory.
Click Browse.
4.
In the Flat File Directory Chooser
dialog select C:\OWB-OBE\sourcefiles or select the path
where you saved your setup files as instructed in Lesson 1 and click OK.
Click Next in the New Module Wizard: Connection
Information page.
5.
The New Module Wizard: Location
page appears. You will create a new location.
To create a new location, click New. In the New
Location Dialog type FF_MODULE_LOC as the name
for the location. Accept the default type File System
and click OK.
6.
The newly created location will automatically
be selected click Next.
7.
The New Module Wizard: Finish page
appears, review the module summary information. Check the box - Proceed
to the Import Metadata Wizard and click Finish.
8.
Click Next on the the Import
Metadata Wizard: Welcome page. The Import Metadata Wizard:
Filter Information page appears. Accept the Default selection
All Data Files and click Next.
9.
The Import Metadata Wizard: Object
Selection page appears. Expand C:\OWB-OBE\sourcefiles
or the path that you selected in step number 4 will be displayed. Select
CUSTOMERS.TXT from the Available Objects list and move
it to the Selected Objects list on the right-hand side using the >
(single greater than) and click Next.
10.
The Import Metadata Wizard: Summary
and Import page appears. Select the customers_txt
record, if it is not already selected and click Sample.
On this page you see file definition (customers_txt) prefixed with a
red ball indicator. The red ball indicator indicates that there is no
metadata available for these files. Before importing, Warehouse Builder
needs to know the structure of these files. Through the sampling process,
information about the structure is gathered.
You will learn how to sample the customers.txt file.
11.
The Flat File Sample Wizard: Name
page appears. On this page you specify the file's name and the number
of rows to sample. Warehouse Builder offers a nearly endless set of character
sets you could use to read the data.
Rename customer_txt the default sampled file's name to CUSTOMERS.
This is the name Warehouse Builder will refer to the file after the import.
Accept the default number of rows to sample as 200 and
click Next.
12.
The Flat File Sample Wizard: Record
Organization page appears. You use Record Organization page to
indicate how records are organized in the file that you are sampling.
Accept the default option of Records delimited by carriage return (<CR>)
and click Next.
13.
In the Flat file Sample Wizard: File
Layout page, accept the default values of 0
for Skip rows. This file is an example of a single record type file, select
Single for Record Type and click Next.
On this page you can specify the number of rows to skip. Warehouse Builder
also supports multiple file record types.
14.
The Flat File Sample Wizard: File
Format page appears. On this page you specify the formatting
for the file structure. Fixed length files as well as character-separated
files are supported.
Accept the default selection, Delimited for File Format,
comma (,) is the delimiter character enclosed within double
quotes (" ").
15.
The Flat File Sample Wizard: Field
Properties page appears.
You can choose to use the values in the first record for the field names,
when the flat files first record is a header. Check the box Use
the first record as the field names.
Notice that some of the data types now have changed. Without the column
names, some values are actually numeric. Click Next.
16.
Review the summary specifications after sampling
the file customer.txt. Click Finish on the Flat
File Sample Wizard: Summary page.
17.
The Import Metadata Wizard: Summary
and Import page reappears. The wizard has now completed the sampling
process (identifying the structure) for the CUSTOMERS file, which is indicated
by the green ball indicator.
Click Finish in the Import Metadata Wizard:
Summary and Import to complete the import.
18.
The Importing ProgressDialog
indicates the progress of the import. You will see that metadata from
the flat files has been imported to the FF_MODULE flat file module.
After performing the following steps you will have
created the CUSTOMERS_EXT external table in the STG target module.
1.
Collapse the Files node, expand the Databases
node, expand the Oracle node and expand the STG
node. You will create your external table in the STG module.
Right click the External Tables node, and select Create
External Table.
2.
Click Next on the New
External Table Wizard: Welcome page. On the New External
Table Wizard: Name page type CUSTOMERS_EXT as
the name for the external table and click Next.
3.
In the New External Table Wizard:
File Selection page select CUSTOMERS flat file
definition from the FF_MODULE. Click Next.
4.
On this page you will specify the deployment
location of the flat file datafile that you want to use at runtime. In
the New External Table Wizard: Locations page select
FF_MODULE_LOC. Click Next.
5.
The New External Table Wizard: Finish
page shows the result of the external table wizard. Click Finish.
You will see the CUSTOMERS_EXT external table in the OWB client.
Generating and Deploying CUSTOMERS_EXT
external table
In this topic you will Validate, Generate and Deploy
the External table CUSTOMERS_EXT that you created.
1.
To validate the external table definition.
Expand the STG target module and expand the External Tables. Right click
the CUSTOMERS_EXT external table and select Validate.
2.
The CUSTOMERS_EXT external table is generated and the
Validation Progress window displays the progress of
the validation and the Validation Results window displays the validation
results. Click Close to close the Validation Results
window.
Note: Make sure the validation is successful without any errors.
3.
At runtime in the database an external table
definition has to have a directory definition in order to get access to
the flat file. This Directory object is a materialization of a connector
that is defined in the context of a location. OWB will conveniently populate
the connector once you validate the External table definition.
4.
You can configure properties of the external
table.
Right click the CUSTOMERS_EXT external table and select
Configure. The Configuration Properties
pops up.
5.
In the Configuration Properties,
the Access Specification property allows you to specify
a bad file name, the discard file and log file name. This is used for
debugging purposes.
In the Configuration Properties, Data Files
property, expand DATAFILE1 you will view the
name of the data file, you can change the name too. For this example you
will use the default. Close the Configuration
Properties window.
By default OWB will add a data file to the External table referring to
the flat file (customers.txt) name you sampled.
6.
In the OWB client select Project > Deployment Manager
7.
Select OBE_RUNTIME_CONNECTION
Runtime Repository Connectionfrom the connection drop-down
list and click OK or use the runtime repository connection
that you created in Lesson 7.
8.
The Connection Information window
opens. Provide the connection Information. You will have to enter only
the password, type connect_user or use the same password
that you provided in Lesson 7.
9.
The OWB Deployment Manager
opens. Expand Target_LOC, expand Connectors
and select TARGET_LOC_FF_MODULE_LOC.
10.
Expand target module STG,
expand the External Tables node. Hold CTRL
and select external table definition CUSTOMERS_EXT. Click
Default Action.
Clicking the Default Action button changes the Deploy Action from None
to Create.
11.
Click the Generate/Deploy button
on the toolbar.
12.
The Location Registration
for location FF_MODULE_LOC appears. In this case Username and Password
are optional. For hostname enter localhost or <name
of your computer>. In the Root Path type C:\OWB-OBE\sourcefiles\
or type the path where you saved you setup files (include the
backslash at the end) and click OK.
13.
The Confirm Commit dialog pops up. Click Commit
to commit your changes before deploying.
14.
After generating the deployment specification
the Pre Deployment generation results window opens.
15.
On the Pre Deployment Generation Results window
select CUSTOMERS_EXT external table definition. Select
Script tab at the bottom, select the CUSTOMERS_EXT
record in the bottom and click View Code. The script
shows a fully defined external table.
16.
In the Pre Deployment generation results window
click Deploy.
The Deploy results window displays and the deployment is successful.
Click OK to close the Deployment results window.
17.
After successful deployment the OWB Deployment
Manager Deploy Status column displays Success. Close
the OWB Deployment Manager.
18.
To check whether the external table definition
works. You will log into SQL Plus.
Click Start > Programs > Oracle [Oracle_Home] > Application
Development > SQL Plus. In the Log on window provide connection
information. For username and password type TARGET_SCHEMA,
or <your target schema name and password that you created in Lesson
7>. For Host String type obeowb or <your database
service name>.
19.
Describe the structure of the table from the
SQL prompt using DESC
CUSTOMERS_EXT.
20.
Run a select statement at the SQL prompt SELECT
statement SELECT * from CUSTOMERS_EXT where rownum < 5;
(use the semi colon to terminate the SQL statement)
External Tables provide a huge benefit to the Oracle Warehouse
builder user. By using external tables, Warehouse Builder developers can eliminate
the need for flat-file staging tables. External Tables thus reduce the processing
time to load flat-file data and the need for extra storage space that flat-files
staging table requires. External Tables can be joined with relational tables
and other external tables. Heterogeneous joins are thus enabled between flat
files and relational tables. Also the flexibility in PL/SQL transformations
versus SQL loader transformations will in most cases lead to tremendous performance
gain, simply because of elimination of an additional step.
Move your mouse over this icon to hide all screenshots