This lesson describes how to design mappings and process flows, and use them to define data movement and transformation in Warehouse Builder. When you design a mapping in Warehouse Builder, you use the Mapping Editor interface. There is a large set of mapping operators available to you for use within the Mapping Editor. Oracle Warehouse Builder allows you to use activities external to Warehouse Builder (such as e-mail, FTP commands, and operating system executables). External activities will not be covered in this course. Using the Process Flow Editor, you learn how to design process flows that interrelate mappings and other activities.
In this lesson, you learn how to create mappings to extract data from sources, transform it, and load it into targets. It also briefly introduces the Debugging Editor to debug data flows in the Mapping Editor.
Approximately 60 minutes
This tutorial covers the following topics:
Overview | ||
Prerequisites | ||
Designing Mappings | ||
Testing and Debugging | ||
Summary |
Place the cursor over this icon to load and view all the screenshots for this tutorial. (Caution: Because this action loads all screenshots simultaneously, response time may be slow depending on your Internet connection.)
Note: Alternatively, you can place the cursor over each individual icon in the following steps to load and view only the screenshot associated with that step.
Before starting this tutorial, you should have:
1. |
Downloaded the owbdemo_files.zip file and extracted the files to the d drive on your computer's hard drive
|
2. |
Completed Lessons 1, 2, and 3
|
3. |
Download Oracle Workflow Server 2.6.4 from here and install it in your Oracle database home. (Download the Oracle Database 10g Companion CD Release 2, which contains Workflow Server. This installation uses Oracle Universal Installer.) Note: You can also use Oracle Workflow Server 2.6.3 with Oracle Warehouse Builder 10g Release 2.
|
4. |
After installing Oracle Workflow Server in your Oracle database home, run the Workflow Configuration Assistant to create the owf_mgr Workflow schema.
Note for Oracle Workflow 2.6.3: Enter the following values: Workflow Account: (Accept the default ) owf_mgr
The Workflow Configuration Assistant window disappears and the command window displays the owf_mgr schema creation progress details. Do not close this command window. On successful completion, the command window will close automatically. The schema creation takes several minutes. A dialog box appears saying that Workflow Configuration has successfully completed. Click OK. The command window
closes automatically. Note for Oracle Workflow 2.6.4: When you run the Workflow Configuration Assistant, you need to specify the database connect string in the TNS Connect Descriptor box. Enter the following values: Accept the default for Install Option Workflow Account: (Accept the default ) owf_mgr
SYS Password : <Enter the SYS account password> TNS Connect Descriptor: localhost:1521:orcl Click Submit. |
Mappings extract data from the source, transform the data, and load the data into the target module (EXPENSE_WH), which you defined in the previous lesson.
In this topic, you learn to:
Examine a Predefined Mapping between a Relational Table and a Dimension | ||
Design a Mapping between an External Table and a Dimension | ||
Design a Mapping between an External Table and a Cube |
In this subtopic, you examine the predefined LOAD_CHANNELS mapping between the CHANNELS relational table and the CHANNELS dimension. This example uses a different target module, SALES_WH, and sources and targets different from those you have defined earlier. To become familiar with the Mapping Editor, perform the following steps:
1. |
In the Project Explorer panel of the Design Center, expand OWB_DEMO. Expand Databases > Oracle > SALES_WH > Mappings. Double-click LOAD_CHANNELS. The Mapping Editor launches the LOAD_CHANNELS mapping.
|
2. | You design a mapping in Warehouse Builder using the Mapping Editor interface. As you design a mapping, you select operators from the Mapping Editor Palette panel and place them on the canvas. Tip: If you are not able to see all the objects similar to the following screenshot, then in the Mapping Editor, click Auto-Layout on the toolbar to bring the mapping to its default size. Maximize the operators on the canvas by clicking the maximize icon . In the Mapping Editor, you see the following operators on the canvas: a) A Table operator named CHANNELS_IN b) A Constant operator named TOTALS c) A Dimension operator named CHANNELS_OUT d) Connection lines between attributes
|
3. | In the LOAD_CHANNELS mapping, click the table operator CHANNELS_IN header. This now becomes the currently selected object. Right-click the table operator CHANNELS_IN header and select Open Details. This launches the Operator Editor in tab format. Each tab helps in performing tasks associated with the operator. For example, the Name tab enables you to specify a name and optional description for the operator. Click Cancel to close Table Editor.
|
4. | In the LOAD_CHANNELS mapping, ensure that the CHANNELS_IN table operator is selected. (Click the table operator CHANNELS_IN header.) To determine the operator's bound name, scroll down to the Bound Name properties in the Table Operator Properties panel. This operator is bound to the CHANNELS table in the XSALES schema. Tip: Alternatively, move your mouse over the header of the operator. If the operator is currently bound, you will see the complete details in the following format: Operator <operator name>: bound to /<project name>/module name/operator bound name (for example, Operator CHANNELS_IN: bound to /OWB_DEMO/XSALES/CHANNELS). Note: Bound name is the physical name of the object that is connected to an object operator in a mapping. Bound names are used to reference the object during code generation. |
5. | In the LOAD_CHANNELS mapping, the dimension operator is named CHANNELS_OUT. The operator is bound to the CHANNELS dimension in the SALES_WH target schema.
|
6. |
In the LOAD_CHANNELS mapping, the constant operator is named TOTALS. The Constant operator enables you to define constant values. Constants can be used anywhere in a mapping. The Constant operator produces a single output group that can contain one or more constant attributes. To add a Constant operator on the canvas, you should drag a constant operator from the Palette. Tip: An example of using a constant operator is to load the value of the current system date into a table operator. To add the attributes to the TOTALS constant object, right-click the constant operator TOTALS header and select Open Details. Click the Output Attributes tab. Clicking Add will add attributes to the Totals constant object. As you see in the screenshot two attributes have been added. Click Cancel to close the Constant editor.
|
7. |
In the LOAD_CHANNELS mapping, select the TOT attribute by selecting the TOT attribute in the TOTALS constant operator. On the left in the Mapping Editor, in the Attribute Properties panel, inspect the Expression property. In the Attribute Properties panel, click the field next to the Expression property. Click the button on the right. The Expression Builder is launched. The value of the TOT attribute is "Channels Total." Close the Expression Editor.
|
8. | In the LOAD_CHANNELS mapping, you see connection lines from:
In this example, you connect operators by connecting individual operator attributes to each other. Connection lines graphically represent how the data flows from a source, through operators, and to a target. Close the Mapping Editor.
|
In this subtopic, you perform a hands-on exercise to create the REL_CATEGORY_MAP mapping that maps an external table called EXPENSE_CATEGORIES_CSV to a dimension called REL_CATEGORY.
1. |
In the Project Explorer, expand OWB_DEMO. Expand Databases > Oracle > EXPENSE_WH > Mappings. Note that you see a REL_TIME_MAP mapping, which was automatically created by OWB when you created the time dimension REL_TIME using the Time Wizard. Right-click Mappings and select New. The Create Mapping window appears.
|
2. | In the Create Mapping window, enter REL_CATEGORY_MAP as name of the mapping. Click OK. In the Project Explorer panel, you see an entry for the REL_CATEGORY_MAP mapping under the EXPENSE_WH module. The Mapping Editor launches the REL_CATEGORY_MAP mapping.
|
3. | From the Palette, drag the Dimension operator to the canvas. The Add Dimension Operator window appears. In the Add Dimension Operator window, navigate to the EXPENSE_WH module and select the REL_CATEGORY dimension. Click OK.
|
4. | The REL_CATEGORY dimension object is dropped on the canvas.
|
5. |
Alternatively, you can add existing sources and targets to a mapping through the Explorer panel, which is on the left side of the Mapping Editor. In the Explorer panel, ensure that Available objects tab is selected. Expand Oracle > EXPENSE_WH > External Tables. Drag EXPENSE_CATEGORIES to the canvas. Now there are two objects on the canvas, the EXPENSE_CATEGORIES external table and the REL_CATEGORY dimension. Tip: In the Mapping Editor, click Auto-Layout on the toolbar to change the mapping to its default size. By dragging, move the EXPENSE_CATEGORIES external table to the left of the REL_CATEGORY dimension. Maximize the EXPENSE_CATEGORIES external table by clicking the maximize icon . Similarly, maximize the REL_CATEGORY dimension.
|
6. |
Connect the source (EXPENSE_CATEGORIES external table) to the target (REL_CATEGORY dimension) as follows:
Tip: Connection lines graphically represent how the data flows from a source, through operators, and to a target.
|
7. |
You have just completed designing the mapping. Now get ready to generate the code for the mapping. In the Mapping Editor, from the menu, select Mapping > Generate. You have now generated the code that loads the dimension. Based on the ETL logic that you design in a mapping, Warehouse Builder generates the code required to implement your design. The Generation Results window is displayed. The Generation Results window displays the code that has been generated for you. Close the Mapping Editor.
|
In this subtopic, you perform a hands-on exercise to create the REL_EXPENSE_MAP mapping that maps an EXPORT_CSV external table to a REL_EXPENSE cube.
1. |
In the Project Explorer, expand OWB_DEMO. Expand Databases > Oracle > EXPENSE_WH > Mappings. Right-click the Mappings node and select New. The Create Mapping window appears.
|
2. | In the Create Mapping window, enter REL_EXPENSE_MAP as the name of the mapping. Click OK. In the Project Explorer you see an entry for the REL_EXPENSE_MAP mapping under the EXPENSE_WH module. The Mapping Editor is launched.
|
3. |
In the Explorer panel, on the left of the Mapping Editor, ensure that the Available objects tab is selected. Expand Oracle > EXPENSE_WH > External Tables. Drag the EXPENSE_DATA external table to the canvas. The EXPENSE_DATA external table object is on the canvas.
|
4. |
In the Explorer panel, ensure that the Available objects tab is selected. Expand Oracle > EXPENSE_WH > Cubes. Drag the REL_EXPENSE cube to the canvas. Now, there are two objects on the canvas, the EXPENSE_DATA external table operator and the REL_EXPENSE cube operator. Tip: In the Mapping Editor, click Auto-Layout on the toolbar to bring the mapping to its default size. Ensure that the EXPENSE_DATA external table operator is to the left of the REL_EXPENSE cube operator. Maximize the EXPENSE_DATA external table by clicking the maximize icon. In the same way, maximize the REL_EXPENSE cube operator.
|
5. |
In the Mapping Editor, from the Palette panel, drag the Expression operator to the canvas. In this example, the Expression operator is used to transform the EXPENSE_DATE attribute in the EXPENSE_DATA external table from a date data type to a number data type. The target attribute in the cube is a number data type, therefore, this data type conversion is required to match the data type.
|
6. |
Drag a connection line from EXPENSE_DATA.OUTGRP1.EXPENSE_DATE to EXPRESSION.INGRP1.
|
7. |
Right-click the expression operator EXPRESSION header and select Open Details.
|
8. | In the Expression Editor, click the Output Attributes tab. Click Add.
|
9. | Rename the OUTPUT1 attribute to EXPENSE_DATE. Ensure that the data type is NUMBER. Click OK.
|
10. | In the Mapping Editor, click the expression operator EXPRESSION header, and select OUTGRP1.EXPENSE_DATE. In the Attribute Properties panel, click the field next to the Expression property. Click the button on the right. The Expression Builder is launched.
|
11. | In the Expression Builder, enter the following statement in Expression for EXPRESSION: TO_NUMBER(TO_CHAR(INGRP1.EXPENSE_DATE , 'YYYYMMDD'), '99999999') Click OK.
|
12. | Connect the source operator to the target operator as follows:
You have just completed designing the mapping. Now you can generate the code for the mapping.
|
13. |
In the Mapping Editor, from the menu, select Mapping > Generate.
Warehouse Builder generates the Tip: To view these generated results at a later time, select Generation Results from the Window menu. From the toolbar, click Save All to commit your work. Close the Mapping Editor.
|
After you create mappings that define the operations for moving data from sources to targets, you can create and define process flows. Process flows interrelate mappings and activities external to Warehouse Builder, for example, e-mail. In OWB, a process flow is used to manage the execution of mappings and related activities, such as uploading a file to the warehouse machine using FTP or notifying the administrator in case of an error in the load process.
The scenario used in this example is very common. The dimensions (REL_TIME and REL_CATEGORY) must be loaded before the cube (REL_EXPENSE), because the cube references the dimension values. Only after all the dimensions are successfully loaded will the loading of the cube start. You now learn how to design the process flow.
The Process Flow Editor includes a variety of activities that you add and then connect with transitions to design a flow. An Activity represents units of work in a process flow. These units of work can involve components internal or external to Warehouse Builder. Transitions indicate the sequence and conditions in which to launch activities.
In this topic, you learn to:
Register the Oracle Workflow User | ||
Design a Process Flow |
Register the Oracle Workflow User
You need to grant specific roles to the Oracle Workflow(OWF) user, owf_mgr, for it to have the privileges to execute a process flow in the Control Center.
The step of registering the Oracle Workflow user, owf_mgr, is because of the new security model. This new security model means that you do not have to embed the password for the Control Center in database-links owned by the OWF user. The Control Center user is highly privileged and its password is tightly controlled.
To register the OWF user, you use the Register Warehouse Builder Users wizard from the Security > Users node in Global Explorer panel. To get access to the security node, ensure you log in to the design center as the repository owner.
1. |
If not already logged in, log in to the Design Center with username/password as rep_owner/rep_owner. In the Global Explorer panel, expand the Security node. Right-click the Users node and select New. Note: You must save or revert your changes before creating a new user. To save previous changes, select Save All from the Design menu or click on the toolbar. The Register Warehouse Builder Users Wizard is launched. Click Next on the Welcome page.
|
2. |
On the Select DB user to register page, you can select from the available list of database users or create a new one that will be automatically registered as Warehouse Builder target user. Select owf_mgr from the Available DB Users list and click > to move it to Selected Users list.
|
3. |
On the Check user as target schema page, de-select the Used as target schema option. You need not make OWF user, a target user. Click Next. On the Summary page, examine the details and click Finish. The Fix Database Default Role for Users dialog box diaplays.
|
4. |
In the Fix Database Default Role for Users dialog box, enter sys or <your SYS password> in SYSDBA Password field in the Fix Now section. Click OK. Notice that owf_mgr user gets added inside the Security > Users node, in the Global Explorer panel. |
Oracle Warehouse Builder has a graphical mapping debugger that enables you to see the data flow through the mapping in a graphical interface. The mapping debugger is embedded in the Mapping Editor. This section uses text and screenshots (not as hands-on) to demonstrate how to use the debugger to debug mappings.
1. |
These steps are read-only, not to be performed. Assumptions: The REL_CATEGORY dimension has been deployed to the target schema. The EXPENSE_CATEGORIES external table is not deployed. You will debug the REL_CATEGORY_MAP mapping. Expand OWB_DEMO, expand Databases, expand EXPENSE_WH, and double-click REL_CATEGORY_MAP. The REL_CATEGORY_MAP mapping opens in the Mapping Editor. Maximize the window.
|
2. |
Warehouse Builder provides debugging capabilities for data flows within the Mapping Editor. Debugging functions are available on the toolbar and the Debug menu. Start the debug session by clicking Debug Start ( ) on the toolbar. The Mapping Editor switches to debug mode with the debug panels appearing at the bottom of the editor.
|
3. |
The debugger tries to locate objects in the target schema based on the names of the operators. When you run the mapping in debug mode for the first time Warehouse Builder will not be able to identify links to all source tables and gives the following error message:
|
4. |
Every source or target operator in the mapping is listed on the Test Data tab in the Debug Info panel. It contains the object name and a check mark that indicates that the database object has already been bound to the source or target operator. Before you can run the mapping in debug mode, each listed source or target operator must be bound and have a check mark. Typically, you need test data for all source operators.
|
5. |
Clicking the Edit button launches the Bind Debug Test Data page, which enables you to add or change the binding of an operator, as well as the test data in the bound database objects.
|
6. |
After you have defined the test data connections for each of the data operators, you can initially generate the debug code by selecting Re-initialize from the Debug menu, or by clicking the Re-initialize button on the toolbar. In the Messages tab, in the Debug Info panel, the Successful Debug initialization indicates that test data is available for all operators.
|
7. |
If you are interested in how a specific operator is processing data, you can set a breakpoint on that operator. Set breakpoint from the debug toolbar. This will cause a break in the debug session. Notice that the Breakpoints tab appears in the Debug Info panel in the lower-left corner of the Mapping Editor. Breakpoints can be set during the mapping debug run and can be activated and deactivated as the mapping runs.
|
8. |
Using the Step icon on the toolbar, to step through the mapping and watch data flow into the selected object. Click the Step button several times to retrieve rows for the debug run. Observe the rows of data in the Step Data tab in the Debug Data panel at the bottom right of the Mapping Editor.
|
In this lesson, you've learned how to:
Design Mappings |
|
Design Process Flows that deploys and executes the data loads together |
|
Use the Mapping Debugger to test and debug with Warehouse Builder |