ODI 12c - Standalone Agent

Before You Begin.

Purpose

This tutorial walks you through the steps that are needed to use Oracle Data Integrator Studio (ODI Studio) graphical tools to create a standalone agent (that is, no WLS is installed) to support (among other things) scheduling of procedures. Once created and configured, the agent runs without the Studio GUI.

Time to Complete

Approximately 40 minutes.

Introduction

A common task that is performed using ODI is to set up and install an ODI Agent. After the ODI scenarios are created, they can be scheduled and orchestrated using an ODI Agent, which is a lightweight Java process that orchestrates the execution of ODI scenarios. This tutorial walks you through the steps that are needed to set up and install the ODI Agent.

This is the third in a series of four OBEs in the Oracle Data Integrator 12c: Getting Started series.

Scenario

Linda works as a database administrator for Example Enterprise Corp (http://www.example.com). In Example Enterprise, Linda is responsible for performing database management and integration tasks on various resources within the organization. In particular, Linda is responsible for data loading, transformation, and validation. To begin working on her projects, Linda created the new Master repository and Work repository. Linda also created the project and the procedure to create a relational table and populate it with data. Now Linda needs to set up and install an ODI Agent as a service. This agent will be used in the next OBE to schedule a later run of the procedure. Agents are used for many other tasks in addition to scheduling.

Hardware and Software Requirements

The following is a list of software requirements:

A supported operating system (the OS shown here is 64-bit Oracle Enterprise Linux release 6.7 (Santiago), other versions such as Windows are supported as well)
Oracle Database (the version shown here is Enterprise Edition 12.1.0.2.0, other versions are supported as well)
Oracle Data Integrator (ODI) 12c (the version shown here is 12.2.1) installer executed twice to install the following components:
- first as ODI Standalone to install ODI Standalone Agent
- second as ODI Enterprise to install ODI Studio
Ideally these two ODIs are on two different hosts, but it could be on the same host, as long as they are in different Middleware Home directories.

The following is a list of OPTIONAL software:

Oracle SQL Developer 12c (included with Database 12c install)

Note: This OBE presumes that WebLogic Server (WLS) is not present in the Middleware home for ODI Standalone, but is present in the Middleware home for ODI Enterprise. If you do have WLS installed and licensed, and want to only install ODI once, then see alternate OBE here.

Prerequisites

Before starting this tutorial, you should have:

Started Oracle Database services and components such as the Listener.
A Database user with DBA role privileges such as the SYS or SYSTEM account.
Installed the HR schema included in the Oracle Database. The Sample Schemas Installation Guide is part of the Oracle Database documentation set, and is also available online at: http://otn.oracle.com.
Already installed and configured ODI Master and Work repositories (instructions on how to do this are contained in the first of four OBEs in the Getting Started series).
Already created an ODI project and mapping for a flat file to a relational table (instructions on how to do this are contained in the second of four OBEs in the Getting Started series).

1. Creating the Domain for the Standalone Agent

The agent in this OBE runs in a domain that is very similar to (but not dependent on) WebLogic Server (WLS) domains. The GUI Configuration Wizard uses templates to determine which components are needed to support the function of the agent. There are three possible kinds of agents:

completely standalone agent (no WLS installed. This OBE uses this kind of agent.)
standalone collocated agent (WLS is installed, but is not being used. The agent is started as its own binary.)
JEE agent (WLS installed and used. The domain, admin server, managed servers, node managers, and so on are started in the normal WLS way. The agent is one of many possible JEE apps running in the domain. )

To create the standalone agent requires ODI Studio running (so that you can configure the agent), and a Standalone ODI deployment that has no user interface. When you see OS prompts below, be mindful of whether they are my_ent_host (Enterprise) where the Studio GUI is, or my_std_host (Standalone) where the database is. To create the domain using CAM, perform the following steps:

1.1 Basic Configuration

To create a domain for the agent, start the Fusion Middleware (FMW) Configuration Wizard on the host with ODI Standalone installed. This is the Common Administration Model (CAM), though you may not see that term used anywhere. It bears a striking resemblance to the WebLogic Server (WLS) domain configurator.

For most of the non-Advanced configuration, the defaults are adequate. On the **Configuration Type** panel, leave the defaults.

If you do not yet have a WLS domain, then the first one created will be called `base_domain` by default. You can rename it to something such as `odi_agent` or `mydomain99`, just make sure that it remains in `$FMW_HOME/user_projects/domains/`. If you already have a domain named `base_domain`, then you cannot "create" another one, you would *have* to rename it to `base_domain2` or something else, or you could **Update an existing domain**. The particulars about updating an existing domain are outside the scope of this OBE, and are covered in the WLS curriculum.

On the **Templates** panel, select **Oracle Data Integrator - Standalone Agent**. It is standalone in the sense that it does not use WLS.

On the **JDK Selection** panel, leave the defaults.

On the **Database Configuration Type** panel, change the **Service** to the database service (not database instance), such as `orcl.us.oracle.com`. Change the **Host Name** to `localhost`, change the **Schema Password** to the password for `DEV_STB` that you picked when you ran RCU.

On the **JDBC Component Schema** panel, leave the defaults.

On the **JDBC Component Schema Test** panel, leave the defaults.

In summary, you have completed the first half, the basic part of the configuration. You must continue with the second half, the advanced configuration.

1.2 Advanced Configuration

On the **System Components** panel, leave the defaults.

On the **ODI Server Configuration** panel, make a note of the **Port** (you will need that number later), and change the **Supervisor Password** to the password you picked when you ran RCU.

On the **Node Manager** panel, you have to put in something for credentials, but it does not need to match anything else.

On the **Configuration Summary** panel, leave the defaults.

On the **Configuration Progress** panel, there is nothing to do. Just make sure that it ends with the progress bar at 100%, and with the message, Domain Created Successfully!

On the **Configuration Success** panel, there is nothing to do.

In summary, you completed both halves of the domain configuration. This created several directories with configuration files and scripts.

2. Creating a Physical Agent

The physical agent is a background OS process that communicates with ODI via TCP/IP ports. To create the standalone agent requires ODI Studio running (so that you can configure the agent), and a Standalone ODI deployment that has no user interface. When you see OS prompts below, be mindful of whether they are my_ent_host (Enterprise) where the Studio GUI is, or my_std_host (Standalone) where the database is.

To create, start, and test the physical agent, perform the following steps:

2.1 Creating a Physical Agent

Start ODI Studio by running `odi.sh` on the host with ODI Enterprise installed.

On the **Designer** or **Topology** tab, click **Connect To Repository...**

plus-in-box — On the **Topology** tab, expand **Physical Architecture**, then expand **Agents**. Right-click **Agents** and select **New Agent**.

On the **Agent** panel, on the **Definition** tab, in **Name**, enter the name that matches the agent you created with the earlier domain wizard: `OracleDIAgent1`.
In **Port**, enter the port that matches the agent you created with the earlier domain wizard: `20910`.

Verify that **Host** is correct. If you are doing everything on one host with different Middleware Home directories, then you can use `localhost` for host (likely you install ODI twice on the same host only for testing proof-of-concept.) More likely in a production environment you have two different hosts: one with ODI Standalone (that is the host that goes here), and the other host with Studio (which is *not* the host for here).

diskette — On the **Agent** panel, on the **Definition** tab, in **Name**, enter the name that matches the agent you created with the earlier domain wizard: `OracleDIAgent1`.
In **Port**, enter the port that matches the agent you created with the earlier domain wizard: `20910`.

Verify that **Host** is correct. If you are doing everything on one host with different Middleware Home directories, then you can use `localhost` for host (likely you install ODI twice on the same host only for testing proof-of-concept.) More likely in a production environment you have two different hosts: one with ODI Standalone (that is the host that goes here), and the other host with Studio (which is *not* the host for here).

You should now see the new agent display on the left under **Topology > Physical Architecture > Agents**.

In summary, you created the physical agent configuration for within ODI. This points to the executables created in the previous CAM steps.

2.2 Starting the Background Process

On the host with ODI Standalone installed, change to the directory of the domain that contains the binary files. Start the agent using the agent name from the domain. Leave that window running (perhaps minimized or sent to the back of the desktop.)

OS Prompt

[myuser@my_std_host ~]$ cd $FMW_HOME/user_projects/domains/base_domain/bin 
[myuser@my_std_host bin]$ pwd 
/u01/app/oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/bin
[myuser@my_std_host bin]$ ls 

agent.sh            restartsession.sh   startComponent.sh    stopComponent.sh
agentstop.sh        setODIDomainEnv.sh  startloadplan.sh     stoploadplan.sh
odi_credtool.sh     setStartupEnv.sh    startNodeManager.sh  stopsession.sh
restartloadplan.sh  startcmd.sh         startscen.sh

[myuser@my_std_host bin]$ ./agent.sh -NAME=OracleDIAgent1 -PORT=20910

Note that this looks like a subset of a WebLogic Server (WLS) install. It is not a WLS install, but does use many of the same components. The very first time you run this, you will get LOTS of output as Java initializes. Subsequent run times the messages will be fewer.

OS Prompt

/u01/app/oracle/Middleware/Oracle_Home/odi/agent/internal/bin/commODIEnv.sh: line 111:
  /u01/app/oracle/Middleware/Oracle_Home/odi/agent/../../wlserver/modules/com.oracle.weblogic.security.encryption_1.0.0.0.jar: Permission denied
/u01/app/oracle/Middleware/Oracle_Home/odi/agent/internal/bin/commODIEnv.sh: line 114:
  /u01/app/oracle/Middleware/Oracle_Home/odi/agent/../../oracle_common/modules/com.oracle.webservices.orasaaj-api_12.1.2.jar: Permission denied
/u01/app/oracle/Middleware/Oracle_Home/odi/agent/internal/bin/commODIEnv.sh: line 117:
  /u01/app/oracle/Middleware/Oracle_Home/odi/agent/../../wlserver/server/lib/wlthint3client.jar: Permission denied
2013-08-02 20:54:24.897 NOTIFICATION New data source:
  [DEV_ODI_REPO/*******@jdbc:oracle:thin:@localhost:1521/ORCL.US.ORACLE.COM]
2013-08-02 20:54:27.058 NOTIFICATION ODI-1128 Agent OracleDIAgent1 is starting.
  Container: STANDALONE. Agent Version: 12.1.2 - 11/07/2013. Port: 20910. JMX Port: 21910.
Allowing : 127.0.0.1,10.216.4.175,127.0.0.1
2013-08-02 20:54:33.457 NOTIFICATION New data source:
  [DEV_ODI_REPO/*******@jdbc:oracle:thin:@(description=(address=(host=localhost)(protocol=tcp)(port=1521))(connect_data=(service_name=ORCL.US.ORACLE.COM)(server=dedicated)))]
2013-08-02 20:54:34.186 NOTIFICATION ODI-1111 Agent OracleDIAgent1 started. Agent version: 12.1.2 - 11/07/2013. Port: 20910. JMX Port: 21910.
2013-08-02 20:54:34.187 NOTIFICATION ODI-1136 Starting Schedulers on Agent OracleDIAgent1.
2013-08-02 20:54:35.678 NOTIFICATION ODI-1137 Scheduler started for work repository workrep on Agent OracleDIAgent1.

You want to see at the bottom that the scheduler has started for the correct work repository.

In summary, you have started the external process associated with the ODI physical agent.

2.3 Testing the Physical Agent

You should have left the **OracleDIAgent1 Agent** panel open from step 2.1.4.

The **ODI Information** dialog box should say, Agent Test Successful.

In summary, you have configured and started a physical agent.

3. Creating a Logical Agent

Most components in ODI have a physical and logical side to them. You already created the physical agent with ports and names, the logical agent is much simpler and shorter.

To create a logical agent that corresponds with the previous physical agent, perform the following steps:

3.1 Creating a Logical Agent

On the **Logical Agent** panel, on the **Definition** tab, in **Name**, enter `MyAgent`. In the **Physical Agents** pull-down, select **OracleDIAgent1**.

You should now see the new agent display on the left under **Topology > Logical Architecture > Agents**.

In summary, you created a logical agent to match the previous physical agent.

Want to Learn More?

In this tutorial, you should have learned how to:

Use CAM to create a domain from a template for a standalone collocated agent
Use ODI Studio to create a physical agent
Start the physical agent background process from the OS prompt
Use ODI Studio to create a logical agent that corresponds to the physical agent

Resources

The following are conceptual or procedural Help topics relevant to the topic of this tutorial:

Online documentation, viewlets, samples, and OLN URLs on OTN:

Current version: here
Older versions: here

The following specific courses:

D82167GC10 Oracle Data Integrator 12c: Integration and Administration
(replaces D64974GC20 Oracle Data Integrator 11g: Integration and Administration Ed2)
D82171GC10 Oracle Data Integrator 12c: New Features

External Web sites for related information:

www.oracle.com/goto/odi
OTN Community Forums

To learn more about Oracle Data Integrator 12c, refer to additional OBEs in the Oracle Learning Library, or on the ODI Studio Start Page.

Credits

Lead Curriculum Developer: Elio Bonazzi
Other Contributors: Viktor Tchemodanov, Alex Kotopoulis, Julien Testut