Configuring Oracle Business Intelligence 11g MDS XML Source Control Management with Git Version Control

by Christian Screen

Published September 2012

Objective

This tutorial was developed so that anyone with a basic understanding of the Oracle Business Intelligence (OBI) development lifecycle can learn how to configure the Oracle Business Intelligence 11g (11.1.1.6) metadata repository for development using the new MDS XML source control management functionality.

Introduction

Oracle BI 11g version 11.1.1.6 has integrated the concept of Source Control Management (SCM) into the Oracle BI server metadata repository (RPD). This allows Oracle BI metadata modeling efforts to conform, in a small way, to modern source control systems. Most organizations currently leverage some SCM system such as ClearCase, Git, Subversion, Perforce, or Mercurial to version development efforts.

This article explains how to use Git, the open source software Version Control System (VCS), in compliance with the new Oracle BI MDS XML SCM option. The focus is on Git because it is the system gaining the quickest adoption in enterprise development efforts. Git has many benefits over other SCM tools, such as its cost, branching model, and localized integration. Git was also developed by the revered father of Linux, Linus Torvalds, making it a great fit for Oracle Linux. Another reason this article focuses on Git is that while the new SCM functionality in Oracle BI 11g is SCM agnostic, the only SCM interaction templates provided for this integration are for the Subversion SCM. Those who need to port to SCM tools other than Git or Subversion will find insight in this article, as well.

Installing the Git Client on Windows

The Administration Tool used to develop the Oracle BI metadata repository is a Microsoft Windows based application. The Administration Tool will communicate with the Git SCM repository using the Git client software. Thus, the first step to configure the new Oracle BI MDS XML SCM functionality is to install the Git client onto the workstation on which you will be developing:

1. Download the Git SCM Windows OS client from http://git-scm.com/downloads, and install it on the local server or workstation where your Oracle BI Administration Tool resides.
2. Run the installation and accept the defaults—except for the Adjusting Your PATH environment step in the installation wizard. On that step be sure to select the Run Git from the Windows Command Prompt option as seen below.
   
   Accept the defaults for all other steps in the installation wizard.

Figure 1

3. Open a new Command Prompt after the installation wizard completes.
4. Execute the following command in the command window to verify that the Git client was installed correctly:
   
   # git
   
   You should get the following Command Line Interface (CLI) usage message:

Figure 2

Leveraging the MDS XML SCM Option

MDS XML syntax varies little when compared to the core XML standard. However, MDS XML has several specific references that make it compatible with the Oracle Metadata Service standard (yes, think MDS schema that gets created using the RCU utility). It is also different from the XML that gets generated when using the Oracle BI XML API, which outputs only a single XML file. The MDS XML outputs a large number of separate but logically-referencing XML files that allow for separation of metadata in order to better leverage XSD compliance and storage capabilities.

Setting up this functionality boils down to three main operations:

• Creating the MDS XML Directory and Files
• Setting up the RPD for MDS XML SCM functionality
• Creating an initial Git Repository on your workstation

Creating the MDS XML Directory and Files

An Oracle BI metadata repository developer needs to have the ability to open the current development RPD in offline mode in order to create the MDS XML file directory necessary to leverage SCM support.

1. Open the Administration Tool and open the RPD with which to develop in offline mode by selecting File | Open | Offline……
2. After the repository opens in the editor select File | Save As… | MDS XML Documents from the file menu.
3. Select a location on your development machine such as, C:\_Book\TennisRPD, in which to save the MDS XML Documents.
   
   The process of saving may take a few minutes depending on the size of your RPD.
4. Open a Windows Explorer window and navigate to the path you selected in Step 3 to view the expansive folder structure.

Figure 3

5. Return to the Administration Tool and select File | Close from to close the RPD model.

You've now created the MDS XML directory structure and files that will be used for the SCM functionality. However, right now your Oracle BI 11.1.1.6+ client tool has no idea what SCM utility you plan to use to manage those newly created files. You have to tell it by configuring an XML file via the Administration Tool options area.

Setting Up the Repository for MDS XML SCM Functionality

Ultimately the Administration Tool will reference an XML-based configuration file to understand how to execute commands via the SCM tool of choice—in this case Git. Because the Oracle BI SCM functionality is SCM client agnostic, the Administration Tool only communicates general function calls to the SCM client. Those commands specific to the SCM client must be outlined in an XML Configuration file that is created with the assistance of an option found within the Administration Tool. To configure the SCM specific commands:

1. In the Administration Tool select Tools | Options from the file menu.
2. Select the Source Control tab.
3. Click the New... button
4. A Specify new configuration file prompt appears asking you to save a new configuration file as an XML file in some location.
   The default location should be similar to C:\Program Files\OBIEEClient11116\oraclebi\orainst\config\OracleBIServerComponent\coreapplication, or the path where you installed your Oracle BI client tools.
5. Enter any name but make sure the XML extension remains intact. For example, artofbi-git-scm.xml.
6. Click the Save button and the SCM Configuration Editor should appear.
7. Click on the Load... button in the SCM Configuration Editor.
   If you are working on a server where the full Oracle BI Server installation is configured, you should see two files suffixed with template.xml. Selecting one of these files and clicking the OK button will populate the command fields in the editor with the appropriate syntax for the respective SCM client listed in that file name. However, if using the Oracle BI client tools installation these templates will not be available to you. In this article's downloads section you can download the template for the Git SCM. But for now in the next steps you'll create the command inputs and save to the configuration file you entered in Step 5.
8. Click the Cancel button to close the Select configuration file prompt.

Now, update the fields according to the Git client command line switches and commands. By the way, if you used the default installation path for the Git client the Git executable is here, C:\Program Files\Git\cmd\. But since you used the PATH variable option when you installed the Git client earlier, you don't need to use the full path name.

Make the command fields appear as indicated in Figure 4.

Figure 4

9. For the SCM Command Add Folder, input git add ${file}
10. For the SCM Command Add File, input git add ${file}
11. For the SCM Command Delete, input git rm -force ${file}.
    • Alternatively and for better performance you can use the ${filelist} option for the Delete command. (This is a best practice.) The syntax would then be git rm -force ${filelist}.
12. For the SCM Command Rename, input git mv -f ${from} ${to}
13. In the Description field, enter some note about what this configuration file does, such as "This is the ArtOfBi.com Git SCM Configuration Template."
14. Click the OK button in the SCM Configuration Editor to close and save the configuration.
15. You'll get a prompt stating that some commands are empty. Just click the OK button to continue past it, then click OK once more to close the Options prompt.

You may have noticed that we didn't configure any items in the Environment Variables or Post-save comment tabs in the SCM Configuration Editor. The environment variables for an SCM client usually get set with the client installation process. The Post-save comment tab allows you to enter a message that will appear after the repository under source control is saved and the SCM commands have been called successfully by the Administration tool. To add a comment that appears after the SCM Commands are called:

1. Re-open the source control configuration file you created above by selecting Tools | Options from the Administration Tool file menu.
2. Click the Source Control tab.
3. Click the Edit… button.
4. Open the artofbi-git-scm.xml file you created in the previous steps.
5. Click the Post-save comment tab
6. In the large text field enter some text similar to "Developers please be sure to commit your Oracle BI RPD SCM Git repository."
7. Click the OK button to save the SCM configuration edit.
8. Click the OK button on any other remaining prompts.

Creating an Initial Git Repository on Your Workstation

At this point you've created the MDS XML directory and files and configured your Oracle BI Administration Tool with the commands necessary to interact with the Git SCM. Now you must make a local Git SCM repository to stage the MDS XML files and hold versioning for your development efforts. Remember at this point we are talking about two separate tools: The Oracle BI Administration Tool, and Git. There is no real transparently integrated relationship here. So you'll create a Git repository to hold the metadata about your MDS XML files. You'll then import those files into the local Git SCM repository so that Git can keep track of them and their changes. Here's how to initially import the MDS XML files:

1. Open a new command prompt window.
2. Change the directory so that you are within the base directory you selected earlier when saving the MDS XML files. For example, cd C:\_Book\TennisRPD
3. Execute the following command to initialize (create) a blank Git repository:
   
   # git init
4. Execute the following command in order to add (import) all files in this root directory to the SCM repository:
   
   # git add .
   
   Be sure to add the period after the word "add" which acts as a wildcard for all objects. Depending on the size of the RPD and MDS XML files this may take a few minutes.
5. Execute the following command to import the records into the new local SCM repository and officially commit the files as objects to be tracked by Git SCM:
   
   # git commit -m "Initial Commit"
   
   This may take several minutes to complete the first time it is conducted because a reference to every file in the MDS XML directory is taken into account. Be sure to use double-quotes around the commit message which comes after the -m flag (single quotes are not accepted and text without quotes is interpreted as part of the command).

Figure 5

You've now executed some rudimentary Git SCM instantiation commands. The git init command creates the repository folder .git in the root directory. The git add . command acknowledges all files and folders that should be added to the Git repository. Finally, the git commit -m "Initial Commit" command puts the files added from the git add . command into the repository with a comment to describe the transaction. Lastly, you can verify that the files were in fact committed by doing the following:

1. Execute the following command from the command prompt terminal in order to see that the commit of files added to the Git repository actually took place.
   
   # git log
2. You can even see a unique hash tag id for that commit which makes it identifiable among any future commits.

Developing with the MDS XML SCM Option

• Opening a Repository Under Source Control
• Making Changes in the RPD
• Verifying Changes to the SCM Repository
• Committing Changes Made to the SCM Repository

Opening a Repository Under Source Control

The concept of opening the Oracle BI repository in the Administration Tool is similar to opening the binary RPD file but opening the MDS XML repository under SCM can only be conducted in Offline mode:

1. Open the Administration Tool
2. Select File | Open | MDS XML… from the file menu or click the green folder from the file menu icons.
3. Select the root folder that you previously used to create the MDS XML. For example, C:\_Book\TennisRPD. The interface should select the directory last used for MDS XML activity.
4. Enter the repository password. For example, Admin123
.
5. Click the Use Source Control option on the Source Control prompt that appears.

Figure 6

6. Click the Browse… button in the Source Control prompt that just appeared to select the artofbi-git-scm.xml file that you created in a previous section.
7. Click the OK button. The metadata repository will now open in the Administration Tool.

If you look at the application title bar you will see that it is suffixed with "(under source control)" in the text. Now that you've opened the MDS XML directory under Source Control your SCM configuration options are automatically set to persist. So the next time you open this MDS XML directory in the Administration Tool you won't to choose the Source Control option—it will load automatically.

Making Changes in the RPD

In order to understand how changes to the repository under SCM affect the local Git repository, this exercise will step you through making a simple change to the repository in order to mimic a real-world development effort:

1. From the Business Model and Mapping (Logic layer) section expand the TennisBM Business Model.
2. Expand the Dim Player table.
3. Right-Click on the Nationality column and select Duplicate from the context menu.
4. Click the File | Save from the file menu to save the change.
5. Click the Yes button on the "Do you wish to check global consistency?" prompt that appears.
   
   After a successful Consistency Check the Source Control Management modified files prompt will appear as seen in the image below, which indicates that there are five files that will be modified based on the simple change that was made.

Figure 7

6. Click the Yes button in the prompt to continue.
   
   The Source Control Log prompt appears. This shows the actions that the Administration Tool has taken on behalf of source control by calling the commands you entered earlier in the artofbi-git-scm.xml file. At this point the SCM commands have been executed in the background and the RPD MDS XML files saved. There are two checkboxes, Close when commands finish and Only show dialog when errors occur. These checkboxes are unchecked and there is no real need to check them since it is likely you will always want to see the output from this process just in case you need to react immediately to any issues.

Figure 8

7. Click the Close button on the prompt to close it.

Verifying Changes to the SCM Repository

Since changes were visible in the Source Control Log and there were no error messages, it is safe to assume that the SCM commands were called properly by the Administration Tool. The files were modified and are placed in what is called the Unstaged Working Area of the Git repository. These unstaged files will need to be "staged." This is the term used before changes are permanently made to the current workspace, in this case the main branch, of the local SCM repository. It is best to verify the status of these files in the SCM system itself using the client tools. This can be done in two different ways: Command-Line or Graphically.

To view status of any modified files using the Command Line Interface, follow these steps:

1. Open a Command Prompt and navigate to the root directory from which you've saved the MDS XML directory and created your Git repository.
2. Execute the command:
   
   # git status

The result of this command shows the same information that was shown in the Administration Tool prompt after you saved the changes you made in the previous step.

To view the status of any modified files using the Git GUI, follow these steps:

1. Open a Windows Explorer Window
2. Navigate to the root directory where you created the Git Repository. For example, C:\_Book\TennisRPD\
3. Right-click on the .git directory (a hidden folder that can be shown using the Folder Options) and select Git GUI from the context menu. Remember Git GUI was installed when you installed the Git client on your Windows machine.

Figure 9

The files inside the Unstaged Changes area show the same files shown to be modified when we saved the repository. This verifies that the Administration Tool and the SCM commands have worked successfully together at this point. You've now completed the sanity check on the modified files and now you're ready to stage and commit those files to the local Git repository in order to solidify the changes.

Committing Changes Made to the SCM Repository

Moving modified files and changes in the Unstaged Changes area to Staged Changes status and conducting the commit of those changes cannot be done by the Administration Tool. It must be done by the developer using the SCM client. This is the case regardless of which SCM tool you've chosen to use. Either the Command Line Interface (CLI) or the GUI interface could be used. The Git GUI interface is fairly straightforward - just click the Stage Changed button, enter a message in the Commit Message area, and click the Commit button. The steps outlined below will use the CLI in order to commit the changes.

1. Open a command prompt window and change the directory to the root directory where you created the Git repository. For example, C:\_Book\TennisRPD.
2. Execute the command:
   
   # git add .
3. Enter the command:
   
   # git commit - m "First full changes to the MDS XML SCM"

You can now verify that the changes were committed to the repository by entering the commands git status or git log.

Conducting Ancillary SCM Tasks

After you've committed the changes manually using the Git SCM client tools you have successfully leveraged the Oracle BI SCM functionality. There are a few other ancillary tasks that you will wish to conduct as part of the development lifecycle.

Deploying an MDS XML Reposoitory to the Oracle BI Server

One of the next tasks that will arise during the development cycle is for you to deploy the repository into online mode so that you can view and test any changes that you've been making offline using the MDS XML SCM functionality. You cannot deploy the MDS XML files directly to the Oracle BI server. You must convert the files back to a binary RPD format before deploying.

To save the MDS XML back to a binary RPD format:

1. While the MDS XML repository under source control is open, click File | Save As | Repository… from the file menu in the Administration Tool.
2. Enter a name for the RPD in the Save As prompt that appears
3. Click the Save button.

Notice that the title of the Administration Tool application has changed back to the name of the RPD and is no longer displaying the text suffix, (under source control). You may now use the Enterprise Manager Fusion Control to upload this saved binary RPD file to the Oracle BI Server.

Getting to the Oracle BI SCM Log Files

Another task that you should be aware of is viewing the log file and handling errors. If an error occurs, always be sure that you use the method described above to save the MDS XML back into a binary RPD format. You can then address the errors you've encountered occurring between the Administration Tool and your SCM repository using the SCM client tools. Once you've fixed the issue you may re-open the MDS XML files and continue with development.

The log file containing all commands sent to your SCM tool is generally available for you to review. While in the MDS XML SCM mode select File | Source Control | View Logs from the file menu bar. This log is also available on the file system from the following location, C:\Program Files\OBIEEClient11116\oraclebi\orainst\diagnostics\logs\OracleBIServerComponent\coreapplication\<windows_username>__NQSAdminTool.log where C:\Program Files\OBIEClient11116\ is the path in which the Oracle BI Client tools were installed.

Final Thoughts

The general idea of using a Source Control Management system is not new to Oracle but is new to the Oracle BI development lifecycle. The ability to split the repository into many smaller XML files allows for using a differencing engine to determine the delta of modified files. It also makes searching for specific keywords across the repository metadata much easier. It is no replacement for the existing Multiuser Development Environment and although there are only two default SCM command templates provided with the Oracle BI installation files there is no guidance on whether one SCM tool is better than another for this functionality. Regardless, the SCM agnostic capability provided by this functionality is a progressive option for the tool which should assist with its adoption by Oracle BI developers.

Addtional References

• http://git-scm.com/docs
• http://git-scm.com/book/en/Getting-Started-Installing-Git
• http://docs.oracle.com/cd/E25178_01/bi.1111/e23950/mds_about.htm
• http://docs.oracle.com/cd/E25178_01/fusionapps.1111/e20836/scm.htm
• https://github.com/artofbi/OBIMDSXML-SCM-Config-Templates

Downloads

• http://git-scm.com/downloads
• https://github.com/artofbi/obi11gbook/downloads
• https://github.com/artofbi/OBIMDSXML-SCM-Config-Templates

About the Author

Christian Screen, a Senior Manager for Capgemini North America's Oracle Analytics Practice, is a Business Intelligence evangelist with over 15 years of experience in technology, ranging from low-level programming, e-commerce, Data Warehousing, Enterprise Performance Management, and Business Intelligence. He is an Oracle ACE, an Oracle Deputy CTO, and holds several technology and project management certifications.