Oracle9i Real Application Clusters 
(RAC) on HP-UX

Authors:        Rebecca Kühn, Rainer Marekwia                        HP/Oracle Cooperative Technology Center ( http://www.hporaclectc.com) Co-Authors:  Sandy Gruver, HP Alliance Team US                         Troy Anthony, Oracle RAC Development

Contents:

1. Module Objectives

2. Overview: What is Oracle9i Real Applications Clusters?

3. Oracle 9i Real Application Clusters � Cache Fusion technology

4. New HP Cluster Interconnect technology

5. HP/Oracle Hardware and Software Requirements
    5.1 General Notes
    5.2 System Requirements
    5.3 HP-UX Operating System Patches
    5.4 Kernel Parameters
    5.5 Asynchronous I/O

6. Configure the HP/Oracle 9i Real Application Cluster
    6.1 Hardware configuration (Hardware planning, Network and disk layout)
    6.2 Configure logical volumes
    6.3 Configure HP ServiceGuard cluster
    6.4 Create a user who will own the Oracle RAC software
    6.5 Install Oracle Software
    6.6 Create a Oracle9i database using DBCA

1. Module Objectives

Purpose

Objectives

• What Oracle9i Real Application Clusters is and how it can be used
• Understand the hardware requirements
• Understand how to configure the HP Cluster and create the raw devices
• Examine and verify the cluster configuration
• Understand the creation of the Oracle9i Real Application Clusters database using DBCA
Back to Top

2. Overview: What is Oracle9i Real Applications Clusters?

In Oracle9i Real Application Clusters (RAC) environments, all nodes concurrently execute transactions against the same database. Oracle9i Real Application Clusters coordinates each node's access to the shared data to provide consistency and integrity.

Oracle9i Real Application Clusters serves as an important component of robust high availability solutions. A properly configured Oracle9i Real Application Clusters environment can tolerate failures with minimal downtime.

Oracle9i Real Application Clusters is also applicable for many other system types. For example, data warehousing applications accessing read-only data are prime candidates for Oracle9i Real Application Clusters. In addition, Oracle9i Real Application Clusters successfully manages increasing numbers of online transaction processing systems as well as hybrid systems that combine the characteristics of both read-only and read/write applications.

Harnessing the power of multiple nodes offers obvious advantages. If you divide a large task into sub-tasks and distribute the sub-tasks among multiple nodes, you can complete the task faster than if only one node did the work. This type of parallel processing is clearly more efficient than sequential processing. It also provides increased performance for processing larger workloads and for accommodating growing user populations. Oracle9i Real Application Clusters can effectively scale your applications to meet increasing data processing demands. As you add resources, Oracle9i Real Application Clusters can exploit them and extend their processing powers beyond the limits of the individual components.

From a functional perspective RAC is equivalent to single-instance Oracle. What the RAC environment does offer is significant improvements in terms of availability, scalability and reliability.

In recent years, the requirement for highly available systems, able to scale on demand, has fostered the development of more and more robust cluster solutions. Prior to Oracle9i, HP and Oracle, with the combination of Oracle Parallel Server and HP ServiceGuard OPS edition, provided cluster solutions that lead the industry in functionality, high availability, management and services. Now with the release of Oracle 9i Real Application Clusters (RAC) with the new Cache Fusion architecture based on an ultra-high bandwidth, low latency cluster interconnect technology, RAC cluster solutions have become more scalable without the need for data and application partitioning.
 

The information contained in this document covers the installation and configuration of Oracle Real Application Clusters in a typical environment; a two node HP cluster, utilizing the HP-UX operating system.

Back to Top

3. Oracle 9i Real Application Clusters � Cache Fusion technology

Oracle 9i cache fusion thus provides Oracle users an expanded database cache for queries and updates with reduced disk I/O synchronization which overall speeds up database operations.
However, the improved performance depends greatly on the efficiency of the inter-node message passing mechanism, which handles the data block transfers between nodes.

The efficiency of inter-node messaging depends on three primary factors:

• The number of messages required for each synchronization sequence. Oracle 9i�s Global Cache Manager GCM (Global Cache Services GCS/ Global Enqueue Services GES) coordinates the fast block transfer between nodes with two inter- node messages and one intra-node message. If the data is in a remote cache, an inter-node message is sent to the Lock Manager Services (LMS) on the remote node. The GCM and Cache Fusion processes then update the in-memory lock structure and send the block to the requesting process.
• The frequency of synchronization (the less frequent the better). The cache fusion architecture reduces the frequency of the inter-node communication by dynamically migrating locks to a node that shows a frequent access pattern for a particular data block. This dynamic lock allocation increases the likelihood of local cache access thus reducing the need for inter-node communication. At a node level, a cache fusion lock controls access to data blocks from other nodes in the cluster.
• The latency of inter-node communication. This is a critical component in Oracle 9i RAC as it determines the speed of data block transfer between nodes. An efficient transfer method must utilize minimal CPU resources, support high availability as well as highly scalable growth without bandwidth constraints.

  

Back to Top

4. New HP Cluster Interconnect technology

• HyperFabric

HyperFabric is a high-speed cluster interconnect fabric that supports both the industry standard TCP/UDP over IP and HP�s proprietary Hyper Messaging Protocol (HMP). HyperFabric extends the scalability and reliability of TCP/UDP by providing transparent load balancing of connection traffic across multiple network interface cards (NICs) and transparent failover of traffic from one card to another without invocation of MC/ServiceGaurd. The HyperFabric NIC incorporates a network processor that implements HP�s Hyper Messaging Protocol and provides lower latency and lower host CPU utilization for standard TCP/UDP benchmarks over HyperFabric when compared to gigabit Ethernet. Hewlett-Packard released HyperFabric in 1998 with a link rate of 2.56 Gbps over copper. In 2001, Hewlett-Packard released HyperFabric 2 with a link rate of 4.0 Gbps over fiber with support for compatibility with the copper HyperFabric interface. Both HyperFabric products support clusters up to 64-nodes.

• HyperFabric Switches

Hewlett-Packard provides the fastest cluster interconnect via its proprietary HyperFabric switches, the latest product being HyperFabric 2, which is a new set of hardware components with fiber connectors to enable low-latency, high bandwidth system interconnect. With fiber interfaces, HyperFabric 2 provides faster speed � up to 4Gbps in full duplex over longer distance � up to 200 meters. HyperFabric 2 also provides excellent scalability by supporting up to 16 hosts via point-to-point connectivity and up to 64 hosts via fabric switches. It is backward compatible with previous versions of HyperFabric and available on IA-64, PA-RISC servers.
 
• Hyper Messaging Protocol (HMP)

Hewlett-Packard, in cooperation with Oracle, has designed a cluster interconnect product specifically tailored to meet the needs of enterprise class parallel database applications. HP�s Hyper Messaging Protocol significantly expands on the feature set provided by TCP/UDP by providing a true Reliable Datagram model for both remote direct memory access (RDMA) and traditional message semantics. Coupled with OS bypass capability and the hardware support for protocol offload provided by HyperFabric, HMP provides high bandwidth, low latency and extremely low CPU utilization with an interface and feature set optimized for business critical parallel applications such as Oracle 9i RAC.
Back to Top

5. HP/Oracle Hardware and Software Requirements

5.1 General Notes

• Each node uses the HP-UX 11.x operating system software. Issue the command "$uname -r" at the operating system prompt to verify the release being used.
• Oracle9i RAC is only available in 64-bit flavour. To determine if you have a 64-bit configuration on an HP-UX 11.0 installation, enter the following command: $/bin/getconf KERNEL_BITS
• Oracle9i 9.0.1 RAC is supported by ServiceGuard OPS Edition 11.09 and 11.13.
• Starting with ServiceGuard OPS Edition 11.13 16 nodes 9i RAC cluster is supported with SLVM.
• Software mirroring with HP-UX MirrorDisk/UX with SLVM is supported in a 2-node configuration only.
• Support for the HP Hyperfabric product is provided.
• A total of 127 RAC instances per cluster is supported.

5.2 System Requirements

• RAM Memory allocation: Minimum 256 MB. Use the following command to verify the amount of memory installed on your system
$ /usr/sbin/dmesg | grep "Physical:"
• Swap Space: Minimum 2 x RAM or 400 MB, whichever is greater. Use the following command to determine the amount of swap space installed on your system:
$ /usr/sbin/swapinfo -a (requires root privileges)
CD-ROM drive: A CD-ROM drive capable of reading CD-ROM disks in the ISO 9660 format with RockRidge extensions.
Available Disk Space: 3 GB
Temporary Disk Space: The Oracle Universal Installer requires up to 512 MB of space in the /tmp directory
• Operating System: HP-UX version 11.0. or 11i (11.11). To determine if you have a 64-bit configuration on an HP-UX 11.0 or HP-UX 11i installation, enter the following command:
$/bin/getconf KERNEL_BITS
• To determine your current operating system information, enter the following command:

$ uname -a

JRE: Oracle applications use JRE 1.1.8.
JDK: Oracle HTTP Server Powered by Apache uses JDK 1.2.2.05.
• Due to a known HP bug (Doc.id. KBRC00003627), the default HP-UX 64 operating system installation does not create a few required X-library symbolic links. These links must be created manually before starting Oracle9i installation. To create these links, you must have superuser privileges, as the links are to be created in the /usr/lib directory. After enabling superuser privileges, run the following commands to create the required links:
$ cd /usr/lib
$ ln -s /usr/lib/libX11.3 libX11.sl
$ ln -s /usr/lib/libXIE.2 libXIE.sl
$ ln -s /usr/lib/libXext.3 libXext.sl
$ ln -s /usr/lib/libXhp11.3 libXhp11.sl
$ ln -s /usr/lib/libXi.3 libXi.sl
$ ln -s /usr/lib/libXm.4 libXm.sl
$ ln -s /usr/lib/libXp.2 libXp.sl
$ ln -s /usr/lib/libXt.3 libXt.sl
$ ln -s /usr/lib/libXtst.2 libXtst.sl
 
• X Server system software. (Refer to the installation guide for more information on the X Server System and emulator issues.)

  

Back to Top

5.3 HP-UX Operating System Patches

11.0 (64bit):

• Sep 2001 HP-UX patch bundle
• PHCO_23963
• PHCO_24148
• PHCO_23919
• PHKL_23226
• PHNE_24034
• PHSS_23440
• hyperfabric driver: 11.00.12 (HP-UX 11.0) (Required only if your system has an older hyperfabric driver version)

11i (64bit):

• Sep 2001 HP-UX patch bundle
• PHCO_23094
• PHCO_23772
• PHSS_23441
• PHNE_23502

HP provides patch bundles at

http://www.software.hp.com/SUPPORT_PLUS

http://itresourcecenter.hp.com

$ /usr/sbin/swlist -l patch

$ /usr/sbin/swlist -l patch patch_number

$ /usr/sbin/swlist -l bundle

Back to Top

5.4 Kernel parameters

Back to Top

5.5  Asynchronous I/O

$ /sbin/mknod /dev/async c 101 0x0
$ chown oracle:dba /dev/async
$ chmod 660 /dev/async

=> Kernel Configuration
    => Kernel
         => the driver is called 'asyncdsk'
Generate new kernel
Reboot

4. Set HP-UX kernel parameter aio_max_ops using SAM. aio_max_ops limits the maximum number of asynchronous i/o operations that can be queued at any time. Set this parameter to the default value (2048), and monitor over time using glance

Back to Top

6. Configure the HP/Oracle 9i Real Application Cluster

6.1 Hardware configuration (Hardware planning, Network and disk layout)

Hardware Planning

The nodes in an Oracle9i RAC cluster are HP 9000 systems with similar memory configuration and processor architecture. A node can be any Series 800 model. It is recommended that both nodes be of similar processing power and memory capacity.

An RAC cluster must have:

• Two or more nodes
• Rredundant high-speed interconnect between the nodes (e.g. Hyperfabric, Hyperfabric switches)
• Redundant network components (Primary and Standby LAN)
• Redundant disk storage or RAID0/1 configuration for disk mirroring
• A dedicated heartbeat LAN (heartbeat traffic is also carried on the primary and standby LAN)
• Redundant power supplies

Network and Disk Layout

1. Use the LAN commands
        $ lanscan
        $ ifconfig lanX, and
        $ netstat

to determine the number of LAN interfaces on each node and the names and addresses of each LAN card and subnet information.

2. Use the IO command
        $ ioscan �fnCdisk
to find the disks connected to each node. Note the type of disks installed. List the hardware addresses and device file names of each disk. Also note which are shared between nodes.

Network Planning

Minimally, a 9i RAC cluster requires three distinct subnets:
• Dedicated cluster heartbeat LAN
• Dedicated Global Cache Management (GCM) LAN
• User/Data LAN, which will also carry a secondary heartbeat
Because the GCM is now integrated into the Oracle9i kernel, the GCM will use the IP address associated with the default host name.
The network should be configured in the /etc/rc.config.d/netconf file. Any time you change the LAN configuration, you need to stop the network and re-start it again:

        $ /sbin/rc2.d/S340net stop
        $ /sbin/rc2.d/S340net start

GCM requires a high speed network to handle high bandwidth network traffic. In the Oracle literature this is referred to as the host interconnect. We recommend using either Hyperfabric or Gigabit Ethernet for this network.
Remote copy(rcp) needs to be enabled for both the root and oracle accounts on all nodes to allow remote copy of cluster configuration files.

There are two ways to enable rcp for root. You can choose the one that fits your site�s security requirements. Include the following lines in either the .rhosts file in root�s home directory or in the /etc/cmcluster/cmclnodelist file:

        node1name root
        node2name root

To enable remote copy (rcp) for Oracle include the following lines in the .rhosts file in the oracle user�s home directory:

        node1name oracle
        node2name oracle

where node1name and node2name are the names of the two systems in the cluster and oracle is the user name of the Oracle owner. The rcp only works if for the respective user a password has been set (root and oracle).

Back to Top

6.2 Configure logical volumes

General Recommendations

· Place redo logs and database files onto different drives
· Ensure that data and indexes were on separate spindles
· Spread the I/O load across as many disk devices as possible

Today with the greatly increased capacity of a single disk mechanism (maximum 181Gb drives on an XP512) and much faster I/O rates or transfer speeds, these rules must be revisited.

The real reason for these rules of thumb was to make sure that the I/O load resulting from an Oracle database would wind up being fairly well spread across all the disk mechanisms.  Before the advent of large capacity disk drives housed in high performance storage systems, if the same disk drive wound up hosting two or more fairly active database objects, performance could deteriorate rapidly, especially if any of these objects needed to be accessed sequentially.

Today, in the era of huge disk arrays, the concept of �separate spindles� is a bit more vague, as the internal structure of the array is largely hidden from the view of the system administrator.  The smallest independent unit of storage in an XP array is substantially larger than 1 or 2 GB, which means you have far fewer �spindles� to play with, at a time when there are more database objects (tables, indexes, etc) to �spread�, so it won�t be possible to keep all the objects separate. The good news is that the architecture of the XP array is much more tolerant of multiple simultaneous I/O streams to/from the same disk mechanism than the previous generation of individual small disks.

Given all these advances in the technology, we have found it best to use a simple method for laying out an Oracle database on an XP array (under HP-UX)  with volume manager striping of all of the database objects across large numbers of disk mechanisms. The result is to average out the I/O to a substantial degree. This method does not guarantee the avoidance of disk hotspots, but we believe it to be a reasonable �first pass� which can be improved upon with tuning over time. It�s not only a lot faster to implement than a customized one-object-at-a-time layout, but we believe it to be much more resistant to the inevitable load fluctuations which occur over the course of a day, month, or year.

The layout approach that we are advocating might be described as �Modified Stripe-Everything- -Across-Everything�. Our goal is to provide a simple method which will yield good I/O balance, yet still provide some means of manual adjustment. Oracle suggests the same strategy.  Their name for this strategy is SAME (Stripe and Mirror Everything).

XP basics: an XP512 can be configured with one to four pairs of disk controller modules (ACPs).  Each array group is controlled by only one of these ACP pairs (it is in the domain of only one ACP pair).  Our suggestion is that you logically �separate� the XP�s array groups into four to eight sets. Each set should have array groups from all the ACP domains. Each set of array groups would then be assigned to a single volume group. All LUNs in the XP array will have paths defined via two distinct host-bus adapters; the paths should be assigned within each volume group in such a fashion that their primary path alternates back and forth between these two host-bus adapters. The result of all this: each volume group will consist of space which is �stripable� across multiple array groups spread across all the ACP pairs in the array, and any I/O done to these array groups will be spread evenly across the host-bus adapters on the server.

LVM Steps

1. Disks need to be properly initialized before being added into volume groups by the pvcreate command. Do the following step for all the disks (LUNs) you want to configure for your 9i RAC volume group(s):
        $ pvcreate �f /dev/rdsk/cxtydz ( where x=instance, y=target, and z=unit)

2. Create the volume group directory with the character special file called group:
        $ mkdir /dev/vg_rac
        $ mknod /dev/vg_rac/group c 64 0x060000

Note: The minor numbers for the group file should be unique among all the volume groups on the system.

3. Create PV-LINKs and extend the volume group:
        $ vgcreate /dev/vg_rac /dev/dsk/c0t1d0 /dev/dsk/c1t0d0
        $ vgextend /dev/vg_rac /dev/dsk/c1t0d1 /dev/dsk/c0t1d1

Continue with vgextend until you have included all the needed disks for the volume group(s).

4. Create logical volumes for the 9i RAC database with the command
        $ lvcreate �i 10 �I 1024 �L 100 �n Name /dev/vg_rac
        -i: number of disks to stripe across
        -I: stripe size in kilobytes
        -L: size of logical volume in MB

5. Logical Volume Configuration

It is necessary to define raw devices for each of the following categories of files. The Oracle Database Configuration Assistant (DBCA) will create a seed database expecting the following configuration:

 

Create a Raw Device for:	File Size	Sample name
SYSTEM tablespace	400 MB	db_name_raw_system_400
USERS tablespace	120 MB	db_name_raw_user_120
TEMP tablespace	100 MB	db_name_raw_temp_100
An undo tablespace per instance	500 MB	db_name_thread_raw_undo_500
OEMREPO	20 MB	db_name_raw_oemrepo_20
INDX tablespace	70 MB	db_name_raw_indx_70
TOOLS tablespace	12 MB	db_name_raw_tools_12
DRYSYS tablespace	90 MB	db_name_raw_dr_90
First control file	110 MB	db_name_raw_control01_110
Second control file	110 MB	db_name_raw_control02_110
Two ONLINE redo log files per instance	120 MB per file	db_name_thread_lognumb_120
Spfile.ora	5 MB	db_name_raw_spfile_5
Srvmconfig	100 MB	db_name_raw_srvmconf_100
EXAMPLE	160 MB	db_name_raw_examples_160

Note: Automatic Undo Management requires an undo tablespace per instance therefore you would require a minimum of 2 tablespaces as described above.

By following the naming convention described in the table above, raw partitions are identified with the database and the raw volume type (the data contained in the raw volume). Raw volume size is also identified using this method. Note : In the sample names listed in the table, the string db_name should be replaced with the actual database name, thread is the thread number of the instance, and lognumb is the log number within a thread.

It is recommended best practice to create symbolic links for each of these raw files on all systems of your RAC cluster.

6. Check to see if your volume groups are properly created and available:
 

        $ strings /etc/lvmtab
        $ vgdisplay �v /dev/vg_rac

7. Change the permission of the database volume group vg_ops to 777, change the permissions of all raw logical volumes to 660 and the owner to oracle:dba.
        $ chmod 777 /dev/vg_rac
        $ chmod 660 /dev/vg_rac/r*
        $ chown oracle:dba /dev/vg_rac/r*
 

8. Export the volume group:

De-activate the volume group
        $ vgchange �a n /dev/vg_rac
Create the volume group map file:
        $ vgexport �v �p �s �m mapfile /dev/vg_rac
Copy the mapfile to all the nodes in the cluster:
        $ rcp mapfile system_name:target_directory
        $ rcp map_ops nodeB:/tmp/scripts
        $ chown oracle:dba /dev/vg_rac/r*
9. Import the volume group on the second node in the cluster
• Create a volume group directory with the character special file called group:

        $ mkdir /dev/vg_rac
        $ mknod /dev/vg_rac/group c 64 0x060000
Note: The minor number has to be the same as on the other node.
Import the volume group:
        $ vgimport �v �s �m mapfile /dev/vg_rac
• Check to see if devices are imported:

        $ strings /etc/lvmtab
Back to Top

6.3 Configure HP ServiceGuard Cluster

At this time, the cluster lock volume group should have been created. Since we
only configured one volume group for the entire OPS cluster vg_ops, we used vg_ops for the lock volume as well.

1. Create a cluster configuration template:
        $ cmquerycl �n nodeA �n nodeB �v �C /etc/cmcluster/rac.asc
2. Edit the cluster configuration file (rac.asc).
The cluster configuration file should be set with both DLM and GMS enabled clauses to NO, since neither need to be configured with Oracle9i RAC. Global Cache management is now handled transparently by the Oracle kernel. For compatibility with older versions of Oracle, the cluster configuration file still contains a section for DLM and GMS.

Make the necessary changes to this file for your cluster. For example, change the ClusterName, and adjust the heartbeat interval and node timeout to prevent unexpected failovers due to GCM traffic.

3. Check the cluster configuration:
        $ cmcheckconf -v -C rac.asc

4. IMPORTANT! Activate the lock disk on the configuration node ONLY. Lock volume can only be activated on the node where the cmapplyconf command is issued so that the lock disk can be initialized accordingly.


        $ vgchange -a y /dev/vg_rac

5. Create the binary configuration file and distribute the cluster configuration to all the nodes in the cluster:


        $ cmapplyconf -v -C rac.asc
Note: the cluster is not started until you run cmrunnode on each node or cmruncl.

6. De-activate the lock disk on the configuration node after cmapplyconf
        $ vgchange -a n /dev/vg_rac
7. Start the cluster and view it to be sure its up and running. See the next section for instructions on starting and stopping the Cluster. After testing the cluster, shut it down in order to make changes later to the HP-UX kernel parameters.

Starting the Cluster

Start the cluster from any node in the cluster
        $ cmruncl -v

Or, on each node

        $ cmrunnode -v

Make all RAC volume groups and Cluster Lock volume groups sharable and cluster aware (not packages) from the cluster configuration node:

        $ vgchange -S y -c y /dev/vg_rac

On all the nodes, activate the volume group in shared mode in the cluster:

        $ vgchange �a s /dev/vg_rac
Check the cluster status:

        $ cmviewcl �v

Shutting Down the Cluster

Shut down the 9i RAC instances (If up and running)
On all the nodes, deactivate the volume group in shared mode in the cluster:
        $ vgchange �a n vg_rac

Halt the cluster from any node in the cluster

        $ cmhaltcl �v
Check the cluster status:

        $ cmviewcl �v

Back to Top

6.4 Create a user who will own the Oracle RAC software

1. Log in as the root user.

2. Create database administrator groups by using the System Administrator's Menu (SAM).

The OSDBA group, typically dba.
• The optional OSOPER, group, typically oper.
• The optional ORAINVENTORY group, typically oinstall.
Grant the OSDBA group RTSCHED, RTPRIO and MLOCK privileges.
A new HP scheduling policy called SCHED_NOAGE enhances Oracle9i's performance by scheduling Oracle processes so that they do not increase or decrease in priority, or become preempted.
The RTSCHED and RTPRIO privileges grant Oracle the ability to change its process scheduling policy to SCHED_NOAGE and also tell Oracle what priority level it should use when setting the policy. The MLOCK privilege grants Oracle the ability to execute asynch I/Os through the HP asynch driver. Without this privilege, Oracle9i generates trace files with the following error message: "Ioctl ASYNCH_CONFIG error, errno = 1".

As root, do the following:

If it does not already exist, create the /etc/privgroup file. Add the following line to the file:
        dba  MLOCK RTSCHED RTPRIO
Use the following command syntax to assign these privileges:
        $ setprivgrp -f /etc/privgroup

In the preceding command, groupname is the name of the group that receives the privileges, and privileges are the privileges that are granted to the group.

If it does not already exist, create a local bin directory, such as /usr/local/bin or /opt/bin. Set and verify that this directory is included in each user's PATH statement, and that users have execute permissions on the directory.

Determine if your X Window system is working properly on your local system. On the system where you will run the Oracle Universal Installer, set DISPLAY to that system's name, or the IP address, X server, and screen.

Use the database server's name, or the IP address, X server, and screen only if you are performing the installation from your database server's X Window console. If you are not sure what the X server and screen should be set to, use 0 (zero) for both.

Set a temporary directory path for the TMPDIR variable with at least 512 MB of free space where the installer has write permission. Example: /var/tmp.

# Oracle Environment
export ORACLE_BASE=/opt/
export ORACLE_HOME=$ORACLE_BASE/oracle/product/9.0.1
export ORACLE_SID=rac2
export ORACLE_TERM=xterm
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export LD_LIBRARY_PATH
SHLIB_PATH=ORACLE_HOME/lib32:$ORACLE_HOME /rdbms/lib32
export SHLIB_PATH

# Set shell search paths:
export PATH=$PATH:$ORACLE_HOME/bin

#CLASSPATH must include the following JRE locations:
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib

Create the /var/opt/oracle directory and make it owned by the oracle account. After installation, this directory will contain a few small text files that briefly describe the Oracle software installations and databases on the server. These commands will create the directory and give it appropriate permissions:

        $ mkdir /var/opt/oracle
        $ chown oracle:dba /var/opt/oracle
        $ chmod 755 /var/opt/oracle

A user needs to be created, on all nodes, to manage the installation and administration of the Oracle software. The username oracle is used in this example, but it need not be. The user account and associated group entries must be defined on all nodes of the cluster

Back to Top

6.5 Install Oracle Software

    /Disk1/<contents of disk1>
    /Disk2/<contents of disk2>
    /Disk3/<contents of disk3>

Once Install is selected, the OUI will install the Oracle RAC software on to the local node, and then copy software to the other nodes selected earlier. This will take some time. During the installation process, the OUI does not display messages indicating that components are being installed on other nodes - I/O activity may be the only indication that the process is continuing.

Back to Top

6.6 Create a Database using the Oracle Database Configuration Assistant (DBCA)

DBCA provides three primary processing phases:

• Verification of shared disk configuration (for non-cluster file system platforms)
• Create the database
• Configure the Oracle network services

 

1.	Run the scripts $ORACLE_HOME/rdbms/admin/catclust.sql after DB creating
2.	During the creation of the database set the parameter UNDO_TABLESPACE = FALSE
3.	Specify a tablespace name as the UNDO_TABLESPACE, not a filename.

 

1.	DBCA will launch as part of the installation process, but can be run manually by executing the command dbca from the $ORACLE_HOME/bin directory on UNIX platforms. Choose Oracle Cluster Database option and select Next.
2.	The Operations page is displayed. Choose the option Create a Database and click Next.
3.	The Node Selection page appears. Select the nodes that you want to configure as part of the RAC database and click Next.
4.	The Database Templates page is displayed. The templates other than New Database include datafiles. Choose New Database and then click Next .
5.	The Show Details button provides information on the database template selected.
6.	DBCA now displays the Database Identification page. Enter the Global Database Name and Oracle System Identifier (SID) . The Global Database Name is typically of the form name.domain. Note: Please specify a TABLESPACE as the UNDO_TABLSPACE, not a file name. During database creating set UNDO_MANAGEMENT=FALSE.
7.	The Database Options page is displayed. Select the options you wish to configure and then choose Next. Note: If you did not choose New Database from the Database Template page, you will not see this screen.
8.	The Additional database Configurations button displays additional database features. Make sure both are checked and click OK.
9.	Select the connection options desired from the Database Connection Options page. Note: If you did not choose New Database from the Database Template page, you will not see this screen. Click Next .
10.	DBCA now displays the Initialization Parameters page. This page comprises a number of Tab fields. Modify the Memory settings if desired and then select the File Locations tab to update information on the Initialization Parameters filename and location. Then click Next .
11.	The option Create persistent initialization parameter file is selected by default. Enter raw device name for the location of the server parameter file (spfile). Then click Next .
12.	The button File Location Variables� displays variable information. Click OK.
13.	The button All Initialization Paramters� displays the Initialization Parameters dialog box. This box presents values for all initialization parameters and indicates whether they are to be included in the spfile to be created through the check box, included (Y/N). Instance specific parameters have an instance value in the instance column. Complete entries in the All Initialization Parameters page and select Close . Ensure all entries in the Initialization Parameters page are complete and select Next.
14.	DBCA now displays the Database Storage Window. This page allows you to enter file names for each tablespace in your database.
15.	The file names are displayed in the Datafiles folder, but are entered by selecting the Tablespaces icon, and then selecting the tablespace object from the expanded tree. Any names displayed here can be changed. Complete the database storage information and click Next.
16.	The Database Creation Options page is displayed. Ensure that the option Create Database is checked and click Finish .
17.	The DBCA Summary window is displayed. Review this information and then click OK.
18.	Once the Summary screen is closed using the OK option, DBCA begins to create the database according to the values specified.

A new database now exists. It can be accessed via Oracle SQL*PLUS or other applications designed to work with an Oracle RAC database.