How to Deploy Oracle RAC 11.2.0.2 On Oracle Solaris Zone Clusters

Using Oracle RAC 11.2.0.2, Oracle Solaris 10, and Oracle Solaris Cluster 3.3 5/11

by Vinh Tran

How to create an Oracle Solaris Zone cluster, install and configure Oracle Grid Infrastructure 11.2.0.2 and Oracle Real Application Clusters 11.2.0.2 in the zone cluster, and create an Oracle Solaris Cluster resource for Oracle RAC.


OTN is all about helping you become familiar enough with Oracle technologies to make an informed decision. Articles, software downloads, documentation, and more. Join up and get the technical resources you need to do your job.

Published May 2012

Download the Oracle Software Patches
Prerequisites
Create the Zone Cluster Using the cfg File
Create the Oracle RAC Framework for the Zone Cluster
Set Up the Root Environment in Local Zone Cluster z11gr2A
Create a User and Group for the Oracle Software
Install Oracle Grid Infrastructure 11.2.0.2 in the Oracle Solaris Zone Cluster Nodes
Replace OPatch Version 11.2.0.1.1 with Version 11.2.0.1.6
Apply Oracle Grid Infrastructure 11.2.0.2.3 GI PSU patch to GRID_HOME Using OPatch
Apply Patch 10055663 to GRID_HOME Using OPatch
Configure Oracle Grid Infrastructure
Install and Patch Oracle Database
Apply Oracle Grid Infrastructure 11.2.0.2.3 GI PSU patch  to Oracle Database Using OPatch
Create the Database
Create the Oracle Solaris Cluster Resource
See Also
About the Author

This article describes how to use Oracle Solaris Cluster 3.3 5/11 to provide a virtual, multitiered architecture for Oracle Real Application Cluster (Oracle RAC) 11.2.0.2.  This is one of three similar articles that explain how to do the same thing but with different versions of the software.  This table summarizes them:

Article RAC Version Solaris Version Cluster Version
How to Deploy Oracle RAC 11.2.0.2 on Oracle Solaris Zone Clusters 11.2.0.2 10 3.3
How to Deploy Oracle RAC 11.2.0.3 on Oracle Solaris Zone Clusters 11.2.0.3 10 3.3
How to Deploy Oracle RAC 11.2.0.3 on Oracle Solaris 11 Zone Clusters 11.2.0.3 11 4.0

 

Oracle Solaris Cluster 3.3 provides the capability to create high-availability zone clusters. A zone cluster consists of several Oracle Solaris Zones, each of which resides on its own separate server; the zones that comprise the cluster are linked together into a single virtual cluster. Because zone clusters are isolated from each other, they provide increased security. Because the zones are clustered, they provide high availability for the applications they host.

By installing Oracle RAC inside a zone cluster, you can run multiple instances of an Oracle database at the same time. This allows you to have separate database versions or separate deployments of the same database (for example, one for production and one for development). Using this architecture, you can also deploy different parts of your multitiered solution into different virtual zone clusters. For example, you could deploy Oracle RAC and an application server in different zones of the same cluster. This approach allows you to isolate tiers and administrative domains from each other, while taking advantage of the simplified administration provided by Oracle Solaris Cluster.

Note: This document is not a performance best practices guide and it does not cover the following topics:

  • Oracle Solaris OS installation
  • Storage configuration
  • Network configuration
  • Oracle Solaris Cluster installation

Download the Oracle Software Patches

To download Oracle patches, do the following after you have logged in to My Oracle Support:

  1. Click the Patches & Updates tab.
  2. In the Patch Name or Number field, type the <patch ID>.
  3. In the Platform field, select the relevant platform.
  4. Click the Search button.
  5. Select the patch that corresponds to the Oracle release that is installed on your system.

These are the Oracle software patches you need to download:

  • 11.2.0.2 Patch Set for Oracle Database Server, patch ID 10098816
  • 11.2.0.2.3 GI PSU patch, patch ID 12419353
  • Patch ID 6880880 for OPatch
  • Patch ID 10055663 for the bug CONFIGURATION WIZARD FAILS TO PROCEED AFTER NODES ARE SPECIFIED

Prerequisites

Ensure that the following prerequisites have been met.

  • Oracle Solaris 10 9/10 and Oracle Solaris Cluster 3.3 5/11 are already installed and configured.
     
  • Oracle Solaris 10 kernel parameters are configured in the /etc/system file in the global zone. Examples of the recommended values are as follows:

    noexec_user_stack 1 
    semsys:seminfo_semmni 100 
    semsys:seminfo_semmns 1024 
    semsys:seminfo_semmsl  256 
    semsys:seminfo_semvmx 32767 
    shmsys:shminfo_shmmax 4294967295 
    shmsys:shminfo_shmmni 100
    
  • Shared disks (also known as /dev/did/rdsk devices) are known. Listing 1 is an example of how to identify a shared disk from the global zone of any cluster node.

    phyhost1# cldev status
    
    === Cluster DID Devices ===
    
    Device Instance	Node		Status
    
    /dev/did/rdsk/d1	phyhost1	Ok
    
    /dev/did/rdsk/d10	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d14	phyhost		Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d15	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d16	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d17	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost3	Ok
    
    /dev/did/rdsk/d18	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d19	phyhost2	Ok
    
    /dev/did/rdsk/d2	phyhost1	Ok
    
    /dev/did/rdsk/d20	phyhost2	Ok
    
    /dev/did/rdsk/d21	phyhost3	Ok
    
    /dev/did/rdsk/d22	phyhost3	Ok
    
    /dev/did/rdsk/d23	phyhost4	Ok
    
    /dev/did/rdsk/d24	phyhost4	Ok
    
    /dev/did/rdsk/d6	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d7	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d8	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    /dev/did/rdsk/d9	phyhost1	Ok
    			phyhost2	Ok
    			phyhost3	Ok
    			phyhost4	Ok
    
    Listing 1. Identifying a Shared Disk

    The output shows that the disk is shared by phyhost1, phyhost2, phyhost3, and phyhost4 as device d10. Devices d14, d15, d16, d17, d18, d6, d7, d8, and d9 are also shared.

  • The following shared disks will be used for the Oracle Automatic Storage Management disk group to store Oracle Cluster Registry and voting files:

    /dev/did/rdsk/d6s6
    /dev/did/rdsk/d7s6
    ev/did/rdsk/d8s6
    

    In this example, slice 6 is 6 GB. Please check the Oracle Grid Infrastructure Installation Guide for disk size requirements.

  • The following shared disks will be used for the Oracle Automatic Storage Management disk group to store data files:

    /dev/did/rdsk/d9s6
    /dev/did/rdsk/d10s6
    /dev/did/rdsk/d14s6
    /dev/did/rdsk/d15s6
    

    In this example, slice 6 is 102 GB.

  • Oracle virtual IP (VIP) and Single Client Access Name (SCAN) IP requirements have been set up, for example:

    • vzhost1d, IP address 10.134.35.99, is used for SCAN IP.
    • vzhost1e, IP address 10.134.35.100, is used as VIP for vzhost1c.
    • vzhost2e, IP address 10.134.35.101, is used as VIP for vzhost2c.
    • vzhost3e, IP address 10.134.35.102, is used as VIP for vzhost3c.
    • vzhost4e, IP address 10.134.35.103, is used as VIP for vzhost4c.
       
  • There is an IPMP group for the public network with one active and one standby interface. Here is an example of the /etc/hostname.e1000g0 and /etc/hostname.e1000g1 settings for an IPMP group called SC_ipmp0 from the global zone:

    cat /etc/hostname.e1000g0
    phyhost1 netmask + broadcast + group sc_ipmp0 up
    cat /etc/hostname.e1000g1
    group sc_ipmp0 standby up
    

Create the Zone Cluster Using the cfg File

Perform the following steps to create an Oracle Solaris Zone cluster:

  1. Create a cfg file with the content shown in Listing 2.

    cat /var/tmp/zone.cfg
    
    create 
    set zonepath=/export/zones/z11gR2A
    add node
    set physical-host=phyhost1
    set hostname=vzhost1c
    add net
    set address=10.134.35.56
    set physical=e1000g0
    end
    end
    add sysid
    set name_service="NIS{domain_name=solaris.us.oracle.com}"
    set root_password=passwd
    end
    add node
    set physical-host=phyhost2
    set hostname=vzhost2c
    add net
    set address=10.134.35.59
    set physical=e1000g0
    end
    end
    add node
    set physical-host=phyhost3
    set hostname=vzhost3c
    add net
    set address=10.134.35.62
    set physical=e1000g0
    end
    end
    add node
    set physical-host=phyhost4
    set hostname=vzhost4c
    add net
    set address=10.134.35.65
    set physical=e1000g0
    end
    end
    add net
    set address=vzhost1d
    end
    add net
    set address=vzhost2d
    end
    add net
    set address=vzhost3d
    end
    add net
    set address=vzhost4d
    end
    add net
    set address=vzhost1e
    end
    add net
    set address=vzhost2e
    end
    add net
    set address=vzhost3e
    end
    add net
    set address=vzhost4e
    end
    set limitpriv ="default,proc_priocntl,proc_clock_highres,sys_time"
    add dedicated-cpu
    set ncpus=16
    end
    add capped-memory
    set physical=12g
    set swap=12g
    set locked=12g
    end
    add device
    set match="/dev/did/rdsk/d6s6"
    end
    add device
    set match="/dev/did/rdsk/d7s6"
    end
    add device
    set match="/dev/did/rdsk/d8s6"
    end
    add device
    set match="/dev/did/rdsk/d9s6"
    end
    add device
    set match="/dev/did/rdsk/d10s6"
    end
    add device
    set match="/dev/did/rdsk/d14s6"
    end
    add device
    set match="/dev/did/rdsk/d15s6"
    end
    
    Listing 2. Creating the cfg File
  2. Run the commands shown in Listing 3 from one node:

    # clzonecluster configure -f /var/tmp/zone.cfg z11gr2A 
    # clzonecluster install z11gr2A
    
    # clzonecluster status
    
    === Zone Clusters ===
    
    --- Zone Cluster Status ---
    
    Name      Node Name    Zone HostName   Status   Zone Status
    ----      ---------    -------------   ------   -----------
    z11gr2A   phyhost1     vzhost1c       Offline   Installed
              phyhost2     vzhost2c       Offline   Installed
              phyhost3     vzhost3c       Offline   Installed
              phyhost4     vzhost4c       Offline   Installed
    
    # clzc boot z11gr2A
    
    # clzc status
    
    === Zone Clusters ===
    
    --- Zone Cluster Status ---
    
    Name      Node Name     Zone HostName   Status   Zone Status
    ----      ---------     -------------   ------   -----------
    z11gr2A   phyhost1      vzhost1c        Online   Running
              phyhost2      vzhost2c        Online   Running
              phyhost3      vzhost3c        Online   Running
              phyhost4      vzhost4c        Online   Running
    
    Listing 3. Creating the Oracle Solaris Zone Cluster

Create the Oracle RAC Framework for the Zone Cluster

Perform the following steps to create the Oracle RAC framework:

  1. As root, execute clsetup from one global zone cluster node, as shown in Listing 4.

    # /usr/cluster/bin/clsetup
      *** Main Menu ***
    
        Please select from one of the following options:
    
            1) Quorum
            2) Resource groups
            3) Data Services
            4) Cluster interconnect
            5) Device groups and volumes
            6) Private hostnames
            7) New nodes
            8) Other cluster tasks
    
            ?) Help with menu options
            q) Quit
    
        Option:  3
    
      *** Data Services Menu ***
    
        Please select from one of the following options:
    
          * 1) Apache Web Server
          * 2) Oracle
          * 3) NFS
          * 4) Oracle Real Application Clusters
          * 5) SAP Web Application Server
          * 6) Highly Available Storage
          * 7) Logical Hostname
          * 8) Shared Address
    
          * ?) Help
          * q) Return to the Main Menu
    
        Option:  4
    
      *** Oracle Solaris Cluster Support for Oracle RAC ***
    
        Oracle Solaris Cluster provides a support layer for running Oracle 
        Real Application Clusters (RAC) database instances. This option allows
        you to create the Oracle RAC framework resource group, storage resources, 
        database resources and administer them, for managing the Oracle 
        Solaris Cluster support for Oracle RAC.
    
        After the Oracle RAC framework resource group has been created, you can use 
        the Oracle Solaris Cluster system administration tools to administer a
        Oracle RAC framework resource group that is configured on a global cluster. 
        To administer a Oracle RAC framework resource group that is configured on a 
        zone cluster, instead use the appropriate Oracle Solaris Cluster command.
    
    Listing 4. Executing clsetup
  2. Enter yes (the default) to continue and then answer subsequent questions, as shown in Listing 5.

        Is it okay to continue (yes/no) [yes]?  
        Please select from one of the following options:
    
            1)  Oracle RAC Create Configuration 
            2)  Oracle RAC Ongoing Administration 
    
            q)  Return to the Data Services Menu 
    
        Option:  1
    
       >>> Select Oracle Real Application Clusters Location <<<
    
        Oracle Real Application Clusters Location:
    
            1) Global Cluster
            2) Zone Cluster
    
        Option [2]:  2
    
       >>> Select Zone Cluster <<<
    
        From the list of zone clusters, select the zone cluster where you would like to
        configure Oracle Real Application Clusters.
    
            1) z11gr2A
    
            ?) Help
            d) Done
    
        Selected:  [z11gr2A]
    
       >>> Select Oracle Real Application Clusters Components to Configure <<<
    
        Select the component of Oracle Real Application Clusters that you are configuring:
    
            1) RAC Framework Resource Group
            2) Storage Resources for Oracle Files
            3) Oracle Clusterware Framework Resource
            4) Oracle Automatic Storage Management (ASM)
            5) Resources for Oracle Real Application Clusters Database Instances
    
        Option [1]: 1 
    
       >>> Verify Prerequisites <<<
    
    
        This wizard guides you through the creation and configuration of the Real Application
        Clusters (RAC) framework resource group.
        
        Before you use this wizard, ensure that the following prerequisites are met:
    
    
        * All pre-installation tasks for Oracle Real Application Clusters are completed.
    
        * The Oracle Solaris Cluster nodes are prepared.
    
        * The data services packages are installed.
    
        * All storage management software that you intend to use is installed and configured
        on all nodes where Oracle Real Application Clusters is to run.
    
    
    
        Press RETURN to continue 
    
       >>> Select Nodes <<<
    
        Specify, in order of preference, a list of names of nodes where Oracle Real
        Application Clusters is to run. If you do not explicitly specify a list, the list
        defaults to all nodes in an arbitrary order.
    
    
        The following nodes are available on the zone cluster z11skgxn:
    
            1) vzhost1c
            2) vzhost2c
            3) vzhost3c
            4) vzhost4c
    
            r) Refresh and Clear All
            a) All
            ?) Help
            d) Done
    
        Selected:  [vzhost1c, vzhost2c, vzhost3c, vzhost4c]
        Options:  d
    
       >>> Select Clusterware Support <<<
    
    
        Select the vendor clusterware support that you would like to use.
    
            1) Native
            2) UDLM based
    
        Option [1]: 1 
    
       >>> Review Oracle Solaris Cluster Objects <<<
    
    
        The following Oracle Solaris Cluster objects will be created.
    
        Select the value you are changing:
    
              Property Name               Current Setting
              =============               ===============
            1) Resource Group Name        rac-framework-rg
            2) RAC Framework Resource N...rac-framework-rs
    
            d) Done
            ?) Help
    
        Option:  d
    
       >>> Review Configuration of RAC Framework Resource Group <<<
    
    
        The following Oracle Solaris Cluster configuration will be created.
    
        To view the details for an option, select the option.
    
              Name                         Value
              ====                         =====
            1) Resource Group Name        rac-framework-rg
            2) RAC Framework Resource N...rac-framework-rs
    
            c) Create Configuration
            ?) Help
    
        Option:  c
    
    Listing 5. Answering Subsequent Questions
  3. From one global zone cluster node, verify the Oracle RAC framework resources, as shown in Listing 6.

    # clrs status -Z z11gr2A 
    
    === Cluster Resources ===
    
    Resource Name         Node Name      State      Status Message
    -------------         ---------      -----      --------------
    rac-framework-rs      vzhost1c       Online     Online
                          vzhost2c       Online     Online
                          vzhost3c       Online     Online
                          vzhost4c       Online     Online
    
    Listing 6. Verifying the Resources

Set Up the Root Environment in Local Zone Cluster z11gr2A

From each global zone cluster node (phyhost1, phyhost2, phyhost3, and phyhost4), do the following.

  1. Log in to the local zone node and execute the following command as root:

    # /usr/sbin/zlogin z11gr2A
    [Connected to zone 'z11gr2A' pts/2]
    Last login: Thu Aug 25 17:30:14 on pts/2
    Oracle Corporation      SunOS 5.10      Generic Patch   January 2005
    
  2. (Optional) Change the root shell to bash:

    # passwd -e
    Old shell: /sbin/sh
    New shell: bash
    passwd: password information changed for root
    
  3. Include these paths in .bash_profile:

    /u01/grid/product/11.2.0.2/bin
    /usr/cluster/bin
    

Create a User and Group for the Oracle Software

To create a user and group for the Oracle software, do the following from each node.

  1. As root, execute the following commands:

    # groupadd -g 300 oinstall   
    # groupadd -g 301 dba
    # useradd -g 300 -G 301 -u 302 -d /u01/ora_home -s /usr/bin/bash ouser  
    # mkdir -p /u01/ora_home
    # chown ouser:oinstall /u01/ora_home
    # mkdir /u01/oracle
    # chown ouser:oinstall /u01/oracle
    # mkdir /u01/grid
    # chown ouser:oinstall /u01/grid
    # mkdir /u01/oraInventory
    # chown ouser:oinstall /u01/oraInventory
    
  2. Create a password for the software owner, ouser:

    # passwd ouser
    New Password:
    Re-enter new Password:
    passwd: password successfully changed for ouser
    bash-3.00#
    
  3. For the Oracle software owner environment, as the software owner ouser, set up SSH from each node:

    $ mkdir .ssh
    $ chmod 700 .ssh
    $ cd .ssh
    $ ssh-keygen -t rsa
    Generating public/private rsa key pair.
    Enter file in which to save the key (/u01/ora_home/.ssh/id_rsa):
    Enter passphrase (empty for no passphrase): 
    Enter same passphrase again:
    Your identification has been saved in /u01/ora_home/.ssh/id_rsa.
    Your public key has been saved in /u01/ora_home/.ssh/id_rsa.pub.
    The key fingerprint is:
    e6:63:c9:71:fe:d1:8f:71:77:70:97:25:2a:ee:a9:33 local1@vzhost1c
    $
    $ pwd
    /u01/ora_home/.ssh
    
  4. From the first node, vzhost1c, do the following:

    $ cat id_rsa.pub >> authorized_keys
    $ chmod 600 authorized_keys
    $ scp authorized_keys vzhost2c:/u01/ora_home/.ssh
    
  5. From the second node, vzhost2c, do the following:

    $ cd /u01/ora_home/.ssh
    $ cat id_rsa.pub >> authorized_keys
    $ scp authorized_keys vzhost3c:/u01/ora_home/.ssh
    
  6. From the third node, vzhost3c,do the following:

    $ cd /u01/ora_home/.ssh
    $ cat id_rsa.pub >> authorized_keys
    $ scp authorized_keys vzhost4c:/u01/ora_home/.ssh
    
  7. From the fourth node, vzhost4c, do the following:

    $ cd /u01/ora_home/.ssh
    $ cat id_rsa.pub >> authorized_keys
    $ scp authorized_keys vzhost1c:/u01/ora_home/.ssh
    
  8. From the first node, vzhost1c, do the following:

    $ cd /u01/ora_home/.ssh
    $ scp authorized_keys vzhost2c:/u01/ora_home/.ssh
    $ scp authorized_keys vzhost3c:/u01/ora_home/.ssh
    
  9. From each node, test the ssh setup:

    $ ssh vzhost1c date
    $ ssh vzhost2c date
    $ ssh vzhost3c date
    $ ssh vzhost4c date
    
  10. In each local zone cluster node, as root, set up the Oracle Automatic Storage Management candidate disk:

    # for i in 6 7 8 9 10 14 15 
    > do
    > chown ouser:oinstall /dev/did/rdsk/d${i}s6
    > chmod 660 /dev/did/rdsk/d${i}s6
    > done
    
  11. In the local zone cluster, as the software owner, do the following from one node:

    $ for i in 6 7 8 9 10 14 15 
    > do
    > dd if=/dev/zero of=/dev/did/rdsk/d${i}s6 bs=1024k count=200
    > done
    

Install Oracle Grid Infrastructure 11.2.0.2 in the Oracle Solaris Zone Cluster Nodes

On each node, perform the following procedure.

  1. As the software owner, execute the following:

    $ bash
    $ export DISPLAY=<hostname>:<n>
    $ cd <PATH to 11.2.0.2 based software image>/grid/
    $ ./runInstaller
    
  2. Provide the following input to Oracle Universal Installer:

    1. On the Select Installation Option page, select Install Oracle Grid Infrastructure Software Only.
    2. On the Select Product Languages page, select the appropriate language.
    3. On the Privileged Operating System Groups page, select the following, as shown in Figure 1:

      • For Oracle ASM DBA (OSDBA for ASM) Group, select oinstall.
      • For Oracle ASM Operator (OSOPER for ASM) Group (Optional), select a group.
      • For Oracle ASM Administrator (OSASM) Group, select oinstall.

        Figure 1

        Figure 1. Privileged Operating System Groups Page

    4. On the Specify Installation Location page, do the following:
      • For Oracle Base, select /u01/oracle.
      • For Software Location, select /u01/grid/product/11.2.0.2.
    5. On the Create Inventory page, select /u01/oraInventory for Inventory Directory. The oraInventory Group Name should be oinstall.
    6. On the Perform Prerequisite Checks page, select Ignore All.
    7. On the Summary page, check all the settings you selected to ensure they are appropriate, and then click Install to install the product.

      The Execute Configuration Scripts dialog box asks you to execute the /u01/oraInventory/orainstRoot.sh and /u01/grid/product/11.2.0.2/root.sh scripts as root, as shown in Figure 2.

      Figure 2

      Figure 2. Execute Configuration Scripts Dialog Box

  3. Open a terminal window and execute the scripts, as shown in Listing 7.

    # /u01/oraInventory/orainstRoot.sh
    Changing permissions of /u01/oraInventory.
    Adding read,write permissions for group.
    Removing read,write,execute permissions for world.
    
    Changing groupname of /u01/oraInventory to oinstall.
    The execution of the script is complete.
    
    # /u01/grid/product/11.2.0.2/root.sh
    Running Oracle 11g root script...
    
    The following environment variables are set as:
        ORACLE_OWNER= ouser
        ORACLE_HOME=  /u01/grid/product/11.2.0.2
    
    Enter the full pathname of the local bin directory: [/usr/local/bin]: /opt/local/bin
       Copying dbhome to /opt/local/bin ...
       Copying oraenv to /opt/local/bin ...
       Copying coraenv to /opt/local/bin ...
    
    
    Creating /var/opt/oracle/oratab file...
    Entries will be added to the /var/opt/oracle/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    
    To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user:
    /u01/grid/product/11.2.0.2/perl/bin/perl 
    -I/u01/grid/product/11.2.0.2/perl/lib 
    -I/u01/grid/product/11.2.0.2/crs/install 
    /u01/grid/product/11.2.0.2/crs/install/roothas.pl
    
    
    To configure Grid Infrastructure for a Cluster execute the following command:
    /u01/grid/product/11.2.0.2/crs/config/config.sh
    This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent 
    operation, and the parameters can be passed through the response file that is 
    available in the installation media.
    
    Listing 7. Executing the Scripts

Replace OPatch Version 11.2.0.1.1 with Version 11.2.0.1.6

On each node, perform the following procedure to update OPatch to the latest version.

  1. As the software owner, back up the OPatch version in GRID_Home, for example:

    $ cd /u01/grid/product/11.2.0.2/ 
    $ mv OPatch OPatch.bak
    
  2. As the software owner, copy p6880880_112000_SOLARIS64.zip to GRID_HOME:

    $ cd /u01/grid/product/11.2.0.2
    $ unzip p6880880_112000_SOLARIS64.zip
    
  3. Verify the OPatch version:

    $ export ORACLE_HOME=/u01/grid/product/11.2.0.2
    $ /u01/grid/product/11.2.0.2/OPatch/opatch -version
    
    Invoking OPatch 11.2.0.1.6
    
    Oracle Interim Patch Installer version 11.2.0.1.6
    Copyright (c) 2011, Oracle Corporation.  All rights reserved.
    

Apply Oracle Grid Infrastructure 11.2.0.2.3 GI PSU patch to GRID_HOME Using OPatch

Apply patch 11.2.0.2 GIB3 to each node by performing the following steps.

  1. As the software owner, copy the patch to the node, for example:

    $ cd /var/tmp
    $ mkdir gib3
    $ cp p12419353_112020_SOLARIS64.zip /var/tmp/gib3
    $ cd /var/tmp/gib3
    $ unzip p12419353_112020_SOLARIS64.zip
    
  2. As the software owner, apply the patch, as shown in Listing 8:

    $ /u01/grid/product/11.2.0.2/OPatch/opatch napply -oh /u01/grid/product/11.2.0.2 -local /var/tmp/gib3
    
    Invoking OPatch 11.2.0.1.6
    
    Oracle Interim Patch Installer version 11.2.0.1.6
    Copyright (c) 2011, Oracle Corporation.  All rights reserved.
    
    UTIL session
    
    Oracle Home       : /u01/grid/product/11.2.0.2
    Central Inventory : /u01/oraInventory
       from           : /var/opt/oracle/oraInst.loc
    OPatch version    : 11.2.0.1.6
    OUI version       : 11.2.0.2.0
    Log file location : /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-08-
    12_17-50-22PM.log
    
    Verifying environment and performing prerequisite checks...
    
    Patch continues with these patches:   12419331  12419353  
    
    Do you want to proceed? [y|n]
    
    Listing 8. Applying the Patch
  3. Enter y to continue:

    Provide your email address to be informed of security issues, install and
    initiate Oracle Configuration Manager. Easier for you if you use your My
    Oracle Support Email address/User Name.
    Visit http://www.oracle.com/support/policies.html for details.
    Email address/User Name: 
    
  4. Enter your user e-mail address.
    Provide your My Oracle Support password to receive security updates via your My 
    Oracle Support account.
    Password (optional):  
    
  5. No password is required, so press Enter.

    Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
    (Oracle Home = '/u01/grid/product/11.2.0.2')
    
    Is the local system ready for patching? [y|n]
    
  6. Enter y to continue, as shown in Listing 9.

    Backing up files...
    Applying interim patch '12419331' to OH '/u01/grid/product/11.2.0.2'
    ApplySession: Optional component(s) [ oracle.sysman.console.db, 11.2.0.2.0 ] , 
    [ oracle.sysman.oms.core, 10.2.0.4.3 ] , [ oracle.rdbms.dv, 11.2.0.2.0 ] , 
    [ oracle.sysman.plugin.db.main.repository, 11.2.0.2.0 ]  not present in the Oracle Home 
    or a higher version is found.
    
    Patching component oracle.rdbms.rsf, 11.2.0.2.0...
    
    Patching component oracle.rdbms, 11.2.0.2.0...
    Copying file to "/u01/grid/product/11.2.0.2/psu/11.2.0.2.3/catpsu.sql"
    Copying file to "/u01/grid/product/11.2.0.2/psu/11.2.0.2.3/catpsu_rollback.sql"
    Copying file to "/u01/grid/product/11.2.0.2/cpu/scripts/patch_8837510.sql"
    Copying file to "/u01/grid/product/11.2.0.2/cpu/scripts/emdb_recomp_invalids.sql"
    
    Patching component oracle.ldap.rsf, 11.2.0.2.0...
    
    Patching component oracle.rdbms.dbscripts, 11.2.0.2.0...
    
    Patching component oracle.rdbms.rman, 11.2.0.2.0...
    Applying interim patch '12419353' to OH '/u01/grid/product/11.2.0.2'
    
    Patching component oracle.crs, 11.2.0.2.0...
    Copying file to "/u01/grid/product/11.2.0.2/crs/install/crsconfig_lib.pm"
    Copying file to "/u01/grid/product/11.2.0.2/crs/install/crspatch.pm"
    Copying file to "/u01/grid/product/11.2.0.2/crs/install/s_crsconfig_lib.pm"
    
    Patching component oracle.usm, 11.2.0.2.0...
    Patches 12419331,12419353 successfully applied.
    Log file location: /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-08-1
    2_17-50-22PM.log
    
    OPatch succeeded.
    
    Listing 9. Finishing Applying the Patch
  7. As the software owner, verify that the patch was installed correctly, as shown in Listing 10.

    $ /u01/grid/product/11.2.0.2/OPatch/opatch lsinventory -oh /u01/grid/product/11.2.0.2 
    
    Invoking OPatch 11.2.0.1.6
    
    Oracle Interim Patch Installer version 11.2.0.1.6
    Copyright (c) 2011, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/grid/product/11.2.0.2
    Central Inventory : /u01/oraInventory
       from           : /var/opt/oracle/oraInst.loc
    OPatch version    : 11.2.0.1.6
    OUI version       : 11.2.0.2.0
    Log file location : /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-08-12_21-32-53PM.log
    
    Lsinventory Output file location : 
    /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/lsinv/lsinventory2011-08-12_21-32-53PM.txt
    
    --------------------------------------------------------------------------------
    Installed Top-level Products (1): 
    
    Oracle Grid Infrastructure                                           11.2.0.2.0
    There are 1 products installed in this Oracle Home.
    
    
    Interim patches (2) :
    
    Patch  12419353     : applied on Fri Aug 12 18:11:35 PDT 2011
    Unique Patch ID:  13850636
       Created on 18 Jul 2011, 05:19:12 hrs UTC
       Bugs fixed:
         12419353, 10157506, 10178670, 10425672, 12311357, 9959110, 10272615
    	     10314123, 10014392, 10089120, 10057296, 9864003, 11775080, 9916145
         10044622, 12399977, 12421404, 12340700, 10056713, 10637741, 9939306
    	     9902536, 10007185, 10376847, 10038791, 11741224, 11655840, 10048487
         10322157, 10260251, 10052721, 10028235, 10027079, 10357258, 10045436
    	     10231906, 10622973, 9891341, 10072474, 10036834, 10029900, 9974223
         10016083, 9918485, 11781515, 10040647, 10069541, 10029119, 10233159
    	     12332919, 9812956, 10036193, 10015210, 12340501, 10621175, 11877079
         8906163, 10111010, 10115514, 10104377, 10057680, 10280665, 10078086
    	     9944948, 10146768, 10052529, 10011084, 10012319, 10073075, 10233811
         10299006, 10248739, 10236074, 10128191, 11071429, 10019726, 9975837
    	     10253630, 9949676, 11936945, 10637483, 10157622, 11698552, 10385838
         10053985, 10425674, 9812970, 11828633, 11899801, 10083789, 9876201
    	     10073372, 9963327, 11077756, 10375649, 9336825, 11682409, 10062301
         10018215, 10105195, 10419987, 10071992, 10634513, 9926027, 10103954
    	     10028343, 11866171, 10065216, 9907089, 9897335, 10190153, 11744313
         10175855, 10284828, 10028637, 10361177, 9979706, 10324594, 10015460
    	     9971646, 11782423, 11654726, 9978765, 10398810, 11904778, 10397652
         9915329, 10107380, 10110969, 10305361, 10331452, 10083009, 10631693
    	     10008467, 10048027, 10040109, 9944978, 10033106, 9978195, 11840629
         10042143, 10284693, 10638381, 9679401, 11663339, 10075643, 10205290
    	     10124517, 11069614, 9593552, 10168006, 12677816, 11807012, 11846686
         9867867, 10228079, 10015603, 10241696, 9942881, 10252497, 10283058
    	     10157625, 10283167, 9906432, 10216878, 10045316, 10425675, 10061534
         11789566, 10283549, 10311856, 10150020, 12421420, 12378675, 10113899
    	     10069698, 9861790, 10087118, 10056808, 10146744, 10326548, 10019796
         9975343, 9936659, 10244210, 10029794, 10266447, 10193581, 12318560
    	     11804097, 10070563, 10268642, 10283596
    
    Patch  12419331     : applied on Fri Aug 12 18:06:44 PDT 2011
    Unique Patch ID:  13850636
       Created on 12 Jul 2011, 07:01:28 hrs PST8PDT
       Bugs fixed:
         10151017, 10158965, 11724916, 10190642, 12586486, 12586487, 10129643
    	     12586488, 12586489, 10018789, 9744252, 10248523, 9956713, 10356513
         9715581, 9770451, 10378005, 10170431, 10425676, 10222719, 10126094
    	     9591812, 10127360, 10132870, 10094201, 9443361, 10193846, 11664046
         11069199, 10324294, 10245086, 12586490, 10205230, 12586491, 10052141
    	     12586492, 12586493, 12586494, 10142788, 11818335, 11830776, 12586495
         9905049, 11830777, 12586496, 11830778, 6892311, 10040921, 10077191
    	     10358019, 12431716, 10219576, 10258337, 11707699, 10264680, 10209232
         11651810, 10102506, 11067567, 9881076, 10278372, 10040531, 10621169
    	     10155605, 10082277, 10356782, 10218814, 9078442, 9788588, 10157249
         9735237, 10317487, 12326246, 11707302, 10310299, 10636231, 10230571
    	     11065646, 12419321, 10368698, 10079168, 10013431, 10228151, 10233732
         10324526, 8223165, 10238786, 10217802, 10061015, 9953542, 9572787
    	     10052956, 10080579, 11699057, 12620422, 10332111, 10227288, 10329146
         10332589, 10110863, 10073683, 9869401, 10019218, 10229719, 11664719
    	     9539440, 10373381, 9735282, 9748749, 11724984, 10022980, 10411618
         11800854, 12419331, 11674485, 10187168, 6523037, 10648873, 9724970
    	     10053725, 10084145, 10367188, 11800170, 11695285, 10157402, 9651350, 10299224
    
    
    --------------------------------------------------------------------------------
    
    OPatch succeeded.
    
    Listing 10. Verifying the Patch Installation

Apply Patch 10055663 to GRID_HOME Using OPatch

Apply patch 10055663 to each node by performing the following steps.

  1. Download the patch and place it in /var/tmp.
     
  2. As the software owner, uncompress the patch, for example:

    $ cd /var/tmp
    $ unzip p10055663_112020_Generic.zip
    
  3. Apply the patch using OPatch, as shown in Listing 11.

    $ export ORACLE_HOME=/u01/grid/product/11.2.0.2 
    $ cd /var/tmp/10055663
    $ /u01/grid/product/11.2.0.2/OPatch/opatch apply
    
    Invoking OPatch 11.2.0.1.6
    
    Oracle Interim Patch Installer version 11.2.0.1.6
    Copyright (c) 2011, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/grid/product/11.2.0.2
    Central Inventory : /u01/oraInventory
       from           : /var/opt/oracle/oraInst.loc
    OPatch version    : 11.2.0.1.6
    OUI version       : 11.2.0.2.0
    Log file location : /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-08-10_10-46-04AM.log
    
    Applying interim patch '10055663' to OH '/u01/grid/product/11.2.0.2'
    Verifying environment and performing prerequisite checks...
    Do you want to proceed? [y|n]
    
    Listing 11. Applying the Patch
  4. Enter Y to continue.

    User Responded with: Y
    .....
    Patching component oracle.rdbms.install.plugins, 11.2.0.2.0...
    Patch 10055663 successfully applied
    Log file location: /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-08-12_15-24-31PM.log
    
    OPatch succeeded.
    
  5. As the software owner, verify that the patch was installed correctly, as shown in Listing 12.

    $ /u01/grid/product/11.2.0.2/OPatch/opatch lsinventory -oh /u01/grid/product/11.2.0.2
    
    Invoking OPatch 11.2.0.1.6
    
    Oracle Interim Patch Installer version 11.2.0.1.6
    Copyright (c) 2011, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/grid/product/11.2.0.2
    Central Inventory : /u01/oraInventory
       from           : /var/opt/oracle/oraInst.loc
    OPatch version    : 11.2.0.1.6
    OUI version       : 11.2.0.2.0
    Log file location : /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-08-10_10-48-42AM.log
    
    Lsinventory Output file location : 
    /u01/grid/product/11.2.0.2/cfgtoollogs/opatch/lsinv/lsinventory2011-08-10_10-48-42AM.txt
    
    --------------------------------------------------------------------------------
    Installed Top-level Products (1): 
    
    Oracle Grid Infrastructure                                           11.2.0.2.0
    There are 1 products installed in this Oracle Home.
    
    
    Interim patches (3) :
    
    Patch  10055663     : applied on Wed Aug 10 10:47:16 PDT 2011
    Unique Patch ID:  13606606
       Created on 22 Mar 2011, 01:03:34 hrs PST8PDT
       Bugs fixed:
         10055663
    
    Patch  12419353     : applied on Wed Aug 10 10:32:47 PDT 2011
    Unique Patch ID:  13850636
       Created on 18 Jul 2011, 05:19:12 hrs UTC
       Bugs fixed:
         12419353, 10157506, 10178670, 10425672, 12311357, 9959110, 10272615
    	     10314123, 10014392, 10089120, 10057296, 9864003, 11775080, 9916145
         10044622, 12399977, 12421404, 12340700, 10056713, 10637741, 9939306
    	     9902536, 10007185, 10376847, 10038791, 11741224, 11655840, 10048487
         10322157, 10260251, 10052721, 10028235, 10027079, 10357258, 10045436
    	     10231906, 10622973, 9891341, 10072474, 10036834, 10029900, 9974223
         10016083, 9918485, 11781515, 10040647, 10069541, 10029119, 10233159
    	     12332919, 9812956, 10036193, 10015210, 12340501, 10621175, 11877079
         8906163, 10111010, 10115514, 10104377, 10057680, 10280665, 10078086
    	     9944948, 10146768, 10052529, 10011084, 10012319, 10073075, 10233811
         10299006, 10248739, 10236074, 10128191, 11071429, 10019726, 9975837
    	     10253630, 9949676, 11936945, 10637483, 10157622, 11698552, 10385838
         10053985, 10425674, 9812970, 11828633, 11899801, 10083789, 9876201
    	     10073372, 9963327, 11077756, 10375649, 9336825, 11682409, 10062301
         10018215, 10105195, 10419987, 10071992, 10634513, 9926027, 10103954
    	     10028343, 11866171, 10065216, 9907089, 9897335, 10190153, 11744313
         10175855, 10284828, 10028637, 10361177, 9979706, 10324594, 10015460
    	     9971646, 11782423, 11654726, 9978765, 10398810, 11904778, 10397652
         9915329, 10107380, 10110969, 10305361, 10331452, 10083009, 10631693
    	     10008467, 10048027, 10040109, 9944978, 10033106, 9978195, 11840629
         10042143, 10284693, 10638381, 9679401, 11663339, 10075643, 10205290
    	     10124517, 11069614, 9593552, 10168006, 12677816, 11807012, 11846686
         9867867, 10228079, 10015603, 10241696, 9942881, 10252497, 10283058
    	     10157625, 10283167, 9906432, 10216878, 10045316, 10425675, 10061534
         11789566, 10283549, 10311856, 10150020, 12421420, 12378675, 10113899
    	     10069698, 9861790, 10087118, 10056808, 10146744, 10326548, 10019796
         9975343, 9936659, 10244210, 10029794, 10266447, 10193581, 12318560
    	     11804097, 10070563, 10268642, 10283596
    
    Patch  12419331     : applied on Wed Aug 10 10:27:15 PDT 2011
    Unique Patch ID:  13850636
       Created on 12 Jul 2011, 07:01:28 hrs PST8PDT
       Bugs fixed:
         10151017, 10158965, 11724916, 10190642, 12586486, 12586487, 10129643
    	     12586488, 12586489, 10018789, 9744252, 10248523, 9956713, 10356513
         9715581, 9770451, 10378005, 10170431, 10425676, 10222719, 10126094
    	     9591812, 10127360, 10132870, 10094201, 9443361, 10193846, 11664046
         11069199, 10324294, 10245086, 12586490, 10205230, 12586491, 10052141
    	     12586492, 12586493, 12586494, 10142788, 11818335, 11830776, 12586495
         9905049, 11830777, 12586496, 11830778, 6892311, 10040921, 10077191
    	     10358019, 12431716, 10219576, 10258337, 11707699, 10264680, 10209232
         11651810, 10102506, 11067567, 9881076, 10278372, 10040531, 10621169
    	     10155605, 10082277, 10356782, 10218814, 9078442, 9788588, 10157249
         9735237, 10317487, 12326246, 11707302, 10310299, 10636231, 10230571
    	     11065646, 12419321, 10368698, 10079168, 10013431, 10228151, 10233732
         10324526, 8223165, 10238786, 10217802, 10061015, 9953542, 9572787
    	     10052956, 10080579, 11699057, 12620422, 10332111, 10227288, 10329146
         10332589, 10110863, 10073683, 9869401, 10019218, 10229719, 11664719
    	     9539440, 10373381, 9735282, 9748749, 11724984, 10022980, 10411618
         11800854, 12419331, 11674485, 10187168, 6523037, 10648873, 9724970
    	     10053725, 10084145, 10367188, 11800170, 11695285, 10157402, 9651350, 10299224
    
    --------------------------------------------------------------------------------
    
    
    OPatch succeeded.
    
    Listing 12. Verifying the Patch Installation

Configure Oracle Grid Infrastructure

To configure Oracle Grid Infrastructure, do the following from one node.

  1. Run the following command, providing values appropriate for your environment:

    $ export DISPLAY=<hostname>:<n>
    $ /u01/grid/product/11.2.0.2/crs/config/config.sh
    
  2. Provide the following input to Oracle Universal Installer:

    1. On the Select Configuration Option page, select the Configure Oracle Grid Infrastructure for a Cluster option.
    2. On the Grid Plug and Play Information page, do the following, as shown in Figure 3:
      • For Cluster Name, select vzhost-cluster.
      • For SCAN Name, select vzhost1d.
      • For SCAN Port, select 1521.

        Figure 3

        Figure 3. Grid Plug and Play Information Page

    3. On the Cluster Node Information page, specify the following, as shown in Figure 4:

      Hostname       Virtual IP Name
      vzhost1c       vzhost1e
      vzhost2c       vzhost2e
      vzhost3c       vzhost3e
      vzhost4c       vzhost4e
      
      Figure 4

      Figure 4. Cluster Node Information Page

    4. On the Specify Network Interface Usage page, specify the following, as shown in Figure 5:

      Interface Name   Subnet         Interface Type
      e1000g0          10.134.35.0    Public
      clprivnet0       172.16.7.0     Private
      
      Figure 5

      Figure 5. Specify Network Interface Usage Page

    5. On the Storage Option Information page, select the Oracle Automatic Storage Management (Oracle ASM) option.
       
    6. On the Create ASM Disk Group page, create a disk group by doing the following, as shown in Figure 6 through Figure 8:

      • For Disk Group Name, specify crsdg.
      • Click Change Discovery Path.
      • In the Change Discovery Path dialog box, specify the discovery path as /dev/did/rdsk/d*s6, click OK, and ensure the user has the correct permissions.
      • Select /dev/did/rdsk/d6s6, /dev/did/rdsk/d7s6, and /dev/did/rdsk/d8s6 for the crsdg disk group.

        Figure 6

        Figure 6. Specifying the Disk Group Name

        Figure 7

        Figure 7. Specifying the Discovery Path

        Figure 8

        Figure 8. Created Disk Group

    7. On the Specify ASM Password page, choose whether to use the same password for the SYS and ASMSNMP accounts, and then specify the password, as shown in Figure 9. If desired, you can choose to use different password for the accounts.

      Figure 9

      Figure 9. Specify ASM Password Page

    8. On the Perform Prerequisite Checks page, select Ignore All.
       
    9. On the Summary page, check all the settings you selected to ensure they are appropriate and click Install.

      The Execute Configuration Scripts dialog box asks you to execute the GRID_Home/root.sh script on each node as root, as shown in Figure 10.

      Figure 10

      Figure 10. Execute Configuration Scripts Dialog Box

  3. Open a terminal window and execute the script, as shown in Listing 13.

    # /u01/grid/product/11.2.0.2/root.sh
    
    Running Oracle 11g root script...
    
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /u01/grid/product/11.2.0.2
    
    Enter the full pathname of the local bin directory: [/usr/local/bin]: /opt/local/bin
    The contents of "dbhome" have not changed. No need to overwrite.
    The contents of "oraenv" have not changed. No need to overwrite.
    The contents of "coraenv" have not changed. No need to overwrite.
    
    Entries will be added to the /var/opt/oracle/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    Relinking oracle with rac_on option
    Using configuration parameter file: /u01/grid/product/11.2.0.2/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE 
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    OLR initialization - successful
      root wallet
      root wallet cert
      root cert export
      peer wallet
      profile reader wallet
      pa wallet
      peer wallet keys
      pa wallet keys
      peer cert request
      pa cert request
      peer cert
      pa cert
      peer root cert TP
      profile reader root cert TP
      pa root cert TP
      peer pa cert TP
      pa peer cert TP
      profile reader pa cert TP
      profile reader peer cert TP
      peer user cert
      pa user cert
    Adding daemon to inittab
    ACFS-9459: ADVM/ACFS is not supported on this OS version: 'Solaris Container: Solaris 10 update 9'
    ACFS-9201: Not Supported
    ACFS-9459: ADVM/ACFS is not supported on this OS version: 'Solaris Container: Solaris 10 update 9'
    CRS-2672: Attempting to start 'ora.mdnsd' on 'vzhost1c'
    CRS-2676: Start of 'ora.mdnsd' on 'vzhost1c' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'vzhost1c'
    CRS-2676: Start of 'ora.gpnpd' on 'vzhost1c' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'vzhost1c'
    CRS-2672: Attempting to start 'ora.gipcd' on 'vzhost1c'
    CRS-2676: Start of 'ora.cssdmonitor' on 'vzhost1c' succeeded
    CRS-2676: Start of 'ora.gipcd' on 'vzhost1c' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'vzhost1c'
    CRS-2672: Attempting to start 'ora.diskmon' on 'vzhost1c'
    CRS-2676: Start of 'ora.diskmon' on 'vzhost1c' succeeded
    CRS-2676: Start of 'ora.cssd' on 'vzhost1c' succeeded
    
    ASM created and started successfully.
    
    Disk Group crsdg created successfully.
    
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-4256: Updating the profile
    Successful addition of voting disk 3e08bbbbc8e94fa4bf78bfe416d5f4c9.
    Successful addition of voting disk 2a0642961e9c4f84bf0d8f87c47fc201.
    Successful addition of voting disk 994f063bcf5b4fb9bf328f34eecc95d9.
    Successfully replaced voting disk group with +crsdg.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ##  STATE    File Universal Id                File Name Disk group
    --  -----    -----------------                --------- ---------
     1. ONLINE   3e08bbbbc8e94fa4bf78bfe416d5f4c9 (/dev/did/rdsk/d6s6) [CRSDG]
     2. ONLINE   2a0642961e9c4f84bf0d8f87c47fc201 (/dev/did/rdsk/d7s6) [CRSDG]
     3. ONLINE   994f063bcf5b4fb9bf328f34eecc95d9 (/dev/did/rdsk/d8s6) [CRSDG]
    Located 3 voting disk(s).
    CRS-2672: Attempting to start 'ora.asm' on 'vzhost1c'
    CRS-2676: Start of 'ora.asm' on 'vzhost1c' succeeded
    CRS-2672: Attempting to start 'ora.CRSDG.dg' on 'vzhost1c'
    CRS-2676: Start of 'ora.CRSDG.dg' on 'vzhost1c' succeeded
    ACFS-9459: ADVM/ACFS is not supported on this OS version: 'Solaris Container: Solaris 10 update 9'
    ACFS-9201: Not Supported
    ACFS-9459: ADVM/ACFS is not supported on this OS version: 'Solaris Container: Solaris 10 update 9'
    ACFS-9201: Not Supported
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    
    Listing 13. Executing the Script
  4. Run root.sh from the remaining nodes, as shown in Listing 14.

    # /u01/grid/product/11.2.0.2/root.sh
    Running Oracle 11g root script...
    
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /u01/grid/product/11.2.0.2
    
    Enter the full pathname of the local bin directory: [/usr/local/bin]: /opt/local/bin
    The contents of "dbhome" have not changed. No need to overwrite.
    The contents of "oraenv" have not changed. No need to overwrite.
    The contents of "coraenv" have not changed. No need to overwrite.
    
    Entries will be added to the /var/opt/oracle/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    Relinking oracle with rac_on option
    Using configuration parameter file: /u01/grid/product/11.2.0.2/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE 
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    OLR initialization - successful
    Adding daemon to inittab
    ACFS-9459: ADVM/ACFS is not supported on this OS version: 'Solaris Container: Solaris 10 update 9'
    ACFS-9201: Not Supported
    ACFS-9459: ADVM/ACFS is not supported on this OS version: 'Solaris Container: Solaris 10 update 9'
    CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node unknown, 
    number unknown, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    
    Listing 14. Running root.sh
  5. After the root.sh script has been completed on the last node, proceed to the next step of the Oracle Universal Installer.

    Note: There will be an error: [INS-20802] Oracle Verification Utility failed. You can ignore this error by selecting OK, selecting Skip, selecting Next, and selecting Yes to the continue in spite of the [INS-32091] warning messages, as shown in Figure 11 through Figure 14.

    Figure 11

    Figure 11. Error Message

    Figure 12

    Figure 12. Click Skip Button

    Figure 13

    Figure 13. Click Yes to Continue

    Figure 14

    Figure 14. Successful Cluster Configuration

  6. From any node, check the Cluster Ready Services, as shown in Listing 15.

    # /u01/grid/product/11.2.0.2/bin/crsctl check cluster -all
    **************************************************************
    vzhost1c:
    CRS-4537: Cluster Ready Services is online
    CRS-4529: Cluster Synchronization Services is online
    CRS-4533: Event Manager is online
    **************************************************************
    vzhost2c:
    CRS-4537: Cluster Ready Services is online
    CRS-4529: Cluster Synchronization Services is online
    CRS-4533: Event Manager is online
    **************************************************************
    vzhost3c:
    CRS-4537: Cluster Ready Services is online
    CRS-4529: Cluster Synchronization Services is online
    CRS-4533: Event Manager is online
    **************************************************************
    vzhost4c:
    CRS-4537: Cluster Ready Services is online
    CRS-4529: Cluster Synchronization Services is online
    CRS-4533: Event Manager is online
    **************************************************************
    
    Listing 15. Checking the Cluster Ready Services
  7. Check the status of the resources, as shown in Listing 16.

    # /u01/grid/product/11.2.0.2/bin/crsctl status res -t
    --------------------------------------------------------------------------------
    NAME           TARGET  STATE        SERVER                 STATE_DETAILS       
    --------------------------------------------------------------------------------
    Local Resources
    --------------------------------------------------------------------------------
    ora.CRSDG.dg
                   ONLINE  ONLINE       vzhost1c                                   
                   ONLINE  ONLINE       vzhost2c                                   
                   ONLINE  ONLINE       vzhost3c                                   
                   ONLINE  ONLINE       vzhost4c                                   
    ora.LISTENER.lsnr
                   ONLINE  ONLINE       vzhost1c                                   
                   ONLINE  ONLINE       vzhost2c                                   
                   ONLINE  ONLINE       vzhost3c                                   
                   ONLINE  ONLINE       vzhost4c                                   
    ora.asm
                   ONLINE  ONLINE       vzhost1c               Started             
                   ONLINE  ONLINE       vzhost2c                                   
                   ONLINE  ONLINE       vzhost3c                                   
                   ONLINE  ONLINE       vzhost4c                                   
    ora.gsd
                   OFFLINE OFFLINE      vzhost1c                                   
                   OFFLINE OFFLINE      vzhost2c                                   
                   OFFLINE OFFLINE      vzhost3c                                   
                   OFFLINE OFFLINE      vzhost4c                                   
    ora.net1.network
                   ONLINE  ONLINE       vzhost1c                                   
                   ONLINE  ONLINE       vzhost2c                                   
                   ONLINE  ONLINE       vzhost3c                                   
                   ONLINE  ONLINE       vzhost4c                                   
    ora.ons
                   ONLINE  ONLINE       vzhost1c                                   
                   ONLINE  ONLINE       vzhost2c                                   
                   ONLINE  ONLINE       vzhost3c                                   
                   ONLINE  ONLINE       vzhost4c                                   
    --------------------------------------------------------------------------------
    Cluster Resources
    --------------------------------------------------------------------------------
    ora.LISTENER_SCAN1.lsnr
          1        ONLINE  ONLINE       vzhost1c                                   
    ora.cvu
          1        ONLINE  ONLINE       vzhost1c                                   
    ora.oc4j
          1        ONLINE  ONLINE       vzhost1c                                   
    ora.scan1.vip
          1        ONLINE  ONLINE       vzhost1c                                   
    ora.vzhost1e.vip
          1        ONLINE  ONLINE       vzhost1c                                   
    ora.vzhost2e.vip
          1        ONLINE  ONLINE       vzhost2c                                   
    ora.vzhost3e.vip
          1        ONLINE  ONLINE       vzhost3c                                   
    ora.vzhost4e.vip
          1        ONLINE  ONLINE       vzhost4c           
    
    Listing 16. Checking the Status of the Resources

Install and Patch Oracle Database

Note: Installing and creating the database in one step is optional. Create an Oracle Automatic Storage Management disk group for the database file if you are planning to create the database at this point.

  1. From one node, install Oracle Database 11.2.0.2, for example:

    $ export DISPLAY=<hostname>:<n>
    $ cd 10098816_112020_SOLARIS64/database
    $ ./runInstaller
    
  2. Provide input to Oracle Universal Installer:

    1. Provide the requested information on the Configure Security Updates page and the Download Software Updates page.
    2. On the Select Installation Option page, select the Install database software only option.
    3. On the Grid Installation Options page, select the Oracle Real Application Clusters database installation option.
    4. On the Select Product Languages page, select the appropriate language.
    5. On the Select Database Edition page, select the Enterprise Edition option.
    6. On the Specify Installation Location page, specify the following, as shown in Figure 15:
      • For Oracle Base, select /u01/oracle.
      • For Software Location, select /u01/oracle/product/11.2.0.2.

        Figure 15

        Figure 15. Specify Installation Location Page

    7. On the Privileged Operating System Groups page, specify the following, as shown in Figure 16:

      • For Database Administrator (OSDBA) Group, select dba.
      • For Database Operator (OSOPER) Group (Optional), select oinstall.

        Figure 16

        Figure 16. Privileged Operating System Groups Page

    8. On the Perform Prerequisite Checks page, select Ignore All.
    9. On the Summary page, check all the settings you selected to ensure they are appropriate and then click Install.

      The Execute Configuration Scripts dialog box asks you to execute root.sh, as shown in Figure 17.

      Figure 17

      Figure 17. Execute Configuration Scripts Dialog Box

  3. To complete the installation, open a terminal window and execute the script on all zone cluster nodes, as instructed, as shown in Listing 17.

    vzhost1c# /u01/oracle/product/11.2.0.2/root.sh
    Running Oracle 11g root script...
    
    The following environment variables are set as:
        ORACLE_OWNER= oracle
        ORACLE_HOME=  /u01/oracle/product/11.2.0.2
    
    Enter the full pathname of the local bin directory: [/usr/local/bin]: /opt/local/bin
    The contents of "dbhome" have not changed. No need to overwrite.
    The contents of "oraenv" have not changed. No need to overwrite.
    The contents of "coraenv" have not changed. No need to overwrite.
    
    Entries will be added to the /var/opt/oracle/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root script.
    Now product-specific root actions will be performed.
    chmod: WARNING: can't change /u01/oracle/product/11.2.0.2/lib/libskgxn2.so
    Finished product-specific root actions
    
    Listing 17. Executing the Script

Apply Oracle Grid Infrastructure 11.2.0.2.3 GI PSU patch to Oracle Database Using OPatch

Apply patch 11.2.0.2 GIB3 to each zone cluster node by performing the following steps.

  1. Replace OPatch with the latest version, 11.2.0.1.6.
  2. Back up the OPatch version in DB_HOME, for example:

    $ cd /u01/oracle/product/11.2.0.2/ 
    $ mv OPatch OPatch.bak
    
  3. As the software owner, copy p6880880_112000_SOLARIS64.zip to DB_HOME:

    $ cp p6880880_112000_SOLARIS64.zip /u01/oracle/product/11.2.0.2
    $ cd /u01/oracle/product/11.2.0.2
    $ unzip p6880880_112000_SOLARIS64.zip
    
  4. As the software owner, create an Oracle Configuration Manager file on each node, for example:

    $ cd /var/tmp
    $ /u01/oracle/product/11.2.0.2/OPatch/ocm/bin/emocmrsp 
    
    OCM Installation Response Generator 10.3.4.0.0 - Production
    Copyright (c) 2005, 2010, Oracle and/or its affiliates.  All rights reserved.
    
    Provide your email address to be informed of security issues, install and
    initiate Oracle Configuration Manager. Easier for you if you use your My
    Oracle Support Email address/User Name.
    Visit http://www.oracle.com/support/policies.html for details.
    
  5. Provide your e-mail address and then press Enter to bypass providing a password, for example:

    Email address/User Name: <your email address>
    Provide your My Oracle Support password to receive security updates via your 
    My Oracle Support account.
    Password (optional):            
    The OCM configuration response file (ocm.rsp) was successfully created.
    
  6. Execute the following as root:

    # /u01/oracle/product/11.2.0.2/OPatch/opatch auto /var/tmp/gib3 -oh /u01/oracle/product/11.2.0.2
    
    Executing /usr/bin/perl /u01/oracle/product/11.2.0.2/OPatch/crs/patch112.pl 
    -patchdir /var/tmp -patchn gib3 -oh /u01/oracle/product/11.2.0.2 -paramfile 
    /u01/grid/product/11.2.0.2/crs/install/crsconfig_params
    opatch auto log file location is 
    /u01/oracle/product/11.2.0.2/OPatch/crs/../../cfgtoollogs/opatchauto2011-08-10_20-30-35.log
    Detected Oracle Clusterware install
    Using configuration parameter file: /u01/grid/product/11.2.0.2/crs/install/crsconfig_params
    OPatch  is bundled with OCM, Enter the absolute OCM response file path:
    
  7. Provide the full path to the ocm.rsp file created earlier, for example:

    /var/tmp/ocm.rsp
    
  8. As the software owner, verify the patch installation, as shown in Listing 18.

    $ export ORACLE_HOME=/u01/oracle/product/11.2.0.2  
    $ /u01/oracle/product/11.2.0.2/OPatch/opatch lsinventory -oh $ORACLE_HOME
    
    Invoking OPatch 11.2.0.1.6
    
    Oracle Interim Patch Installer version 11.2.0.1.6
    Copyright (c) 2011, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/oracle/product/11.2.0.2
    Central Inventory : /u01/oraInventory
       from           : /var/opt/oracle/oraInst.loc
    OPatch version    : 11.2.0.1.6
    OUI version       : 11.2.0.2.0
    Log file location : /u01/oracle/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-08-11_10-38-51AM.log
    
    Lsinventory Output file location : 
    /u01/oracle/product/11.2.0.2/cfgtoollogs/opatch/lsinv/lsinventory2011-08-11_10-38-51AM.txt
    
    --------------------------------------------------------------------------------
    Installed Top-level Products (1): 
    
    Oracle Database 11g                                                  11.2.0.2.0
    There are 1 products installed in this Oracle Home.
    
    
    Interim patches (2) :
    
    Patch  12419331     : applied on Wed Aug 10 20:53:24 PDT 2011
    Unique Patch ID:  13850636
       Created on 12 Jul 2011, 07:01:28 hrs PST8PDT
       Bugs fixed:
         10151017, 10158965, 11724916, 10190642, 12586486, 12586487, 10129643
         12586488, 12586489, 10018789, 9744252, 10248523, 9956713, 10356513
         9715581, 9770451, 10378005, 10170431, 10425676, 10222719, 10126094
         9591812, 10127360, 10132870, 10094201, 9443361, 10193846, 11664046
         11069199, 10324294, 10245086, 12586490, 10205230, 12586491, 10052141
         12586492, 12586493, 12586494, 10142788, 11818335, 11830776, 12586495
         9905049, 11830777, 12586496, 11830778, 6892311, 10040921, 10077191
         10358019, 12431716, 10219576, 10258337, 11707699, 10264680, 10209232
         11651810, 10102506, 11067567, 9881076, 10278372, 10040531, 10621169
         10155605, 10082277, 10356782, 10218814, 9078442, 9788588, 10157249
         9735237, 10317487, 12326246, 11707302, 10310299, 10636231, 10230571
         11065646, 12419321, 10368698, 10079168, 10013431, 10228151, 10233732
         10324526, 8223165, 10238786, 10217802, 10061015, 9953542, 9572787
         10052956, 10080579, 11699057, 12620422, 10332111, 10227288, 10329146
         10332589, 10110863, 10073683, 9869401, 10019218, 10229719, 11664719
         9539440, 10373381, 9735282, 9748749, 11724984, 10022980, 10411618
         11800854, 12419331, 11674485, 10187168, 6523037, 10648873, 9724970
         10053725, 10084145, 10367188, 11800170, 11695285, 10157402, 9651350, 10299224
    
    Patch  12419353     : applied on Wed Aug 10 20:37:44 PDT 2011
     Patch ID:  13850636
       Created on 18 Jul 2011, 05:19:12 hrs UTC
       Bugs fixed:
         12419353, 10157506, 10178670, 10425672, 12311357, 9959110, 10272615
         10314123, 10014392, 10089120, 10057296, 9864003, 11775080, 9916145
         10044622, 12399977, 12421404, 12340700, 10056713, 10637741, 9939306
         9902536, 10007185, 10376847, 10038791, 11741224, 11655840, 10048487
         10322157, 10260251, 10052721, 10028235, 10027079, 10357258, 10045436
         10231906, 10622973, 9891341, 10072474, 10036834, 10029900, 9974223
         10016083, 9918485, 11781515, 10040647, 10069541, 10029119, 10233159
         12332919, 9812956, 10036193, 10015210, 12340501, 10621175, 11877079
         8906163, 10111010, 10115514, 10104377, 10057680, 10280665, 10078086
         9944948, 10146768, 10052529, 10011084, 10012319, 10073075, 10233811
         10299006, 10248739, 10236074, 10128191, 11071429, 10019726, 9975837
         10253630, 9949676, 11936945, 10637483, 10157622, 11698552, 10385838
         10053985, 10425674, 9812970, 11828633, 11899801, 10083789, 9876201
         10073372, 9963327, 11077756, 10375649, 9336825, 11682409, 10062301
         10018215, 10105195, 10419987, 10071992, 10634513, 9926027, 10103954
         10028343, 11866171, 10065216, 9907089, 9897335, 10190153, 11744313
         10175855, 10284828, 10028637, 10361177, 9979706, 10324594, 10015460
         9971646, 11782423, 11654726, 9978765, 10398810, 11904778, 10397652
         9915329, 10107380, 10110969, 10305361, 10331452, 10083009, 10631693
         10008467, 10048027, 10040109, 9944978, 10033106, 9978195, 11840629
         10042143, 10284693, 10638381, 9679401, 11663339, 10075643, 10205290
         10124517, 11069614, 9593552, 10168006, 12677816, 11807012
    
    
    
    Rac system comprising of multiple nodes
      Local node = vzhost1c
    
      Remote node = vzhost2c
      Remote node = vzhost3c
      Remote node = vzhost4c
    
    Listing 18. Verifying the Patch Installation

Create the Database

From one zone cluster node, as the Oracle software owner, create an Oracle ASM disk group for a test database called swb, as follows.

  1. Execute the following commands to launch the ASM Configuration Assistant:

    $ export DISPLAY=<hostname>:<n>
    $ export ORACLE_HOME=/u01/oracle/product/11.2.0.2
    $ /u01/grid/product/11.2.0.2/bin/asmca
    
  2. In the ASM Configuration Assistant, do the following:

    1. In the Disk Groups tab, click Create.
       
    2. Do the following in the Create Disk Group page, as shown in Figure 18.

      • Specify swbdg for Disk Group Name.
      • Select /dev/did/rdsk/d9s6, /dev/did/rdsk/d10s6, /dev/did/rdsk/d14s6, and /dev/did/rdsk/d15s6.
      • For Failure Group, specify fgr1 for /dev/did/rdsk/d9s6 and /dev/did/rdsk/d10s6.
      • For Failure Group, specify fgr2 for /dev/did/rdsk/d14s6 and /dev/did/rdsk/d15s6.
      • Click OK to create the disk group.
      • After the swbdg disk group creation is complete, click Exit to close the ASM Configuration Assistant.

        Figure 18

        Figure 18. Create Disk Group Page

  3. Start the Database Configuration Assistant, for example:

    $ u01/oracle/product/11.2.0.2/bin/dbca
    
  4. In the Database Configuration Assistant, do the following:

    1. Select the Oracle Real Application Clusters (RAC) database option.
    2. On the Operations page, select the Create a Database option.
    3. On the Database Templates page, select the Custom Database option.
    4. On the Database Identification page, click Select All and make the following selections, as shown in Figure 19:

      • For Configuration Type, select Admin-Managed.
      • For Global Database Name, specify swb.
      • For SID Prefix, specify swb.

        Figure 19

        Figure 19. Database Identification Page

    5. On the Management Options page, make sure Configure Enterprise Manager and Enable automatic maintenance tasks are not selected, as shown in Figure 20 and Figure 21.

      Figure 20

      Figure 20. Enterprise Manager Tab

      Figure 21

      Figure 21. Automatic Maintenance Tasks Tab

    6. On the Database Credentials page, select the Use the Same Administrative Password for All Accounts option.
    7. On the Database File Locations page, select the Use Oracle-Managed Files option and specify +SWBDG for Database Area.
    8. On the Recovery Configuration page, make the following selections, as shown in Figure 22:

      • Select the Specify Fast Recovery Area checkbox.
      • For Fast Recovery Area, select +SWBDG.
      • For Fast Recovery Area Size, select 7527 M Bytes.

        Figure 22

        Figure 22. Recovery Configuration Page

    9. On the Database Content page, use the default selections, as shown in Figure 23.

      Figure 23

      Figure 23. Database Content Page

    10. On the Initialization Parameters page, use the default values, as shown in Figure 24, and then click Finish to start the database creation.

      Figure 24

      Figure 24. Initialization Parameters page

  5. Verify that database was created and is running:

    $ /u01/grid/product/11.2.0.2/bin/srvctl status database -d swb
    Instance swb1 is running on node vzhost1c
    Instance swb2 is running on node vzhost2c
    Instance swb3 is running on node vzhost3c
    Instance swb4 is running on node vzhost4c
    
  6. To add the second standby NIC of the IPMP group to Cluster Ready Services, as root from one zone cluster node, run the following command:

    $ GRID_HOME/bin/srvctl modify nodeapps -n <hostname> -A <VIP address>/<subnet>/NIC0\|NIC1
    

    For example:

    $ srvctl modify nodeapps -n vzhost1c -A  10.134.35.100/255.255.255.0/e1000g0\|e1000g1
    
  7. Use srvctl config to show that e1000g1 was added:

    $ /u01/grid/product/11.2.0.3/bin/srvctl config nodeapps
    Network exists: 1/10.134.35.0/255.255.255.0/e1000g0:e1000g1, type static
    VIP exists: /vzhost1e/10.134.35.100/10.134.35.0/255.255.255.0/e1000g0:e1000g1, hosting node vzhost1c
    VIP exists: /vzhost2e/10.134.35.101/10.134.35.0/255.255.255.0/e1000g0:e1000g1, hosting node vzhost2c
    VIP exists: /vzhost3e/10.134.35.102/10.134.35.0/255.255.255.0/e1000g0:e1000g1, hosting node vzhost3c
    VIP exists: /vzhost4e/10.134.35.103/10.134.35.0/255.255.255.0/e1000g0:e1000g1, hosting node vzhost4c
    

Create the Oracle Solaris Cluster Resource

Use the following procedure to create the Oracle Solaris Cluster Resource. Alternatively, you can use clsetup.

  1. From one zone cluster node, execute the following command to register the SUNW.crs_framework resource type in the zone cluster:

    # clrt register SUNW.crs_framework
    
  2. Add an instance of the SUNW.crs_framework resource type to the Oracle RAC framework resource group:

    # clresource create -t SUNW.crs_framework \
      -g rac-framework-rg \
      -p resource_dependencies=rac-framework-rs \
      crs-framework-rs
    
  3. Register the scalable Oracle Automatic Storage Management instance proxy resource type:

    # clresourcetype register SUNW.scalable_asm_instance_proxy
    
  4. Register the Oracle Automatic Storage Management disk group resource type:

    # clresourcetype register SUNW.scalable_asm_diskgroup_proxy
    
  5. Create resource groups asm-inst-rg and asm-dg-rg:

    # clresourcegroup create -S asm-inst-rg asm-dg-rg
    
  6. Set a strong positive affinity on rac-fmwk-rg by asm-inst-rg:

    # clresourcegroup set -p rg_affinities=++rac-framework-rg asm-inst-rg
    
  7. Set a strong positive affinity on asm-inst-rg by asm-dg-rg:

    # clresourcegroup set -p rg_affinities=++asm-inst-rg asm-dg-rg
    
  8. Create a SUNW.scalable_asm_instance_proxy resource and set the resource dependencies:

    # clresource create asm-inst-rg \
      -t SUNW.scalable_asm_instance_proxy \
      -p ORACLE_HOME=/u01/grid/product/11.2.0.2 \
      -p CRS_HOME=/u01/grid/product/11.2.0.2 \
      -p "ORACLE_SID{vzhost1c}"=+ASM1 \
      -p "ORACLE_SID{vzhost2c}"=+ASM2 \
      -p "ORACLE_SID{vzhost3c}"=+ASM3 \
      -p "ORACLE_SID{vzhost4c}"=+ASM4 \
      -p resource_dependencies_offline_restart=crs-framework-rs \
      -d asm-inst-rs
    
  9. Add an Oracle Automatic Storage Management disk group resource type to the asm-dg-rg resource group:

    # clresource create -g asm-dg-rg -t SUNW.scalable_asm_diskgroup_proxy \
      -p asm_diskgroups=CRSDG,SWBDG \
      -p resource_dependencies_offline_restart=asm-inst-rs \
      -d asm-dg-rs
    
  10. On a cluster node, bring the asm-inst-rg resource group online in a managed state:

    # clresourcegroup online -eM asm-inst-rg
    
  11. On a cluster node, bring the asm-dg-rg resource group online in a managed state:

    # clresourcegroup online -eM asm-dg-rg
    
  12. Create a scalable resource group to contain the proxy resource for the Oracle RAC database server.

    # clresourcegroup create -S \ 
      -p rg_affinities=++rac-framework-rg,++asm-dg-rg \ 
      rac-swbdb-rg
    
  13. Register the SUNW.scalable_rac_server_proxy resource type:

    # clresourcetype register SUNW.scalable_rac_server_proxy
    
  14. Add a database resource to the resource group:

    # clresource create -g rac-swbdb-rg \
      -t SUNW.scalable_rac_server_proxy \
      -p resource_dependencies=rac-framework-rs \
      -p resource_dependencies_offline_restart=crs-framework-rs,asm-dg-rs \
      -p oracle_home=/u01/oracle/product/11.2.0.2 \
      -p crs_home=/u01/grid/product/11.2.0.2 \
      -p db_name=swb \
      -p "oracle_sid{vzhost1c}"=swb1 \
      -p "oracle_sid{vzhost2c}"=swb2 \
      -p "oracle_sid{vzhost3c}"=swb3 \
      -p "oracle_sid{vzhost4c}"=swb4 \
      -d rac-swb-srvr-proxy-rs
    
  15. Bring the resource group online:

    # clresourcegroup online -emM rac-swbdb-rg
    
  16. Check the status of the cluster resource, as shown in Listing 19:

    # clrs status
    
    === Cluster Resources ===
    
    Resource Name             Node Name         State      Status Message	
    -------------            ---------         -----       -------------------
    crs_framework-rs          vzhost1c          Online     Online
                              vzhost2c          Online     Online
                              vzhost3c          Online     Online
                              vzhost4c          Online     Online
    
    rac-framework-rs          vzhost1c          Online     Online
                              vzhost2c          Online     Online
                              vzhost3c          Online     Online
                              vzhost4c          Online     Online
    
    asm-inst-rs               vzhost1c          Online     Online - +ASM1 is UP and ENABLED
                              vzhost2c          Online     Online - +ASM2 is UP and ENABLED
                              vzhost3c          Online     Online - +ASM3 is UP and ENABLED
                              vzhost4c          Online     Online - +ASM4 is UP and ENABLED
    
    asm-dg-rs                 vzhost1c          Online     Online - Mounted: SWBDG
                              vzhost2c          Online     Online - Mounted: SWBDG
                              vzhost3c          Online     Online - Mounted: SWBDG
                              vzhost4c          Online     Online - Mounted: SWBDG
    
    rac-swb-srvr-proxy-rs     vzhost1c          Online     Online - Oracle instance UP
                              vzhost2c          Online     Online - Oracle instance UP
                              vzhost3c          Online     Online - Oracle instance UP
                              vzhost4c          Online     Online - Oracle instance UP
    
    Listing 19. Checking the Status of the Cluster Resource

See Also

Here are some additional resources:

About the Author

Vinh Tran is a Quality Engineer in the Oracle Solaris Cluster Group. His responsibilities include, but are not limited to, certification and qualification of Oracle RAC on Oracle Solaris Cluster.

Revision 1.0, 05/09/2012

See sysadmin-related content for all Oracle technologies by following OTN Systems on Facebook and Twitter.