Oracle Validated Configuration

Configuration Summary:	Oracle Database 10g Release 2 (Single Instance) and Oracle Real Application Clusters (RAC) on Red Hat Enterprise Linux 4 AS/ES Update 3 x86-64 using OCFS2
Publication Date:	August 14, 2006
Version:	1.0
Server Platform:	IBM eServer 326
Storage:	NetApp FAS900 Series
Oracle Software:	Oracle Database 10g Release 2 (10.2.0.2) for Linux x86-64
Linux Distribution:	RHEL4 Update 3 x86-64

Server and Storage Platform Details

	2 x IBM eServer 326
    2 x Dual Core AMD Opteron 275 1.8 GHz (4 cores)
    8 GB RAM
    1 x 146.8 GB 10K HD
    2 x Integrated Gigabit NIC
    1 x NetXtreme 1000 Express Ethernet Adapter
 
  Additional models covered by this configuration
    eServer 326m
    BladeCenter LS20 blade

  1 x NetApp FAS960 running ONTAP v7.0.3
    14 x 72 GB 10K HD

  Additional models covered by this configuration
    IBM N Series storage (ONTAP v7.0.3) with iSCSI
    NetApp FAS900 Series storage running ONTAP v7.0.3

  Linux Distribution Details
    RHEL 4 AS Update 3 x86-64 (default install )
      kernel-smp-2.6.9-34.ELsmp.x86_64.rpm (from Update 3)

  Additional packages needed from distribution
    compat-gcc-32-3.2.3-47.3.x86_64.rpm
    compat-libstdc++-33-3.2.3-47.3.x86_64.rpm
    gcc-c++-3.4.5-2.x86_64.rpm
    glibc-devel-2.3.4-2.19.i386.rpm
    glibc-devel-2.3.4-2.19.x86_64.rpm
    glibc-headers-2.3.4-2.19.x86_64.rpm
    glibc-kernheaders-2.4-9.1.98.EL.x86_64.rpm
    iscsi-initiator-utils-4.0.3.0-3.x86_64.rpm
    kernel-devel-2.6.9-34.EL.x86_64.rpm
    kernel-smp-devel-2.6.9-34.EL.x86_64.rpm
    libaio-0.3.105-2.x86_64.rpm
    libaio-devel-0.3.105-2.x86_64.rpm
    libstdc++-devel-3.4.5-2.i386.rpm
    libstdc++-devel-3.4.5-2.x86_64.rpm
    rsh-server-0.17-25.3.x86_64.rpm
    strace-4.5.13-0.EL4.1.x86_64.rpm
    sysstat-5.0.5-7.rhel4.x86_64.rpm

  Additional packages from Oracle
    ocfs2-2.6.9-34.ELsmp-1.2.1-1.x86_64.rpm
    ocfs2console-1.2.1-1.x86_64.rpm
    ocfs2-tools-1.2.1-1.x86_64.rpm

  Boot options
   numa=off elevator=deadline

  /etc/modprobe.conf:
    options e1000 FlowControl=1
    options hangcheck_timer hangcheck_reboot=1
 
  /etc/sysctl.conf settings
    fs.file-max=327679
    kernel.msgmni=2878
    kernel.msgmax=8192
    kernel.sem=250 32000 100 142
    kernel.shmmni=4096
    kernel.shmall=3279547
    # set to a value half the size of physical memory
    kernel.shmmax=3700000000
    # rmem_max can be tuned based on workload to balance performance vs lowmem usage
    net.core.rmem_default=262144
    net.core.rmem_max=2097152
    net.core.wmem_default=262144
    net.core.wmem_max=262144
    fs.aio-max-nr=3145728
    net.ipv4.ip_local_port_range=1024 65000
    vm.lower_zone_protection=100
    kernel.msgmnb=65536
    kernel.sysrq=1
    #  needed to enable hugepage usage
    vm.nr_hugepages=512
  
   /etc/selinux/config:
    SELINUX=disabled

   /etc/security/limits.conf:
    oracle soft nofile 131072
    oracle hard nofile 131072
    oracle soft nproc 131072
    oracle hard nproc 131072
    oracle soft core unlimited
    oracle hard core unlimited
    # set memlock greater than or equal to the sga size to allow
    # oracle to use hugepages if configured
    oracle soft memlock 50000000
    oracle hard memlock 50000000

   /etc/sysconfig/o2cb:
   #Note: If the database is on OCFS2, timeouts need to be adjusted. Set O2CB_HEARTBEAT_THRESHOLD to a value which takes into consideration the underlying storage and the workload to avoid timeouts 
   ( E.g:O2CB_HEARTBEAT_THRESHOLD=61 ) O2CB_HEARTBEAT_THRESHOLD=61

   Filesystems tested:
    ocfs2

   Mount options:
    Datafiles,ocr and voting disk files on OCFS2
    _netdev,datavolume,nointr

   Driver Modules:
    The network driver module used (tg3) is the version supplied with RHEL 4 U3 x86-64
    The hangcheck-timer module used is the version supplied with RHEL 4 U3 x86-64

   Miscellaneous
    2 gigabit networks configured: public and interconnect
    The swap partition's size is 8G 
  
Oracle Software Details

  Oracle Database 10g Release 2 (10.2.0.2) Single Instance and 
  Oracle Real Application Clusters (RAC) for Linux x86-64
      patch 5071492
      patch 4639236
      patch 4690794
      patch 5036588

Configuration Feedback

    Bug      Summary
    Number

    5041764  CFQ io scheduler can delay/hang io (bz 151368)
               affects:  heavy io access to raw/block devices   
               symptom:  io requests will start to hang or take a long to complete 
                         RAC node could be evicted 
               workaround:  boot w/ elevator=deadline

    5058952  e1000 flow control defaults to none in the 2.6 kernel
               affects:  e1000 network interfaces with heavy traffic
               symptom:  RAC interconnect may get lost blocks
               solution:  load e1000 with FlowControl=1 (Rx)
    
    5215593  unable to start RAC with db_cache_size > 4gb
               affects:  RAC instances with > 4gb db_cache_size
               symptom:  nodes that started up correctly with 10.2.0.1
                         may fail to startup with ORA-4031 errors in 10.2.0.2
               workaround:  set _ksmg_granule_size=33554432 in the init.ora.
                            Larger sga sizes may need a higher value.

    5215622  libknlopt.a not relinked
               affects:  10.2.0.2 RAC (did not occur in 10.1)
               symptom:  may get ORA-7445 errors during startup
               workaround:  manually relink on each node

    5041394  relink errors after upgrading from 10.1 to 10.2 
               affects:  10.1 -> 10.2 upgrade process
               symptom:  during 10.2 upgrade, relink step will fail
                         with 'undefined reference' errors
               workaround:  after doing 10.2 prereq steps, rename
                 $ORACLE_HOME/lib/stubs to $ORACLE_HOME/lib/stubs.10.1
                 $ORACLE_HOME/lib32/stubs to $ORACLE_HOME/lib32/stubs.10.1
                 then start installing 10.2.0.1  
       
    5071492  LMS crash with ORA-00600: internal error code, arguments: [kclastf_1], [2], [],
               affects:  10.2.0.2 RAC
               symptom:  the instance could terminate
               solution:  apply patch 5071492

    4639236  ORA-600 [kclcls_5] in RAC instance
               affects:  10.2.0.2 RAC
               symptom:  the instance could terminate 
               solution: apply patch 4639236

    4690794  "ERROR IN KQLMBIVG SEE LCK TRACE FILE" [LT] [LB]  KJUSERCLIENTLOCK 
               affects: 10.2.0.2 RAC
               symptom: Error messages in the alert logs
               solution: apply patch 4690794 

    5036588  LMD0 PROCESS RECEIVED OS SIGNAL #11
               affects: 10.2.0.2 RAC
               symptom: the instance could terminate
               solution: apply patch  5036588

    5017352  warning messages in trace files during certain workloads with async io turned on
               affects: performance
               symptom: Following warnings in trace files
               WARNING: ORACLE PROCESS RUNNING OUT OF OSKERNEL I/O RESOURCES
               workaround : none so far

    5463714 system will hang if reboot or restart iscsi service without unmount ocfs2 partition on iscsi volume
              affects: system hang
              symptom: restart iscsi service or reboot without umount iscsi volume cause system hang
              workaround: unmount iscsi volume before restart iscsi service or reboot server

    5333606 rpm throws errors if selinux is disabled at OS installation and is enabled at runtime
             affects: error at rpm packages installs
             symptom: during packages installation, errors indicate the scriplet failures
             workaround: do not change selinux mode from disable to enable or visa versa at runtime. 
             Or you could execute the preinstall and postinstall scripts manual

    5093059  netdump doesn't load with bonding configured (it 85145)
              affects:  bringing up netdump over a bonded interface
              symptom:  /var/log/messages will have an error:
                        netlog: bond0 doesn't support polling, aborting.
              workaround:  none at the moment

    4593892  repeated warnings in dmesg about lost ticks (it 80129/bz 139376)
              affects:  usually seen on x86-64, both Intel and Opteron based systems
              symptom:  dmesg will have errors like: warning: many lost ticks.
              Your time source seems to be instable or some driver is hogging interupts
              rip __do_softirq+0x4d/0xd0
              workaround:  none at the moment