//*Install the iscsi software on both the RAC nodes (calvin, hobbes).
iscsi-initiator-utils rpm is available for both RHEL3.0 and RHEL4.0 from the RedHat Network.*//
[root@calvin ~]# rpm -ivh iscsi-initiator-utils-4.0.3.0-2.i386.rpm
Preparing...                     ############################## [100%]
   1:iscsi-initiator-utils     ############################## [100%]


//*Edit the /etc/iscsi.conf file on both Linux host(calvin,hobbes) and 
append the following lines to the bottom of the file:*//


#############################
Continuous=no
HeaderDigest=never
DataDigest=never
ImmediateData=yes
MaxDiskCommandTimeout=180
ConnFailTimeout=180
DiscoveryAddress=netapp01
#############################

Password:
netapp01> Wed Mar  1 00:08:51 IST [console_login_mgr:info]: root logged in from console

//*This will create an aggregate of 8 GB comprising of 8 disks each of 1 GB.
It will take some while(around 15 mins) to initialize the disks. *//
netapp01> aggr create aggr1 -t raid4 8
Creation of an aggregate with 8 disks has been initiated.  The disks need
to be zeroed before addition to the aggregate. The process has been initiated
and you will be notified via the system log as disks are added.


//*Below you can view aggregate is getting initialized.
You can take a coffee break here !!!*//
netapp01> aggr status aggr1
           Aggr State      Status            Options
          aggr1 creating   raid4, aggr       snapshot_autodelete=off,
                           initializing      lost_write_protect=off
                Volumes: <none>

                Plex /aggr1/plex0: offline, empty, active

//*Once the aggregate is initialized following would be the status.*//
netapp01> aggr status aggr1
           Aggr State      Status            Options
          aggr1 online     raid4, aggr
                Volumes: <none>

                Plex /aggr1/plex0: online, normal, active
                    RAID group /aggr1/plex0/rg0: normal

//*You can disable snaps creatation to maximize the performance.*// 
netapp01> aggr options aggr1 nosnap on
netapp01> aggr status aggr1
           Aggr State      Status            Options
          aggr1 online     raid4, aggr       nosnap=on
                Volumes: <none>

                Plex /aggr1/plex0: online, normal, active
                    RAID group /aggr1/plex0/rg0: normal

//*Now you create the volume(vol1) of 2GB. This can dynamically extended as per your requirement. 
2GB would be sufficient for creating the seed database.*//
netapp01> vol create vol1 aggr1 2g
Creation of volume 'vol1' with size 2g on containing aggregate
'aggr1' has completed.

//*Status of the volume can be checked with following command.*//
netapp01> vol status vol1
         Volume State      Status            Options
           vol1 online     raid4, flex       create_ucode=on,
                                             convert_ucode=on
                Containing aggregate: 'aggr1'

netapp01> lun setup
        This setup will take you through the steps needed to create LUNs
        and to make them accessible by initiators. You can type ^C (Control-C)
        at any time to abort the setup and no unconfirmed changes will be made
        to the system.
Do you want to create a LUN? [y]: y
Multiprotocol type of LUN (image/solaris/windows/hpux/aix/linux/netware/vmware)
        [image]: linux
        A LUN path must be absolute. A LUN can only reside in a volume or
        qtree root. For example, to create a LUN with name "lun0" in the
        qtree root /vol/vol1/q0, specify the path as "/vol/vol1/q0/lun0".
Enter LUN path: /vol/vol1/lun0
        A LUN can be created with or without space reservations being enabled.
        Space reservation guarantees that data writes to that LUN will never
        fail.
Do you want the LUN to be space reserved? [y]: y
        Size for a LUN is specified in bytes. You can use single-character
        multiplier suffixes: b(sectors), k(KB), m(MB), g(GB) or t(TB).
Enter LUN size: 2g
        You can add a comment string to describe the contents of the LUN.
        Please type a string (without quotes), or hit ENTER if you don't
        want to supply a comment.
Enter comment string: oradata01
        The LUN will be accessible to an initiator group. You can use an
        existing group name, or supply a new name to create a new initiator
        group. Enter '?' to see existing initiator group names.
Name of initiator group []: ora
Type of initiator group ora (FCP/iSCSI) [iSCSI]: iSCSI
        An iSCSI initiator group is a collection of initiator node names.Each
        node name can begin with either 'eui.' or 'iqn.' and should be in the
        following formats: eui.{EUI-64 address} or iqn.yyyy-mm.{reversed domain
        name}:{optional string composed of alphanumeric characters, '-', '.'
        and ':'}
        Eg: iqn.2001-04.com.acme:storage.tape.sys1.xyz or eui.02004567A425678D
        You can separate node names by commas. Enter '?' to display a list of
        connected initiators. Hit ENTER when you are done adding node names to
        this group.
Enter comma separated nodenames: iqn.1987-05.com.cisco:calvin
Enter comma separated nodenames: iqn.1987-05.com.cisco:hobbes
Enter comma separated nodenames:
The initiator group has an associated OS type. The following are
currently supported: solaris, windows, hpux, aix, linux, netware, vmware
or default.
OS type of initiator group "ora" [linux]:
        The LUN will be accessible to all the initiators in the
        initiator group. Enter '?' to display LUNs already in use
        by one or more initiators in group "ora".
LUN ID at which initiator group "ora" sees "/vol/vol1/lun0" [0]:
LUN Path                          : /vol/vol1/lun0
OS Type                           : linux
Size                              : 2g (2097152000)
Comment                           : oradata01
Initiator Group                   : ora
Initiator Group Type              : iSCSI
Initiator Group Members           : iqn.1987-05.com.cisco:calvin
                                  : iqn.1987-05.com.cisco:hobbes
Mapped to LUN-ID                  : 0
Do you want to accept this configuration? [y]: y
Do you want to create another LUN? [n]: n

//*Check the status of the iscsi service on the NetApp Filer.*//
netapp01> iscsi status
iSCSI service is not running

//*Start the service on the NetApp Filer.*//
netapp01> iscsi start
Wed Mar  1 23:13:42 IST [iscsi.adapter.online:notice]: ISCSI: iswta, Adapter brought online.
iSCSI service started
Wed Mar  1 23:13:42 IST [iscsi.service.startup:info]: iSCSI service startup

//*Recheck the status of the iscsi service.*//
netapp01> iscsi status
iSCSI service is running

//*Check the status of the LUN on the NetApp filer.*//
netapp01> lun show
        /vol/vol1/lun0               1.6g (1712324608)    (r/w, online, mapped)



//*Restart the iscsi service on both the host.*//
[root@calvin ~]# service iscsi restart
Stopping iscsid:                                      [  OK  ]
Removing iscsi driver:                                [  OK  ]
Checking iscsi config:                                [  OK  ]
Loading iscsi driver:                                 [  OK  ]
Starting iscsid:                                      [  OK  ]

//*Edit the /etc/initiatorname.iscsi file on the both the RAC nodes.*//
(calvin, hobbes). To add the following lines
"InitiatorName=iqn.1987-05.com.cisco:calvin" only on the first host
"InitiatorName=iqn.1987-05.com.cisco:hobbes" only on the second host

//*Now restart the iscsi service on both the host.*//
[root@calvin ~]# service iscsi restart
Stopping iscsid:                                      [  OK  ]
Removing iscsi driver:                                [  OK  ]
Checking iscsi config:                                [  OK  ]
Loading iscsi driver:                                 [  OK  ]
Starting iscsid:                                      [  OK  ]

//*After restarting the iscsi service you can check the status with the dmesg command.*//
[root@calvin dev]# dmesg
iscsi-sfnet: Loading iscsi_sfnet version 4:0.1.11-1
iscsi-sfnet: Control device major number 254
iscsi-sfnet:host5: Session established
scsi5 : SFNet iSCSI driver
  Vendor: NETAPP    Model: LUN               Rev: 0.2
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sda: 3344384 512-byte hdwr sectors (1712 MB)
SCSI device sda: drive cache: write through
 sda: unknown partition table
Attached scsi disk sda at scsi5, channel 0, id 0, lun 0


//*Now you can create the partitions on disk. This step is only required from the one node.*//
[root@calvin bin]# fdisk /dev/sda
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sda: 1712 MB, 1712324608 bytes
53 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 3286 * 512 = 1682432 bytes

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1017, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1017, default 1017):
Using default value 1017

Command (m for help): p

Disk /dev/sda: 1712 MB, 1712324608 bytes
53 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 3286 * 512 = 1682432 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1        1017     1670900   83  Linux


Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

//*Now you can implement Automatic Storage Management(ASM) or Oracle Cluster Filesystem(OCFS) on /dev/sda1*//