|
1. Introduction
One of the most
efficient ways to become familiar with Oracle Real Application Clusters
(RAC) 10g technology is to have access to an actual
Oracle RAC 10g cluster. There's no better way to
understand its benefits—including fault tolerance,
security, load balancing, and scalability—than to
experience them directly.
Unfortunately, for many
shops, the price of the hardware required for a typical production RAC
configuration makes this goal impossible. A small two-node cluster can
cost from US$10,000 to well over US$20,000. That cost would not even
include the heart of a production RAC environment—typically
a storage area network—which can start at US$10,000.
For those who want to become familiar
with Oracle RAC 10g without a major cash outlay,
this guide provides a low-cost alternative to configuring an Oracle RAC
10g Release 2 system using commercial off-the-shelf
components and downloadable software at an estimated cost of US$1,200
to US$1,800. The system involved comprises a dual-node cluster (each
with a single processor) running Linux (CentOS 4.2) with a shared disk storage based on IEEE1394 (FireWire)
drive technology.
This guide will mark
the last in a series to make use of FireWire technology as the shared
storage medium in order to build an inexpensive Oracle
RAC 10g system. The most current published version takes advantage of iSCSI;
more specifically, it explains how to build a network storage server using Openfiler. Powered
by rPath Linux,
Openfiler is a free browser-based network storage management utility
that delivers file-based Network Attached Storage (NAS) and block-based
Storage Area Networking (SAN) in a single framework. Openfiler supports
CIFS, NFS, HTTP/DAV, FTP, however; I will only be making use of its
iSCSI capabilities to implement an inexpensive SAN for the shared
storage component required by OracleRAC 10g.
Please note, that this
is not the only way to build a low-cost Oracle RAC 10g system. I have
worked on other solutions that utilize an implementation based on SCSI
rather than FireWire for shared storage. In most cases, SCSI will cost
more than a FireWire solution where an inexpensive SCSI configuration
will consist of:
- SCSI Controller:Two SCSI controllers
priced from $20 (Adaptec AHA-2940UW) to $220 (Adaptec
39320A-R) each
- SCSI Enclosure: $70 - (Inclose
1 Bay 3.5" U320 SCSI Case)
- SCSI Hard Drive: $140 - (36GB
15K 68p U320 SCSI Hard Drive)
- SCSI Cables: Two SCSI cables priced at
$20 each - (3ft External HD68 to HD68 U320 Cable)
Keep in mind that some motherboards may already include
built-in SCSI controllers.
It is important to note
that this configuration should never
be run in a production environment and that it
is not supported by Oracle or any other
vendor. In a production environment, fibre
channel—the high-speed serial-transfer interface that can
connect
systems and storage devices in either point-to-point or switched
topologies—is the technology of choice. FireWire offers a
low-cost alternative to fibre channel for testing and development, but
it is not ready for production.
The Oracle9i
and Oracle 10g Release 1 guides used raw
partitions for storing files on shared storage, but here we will make
use of the Oracle Cluster File System Release 2 (OCFS2) and Oracle
Automatic Storage Management (ASM) feature. The two Linux servers will
be configured as follows:
| Oracle Database Files |
| RAC
Node Name |
Instance
Name |
Database
Name |
$ORACLE_BASE |
File
System / Volume Manager for DB Files |
| linux1 |
orcl1 |
orcl |
/u01/app/oracle |
ASM |
| linux2 |
orcl2 |
orcl |
/u01/app/oracle |
ASM |
| Oracle Clusterware Shared Files |
| File Type |
File Name |
Partition |
Mount Point |
File System |
| Oracle Cluster Registry |
/u02/oradata/orcl/OCRFile |
/dev/sda1 |
/u02/oradata/orcl |
OCFS2 |
| CRS Voting Disk |
/u02/oradata/orcl/CSSFile |
/dev/sda1 |
/u02/oradata/orcl |
OCFS2 |
Note that with Oracle
Database 10g Release 2 (10.2), Cluster Ready
Services, or CRS, is now called Oracle Clusterware.
Starting with Oracle Database 10g
Release 2 (10.2), Oracle Clusterware should be installed in a separate
Oracle Clusterware home directory which is non-release specific. This
is a change to the Optimal Flexible Architecture (OFA) rules. You
should not install Oracle Clusterware in a release-specific Oracle home
mount point, (/u01/app/oracle/product/10.2.0/...
for example), as succeeding versions of Oracle Clusterware will
overwrite the Oracle Clusterware installation in the same path. Also,
If Oracle Clusterware 10g Release 2 (10.2) detects
an existing Oracle Cluster Ready Services installation, then it
overwrites the existing installation in the same path.
The Oracle Clusterware software will
be installed to /u01/app/oracle/product/crs on
each of the nodes that make up the RAC cluster. However, the
Clusterware software requires that two of its files—the
Oracle Cluster Registry (OCR) file and the Voting Disk
file—be shared with all nodes in the cluster. These two files
will be installed on shared storage using OCFS2. It is possible (but
not recommended by Oracle) to use RAW devices for these files; however,
it is not possible to use ASM for these two Clusterware files.
The Oracle Database 10g
Release 2 software will be installed into a separate Oracle Home,
namely /u01/app/oracle/product/10.2.0/db_1, on
each of the nodes that make up the RAC cluster. All the Oracle physical
database files (data, online redo logs, control files, archived redo
logs), will be installed to different partitions of the shared drive
being managed by ASM. (The Oracle database files can just as easily be
stored on OCFS2. Using ASM, however, makes the article that much more
interesting!)
Note: This article is only designed
to work as documented with absolutely no substitutions. If you are
looking for an example that takes advantage of Oracle RAC 10g
Release 1 with RHEL 3, click here.
If you are looking for a version that takes advantage of RHEL 4 with iSCSI and OpenFiler for shared storage, click here.
For the previously
published Oracle9i RAC version of this guide, click here.
2.
Oracle RAC 10g Overview
Oracle RAC, introduced
with Oracle9i, is the successor to Oracle Parallel
Server (OPS). RAC allows multiple instances to access the same database
(storage) simultaneously. It provides fault tolerance, load balancing,
and performance benefits by allowing the system to scale out, and at
the same time—because all nodes access the same
database—the failure of one instance will not cause the loss
of
access to the database.
At the heart of Oracle RAC is a
shared disk subsystem. All nodes in the cluster must be able to access
all of the data, redo log files, control files and parameter files for
all nodes in the cluster. The data disks must be globally available to
allow all nodes to access the database. Each node has its own redo log
and control files but the other nodes must be able to access them in
order to recover that node in the event of a system failure.
One of the bigger differences between
Oracle RAC and OPS is the presence of Cache Fusion technology. In OPS,
a request for data between nodes required the data to be written to
disk first, and then the requesting node could read that data. With
cache fusion, data is passed along a high-speed interconnect using a
sophisticated locking algorithm.
Not all clustering solutions use
shared storage. Some vendors use an approach known as a federated
cluster, in which data is spread across several machines
rather than shared by all. With Oracle RAC 10g,
however, multiple nodes use the same set of disks for storing data.
With Oracle RAC, the data files, redo log files, control files, and
archived log files reside on shared storage on raw-disk devices, a NAS,
a SAN, ASM, or on a clustered file system. Oracle's approach to
clustering leverages the collective processing power of all the nodes
in the cluster and at the same time provides failover security.
Pre-configured Oracle10g
RAC solutions are available from vendors such as Dell, IBM and HP for
production environments. This article, however, focuses on putting
together your own Oracle10g RAC environment for
development and testing by using Linux servers and a low cost shared
disk solution; FireWire.
For more background about Oracle RAC,
visit the Oracle RAC Product
Center on OTN.
3.
Shared-Storage Overview
Fibre Channel is one of
the most popular solutions for shared storage. As I mentioned
previously, Fibre Channel is a high-speed serial-transfer interface
used to connect systems and storage devices in either point-to-point or
switched topologies. Protocols supported by Fibre Channel include SCSI
and IP.
Fibre Channel configurations can
support as many as 127 nodes and have a throughput of up to 2.12
gigabits per second. Fibre Channel, however, is very expensive; the
switch alone can start at US$1,000 and high-end drives can reach prices
of US$300. Overall, a typical Fibre Channel setup (including cards for
the servers) costs roughly US$10,000.
A less expensive alternative to Fibre
Channel is SCSI. SCSI technology provides acceptable performance for
shared storage, but for administrators and developers who are used to
GPL-based Linux prices, even SCSI can come in over budget at around
US$2,000 to US$5,000 for a two-node cluster.
Another popular solution is the Sun
NFS (Network File System) found on a NAS. It can be used for shared
storage but only if you are using a network appliance or something
similar. Specifically, you need servers that guarantee direct I/O over
NFS, TCP as the transport protocol, and read/write block sizes of 32K.
4.
FireWire Technology
Developed by Apple Computer and Texas Instruments, FireWire is a cross-platform
implementation of a high-speed serial data bus. With its high bandwidth, long
distances (up to 100 meters in length) and high-powered bus, FireWire is being
used in applications such as
digital video (DV), professional audio, hard drives, high-end digital still cameras
and home entertainment devices. Today, FireWire operates at transfer rates of up to
800 megabits per second while next generation FireWire calls for speeds to a theoretical
bit rate to 1600 Mbps and then up to a staggering 3200 Mbps. That's 3.2 gigabits per
second. This will make FireWire indispensable for transferring massive data files
and for even the most demanding video applications, such as working
with uncompressed high-definition (HD) video or multiple standard-definition (SD)
video streams.
The following chart shows speed comparisons of the various types of disk interfaces.
For each interface, I provide the maximum transfer rates in kilobits (kb), kilobytes (KB),
megabits (Mb), megabytes (MB), gigabits (Gb), and gigabytes (GB) per second.
As you can see, the capabilities of IEEE1394 compare very favorably with
other disk interface and network technologies that are currently available today.
| Disk Interface |
Speed |
| Kb |
KB |
Mb |
MB |
Gb |
GB |
| Serial |
115 |
14.375 |
0.115 |
0.014 |
|
|
| Parallel (standard) |
920 |
115 |
0.92 |
0.115 |
|
|
| 10Base-T Ethernet |
|
|
10 |
1.25 |
|
|
| IEEE 802.11b wireless Wi-Fi (2.4 GHz band) |
|
|
11 |
1.375 |
|
|
| USB 1.1 |
|
|
12 |
1.5 |
|
|
| Parallel (ECP/EPP) |
|
|
24 |
3 |
|
|
| SCSI-1 |
|
|
40 |
5 |
|
|
| IEEE 802.11g wireless WLAN (2.4 GHz band) |
|
|
54 |
6.75 |
|
|
| SCSI-2 (Fast SCSI / Fast Narrow SCSI) |
|
|
80 |
10 |
|
|
| 100Base-T Ethernet (Fast Ethernet) |
|
|
100 |
12.5 |
|
|
| ATA/100 (parallel) |
|
|
100 |
12.5 |
|
|
| IDE |
|
|
133.6 |
16.7 |
|
|
| Fast Wide SCSI (Wide SCSI) |
|
|
160 |
20 |
|
|
| Ultra SCSI (SCSI-3 / Fast-20 / Ultra Narrow) |
|
|
160 |
20 |
|
|
| Ultra IDE |
|
|
264 |
33 |
|
|
| Wide Ultra SCSI (Fast Wide 20) |
|
|
320 |
40 |
|
|
| Ultra2 SCSI |
|
|
320 |
40 |
|
|
| FireWire 400 - (IEEE1394a) |
|
|
400 |
50 |
|
|
| USB 2.0 |
|
|
480 |
60 |
|
|
| Wide Ultra2 SCSI |
|
|
640 |
80 |
|
|
| Ultra3 SCSI |
|
|
640 |
80 |
|
|
| FireWire 800 - (IEEE1394b) |
|
|
800 |
100 |
|
|
| Gigabit Ethernet |
|
|
1000 |
125 |
1 |
|
| PCI - (33 MHz / 32-bit) |
|
|
1064 |
133 |
1.064 |
|
| Serial ATA I - (SATA I) |
|
|
1200 |
150 |
1.2 |
|
| Wide Ultra3 SCSI |
|
|
1280 |
160 |
1.28 |
|
| Ultra160 SCSI |
|
|
1280 |
160 |
1.28 |
|
| PCI - (33 MHz / 64-bit) |
|
|
2128 |
266 |
2.128 |
|
| PCI - (66 MHz / 32-bit) |
|
|
2128 |
266 |
2.128 |
|
| AGP 1x - (66 MHz / 32-bit) |
|
|
2128 |
266 |
2.128 |
|
| Serial ATA II - (SATA II) |
|
|
2400 |
300 |
2.4 |
|
| Ultra320 SCSI |
|
|
2560 |
320 |
2.56 |
|
| FC-AL Fibre Channel |
|
|
3200 |
400 |
3.2 |
|
| PCI-Express x1 - (bidirectional) |
|
|
4000 |
500 |
4 |
|
| PCI - (66 MHz / 64-bit) |
|
|
4256 |
532 |
4.256 |
|
| AGP 2x - (133 MHz / 32-bit) |
|
|
4264 |
533 |
4.264 |
|
| Serial ATA III - (SATA III) |
|
|
4800 |
600 |
4.8 |
|
| PCI-X - (100 MHz / 64-bit) |
|
|
6400 |
800 |
6.4 |
|
| PCI-X - (133 MHz / 64-bit) |
|
|
|
1064 |
8.512 |
1 |
| AGP 4x - (266 MHz / 32-bit) |
|
|
|
1066 |
8.528 |
1 |
| 10G Ethernet - (IEEE 802.3ae) |
|
|
|
1250 |
10 |
1.25 |
| PCI-Express x4 - (bidirectional) |
|
|
|
2000 |
16 |
2 |
| AGP 8x - (533 MHz / 32-bit) |
|
|
|
2133 |
17.064 |
2.1 |
| PCI-Express x8 - (bidirectional) |
|
|
|
4000 |
32 |
4 |
| PCI-Express x16 - (bidirectional) |
|
|
|
8000 |
64 |
8 |
5.
Hardware & Costs
The hardware used to build our
example Oracle10g RAC environment consists of two
Linux servers and components that can be purchased at any local
computer store or over the Internet.
| Server
1 - (linux1) |
Dimension 2400 Series
Intel Pentium 4 Processor at
2.80GHz
1GB DDR SDRAM (at 333MHz)
40GB 7200 RPM Internal Hard
Drive
Integrated Intel 3D AGP Graphics
Integrated 10/100 Ethernet
CDROM (48X Max Variable)
3.5" Floppy
No monitor (Already had one)
USB Mouse and Keyboard
|
US$620 |
| 1 - Ethernet LAN Cards
| |
Each Linux server should
contain two NIC adapters. The Dell Dimension includes an integrated
10/100 Ethernet adapter that will be used to connect to the public
network. The second NIC adapter will be used for the private
interconnect. |
|
US$20 |
| 1 - FireWire Card
| |
The following is a list
of FireWire I/O cards that contain the correct chipset, allow for
multiple logins, and should work with this article (no guarantees
however): |
FireWire 400
FireWire 800
|
Warning:
I was unable to obtain concurrent logins to the FireWire drive when
using the LaCie FireWire 800 PCI Card - (107755) in both nodes. To
resolve this, I used two different FireWire PCI cards: the LaCie
FireWire 800 PCI Card - (107755) in one node and the SIIG FireWire 800
PCI-32T host adapter - (NN-830112) in the second node. Please refer to
the section Troubleshooting
Concurrent Logins to the FireWire Drive for further details. |
|
FireWire I/O cards with
chipsets made by Texas Instruments (TI) or VIA Technologies (VIA) are
known to work. |
|
US$30 |
| Server
2 - (linux2) |
Dimension 2400 Series
Intel Pentium 4 Processor at
2.80GHz
1GB DDR SDRAM (at 333MHz)
40GB 7200 RPM Internal Hard
Drive
Integrated Intel 3D AGP
Graphics
Integrated 10/100 Ethernet
CDROM (48X Max Variable)
3.5" Floppy
No monitor (already had one)
USB Mouse and Keyboard
|
US$620 |
| 1 - Ethernet LAN Cards
| |
Each Linux server should
contain two NIC adapters. The Dell Dimension includes an integrated
10/100 Ethernet adapter that will be used to connect to the public
network. The second NIC adapter will be used for the private
interconnect. |
|
US$20 |
| 1 - FireWire Card
| |
The following is a list
of FireWire I/O cards that contain the correct chipset, allow for
multiple logins, and should work with this article (no guarantees
however): |
FireWire 400
FireWire 800
|
Warning:
I was unable to obtain concurrent logins to the FireWire drive when
using the LaCie FireWire 800 PCI Card - (107755) in both nodes. To
resolve this, I used two different FireWire PCI cards: the LaCie
FireWire 800 PCI Card - (107755) in one node and the SIIG FireWire 800
PCI-32T host adapter - (NN-830112) in the second node. Please refer to
the section Troubleshooting
Concurrent Logins to the FireWire Drive for further details. |
|
FireWire I/O cards with
chipsets made by Texas Instruments (TI) or VIA Technologies (VIA) are
known to work. |
|
US$30 |
| Miscellaneous Components |
FireWire Hard Drive
| |
The following is a list
of FireWire drives (and enclosures) that contain the correct chipset,
allow for multiple logins, and should work with this article (no
guarantees however): |
FireWire 400
FireWire 800
|
|
Ensure that the FireWire
drive that you purchase supports multiple logins. If the drive has a
chipset that does not allow for concurrent access from more than one
server, the disk and its partitions can only be seen by one server at a
time. Disks with the Oxford 911 chipset (FireWire 400), Oxford 912
chipset (FireWire 800), or Oxford 922 chipset (FireWire 800) are known
to work. Note that the Oxford 912 chipset is newer and faster than
Oxford 922. Here are the details about the disk that I purchased for
this test:
Vendor: Maxtor
Model: OneTouch II
Mfg. Part No. or KIT No.: E01G300
Capacity: 300 GB
Cache Buffer: 16 MB
Spin Rate: 7200 RPM
Interface Transfer Rate: 400 Mbits/s
"Combo" Interface: IEEE 1394 / USB 2.0 and USB 1.1 compatible |
|
US$280 |
| 1 - Extra FireWire Cable
Each node in the RAC configuration will need to connect to the shared
storage device (the FireWire hard drive). The FireWire hard drive will
come supplied with one FireWire cable. You will need to purchase one
additional FireWire cable to connect the second node to the shared
storage. Select the appropriate FireWire cable that is compatible with
the data transmission speed (FireWire 400 / FireWire 800) and the
desired cable length.
FireWire 400
FireWire 800
|
US$20 |
| 1 - Ethernet hub or switch
Used for the interconnect between linux1-priv and linux2-priv. A
question I often receive is about substituting the Ethernet switch
(used for interconnect linux1-priv / linux2-priv) with a crossover CAT5
cable. I would not recommend this. I have found that when using a
crossover CAT5 cable for the interconnect, whenever I took one of the
PCs down, the other PC would detect a "cable unplugged" error, and thus
the Cache Fusion network would become unavailable.
|
US$25 |
| 4 - Network Cables
|
US$5
US$5
US$5
US$5 |
| Total |
US$1,685 |
Note that the Maxtor OneTouch
external drive does have two IEEE1394
(FireWire) ports, although it may not appear so at first glance. This
is also true for the other external hard drives I have listed above.
Now that we know the hardware that
will be used in this example, let's take a conceptual look at what the
environment looks like:

Figure 1 Architecture
As we start to go into the details of
the installation, keep in mind
that most tasks will need to be performed on both servers. I will
indicate at the beginning of each section whether or not the task(s)
should be performed on both nodes or not.
6.
Install the Linux Operating System
This section provides a
summary of the screens used to install the Linux operating system. This
guide is designed to work with the Red Hat Enterprise Linux 4 AS/ES
(RHEL4) operating environment. As an alternative, and what I used for
this article, is CentOS 4.2: a free and stable version of the RHEL4
operating environment.
For more detailed installation
instructions, it is possible to use the manuals
from Red Hat Linux. I would suggest, however, that the instructions I
have provided below be used for this configuration.
Before installing the Linux operating
system on both nodes, you should have the FireWire and two NIC
interfaces (cards) installed.
Also, before starting the
installation, ensure that the FireWire drive (our shared storage drive)
is NOT connected to either of the two servers. You
may also choose to connect both servers to the FireWire drive and
simply turn the power off to the drive. Although none of this is
mandatory, it is how I will be performing the installation and
configuration for this article.
Download the following ISO images for
CentOS 4.2:
After downloading
and burning the CentOS images (ISO files) to CD, insert CentOS Disk #1
into the first server (linux1 in this example),
power it on, and answer the installation screen prompts as noted below.
After completing the Linux installation on the first node, perform the
same Linux installation on the second node while substituting the node
name linux1 for linux2 and
the different IP addresses where appropriate.
Boot Screen
The first screen is the CentOS Enterprise Linux boot screen. At the
boot: prompt, hit [Enter] to start the installation process.
Media Test
When asked to test the CD media, tab over to [Skip] and hit [Enter]. If
there were any errors, the media burning software would have warned us.
After several seconds, the installer should then detect the video card,
monitor, and mouse. The installer then goes into GUI mode.
Welcome to
CentOS Enterprise Linux
At the welcome screen, click [Next] to continue.
Language /
Keyboard Selection
The next two screens prompt you for the Language and Keyboard settings.
Make the appropriate selections for your configuration.
Installation
Type
Choose the [Custom] option and click [Next] to continue.
Disk
Partitioning Setup
Select [Automatically partition] and click [Next] continue.
If there were a previous installation
of Linux on this machine, the next screen will ask if you want to
"remove" or "keep" old partitions. Select the option to [Remove all
partitions on this system]. Also, ensure that the [hda] drive is
selected for this installation. I also keep the checkbox [Review (and
modify if needed) the partitions created] selected. Click [Next] to
continue.
You will then be prompted with a
dialog window asking if you really want to remove all partitions. Click
[Yes] to acknowledge this warning.
Partitioning
The installer will then allow you to view (and modify if needed) the
disk partitions it automatically selected. In almost all cases, the
installer will choose 100MB for /boot, double the amount of RAM for
swap, and the rest going to the root (/) partition. I like to have a
minimum of 1GB for swap. For the purpose of this install, I will accept
all automatically preferred sizes. (Including 2GB for swap since I have
1GB of RAM installed.)
Starting with RHEL 4, the installer
will create the same disk configuration as just noted but will create
them using the Logical Volume Manager (LVM). For example, it will
partition the first hard drive (/dev/hda for my configuration) into two
partitions—one for the /boot partition (/dev/hda1) and the
remainder of the disk dedicate to a LVM named VolGroup00 (/dev/hda2).
The LVM Volume Group (VolGroup00) is then partitioned into two LVM
partitions - one for the root filesystem (/) and another for swap. I
basically check that it created at least 1GB of swap. Since I have 1GB
of RAM installed, the installer created 2GB of swap. Saying that, I
just accept the default disk layout.
Boot Loader
Configuration
The installer will use the GRUB boot loader by default. To use the GRUB
boot loader, accept all default values and click [Next] to continue.
Network
Configuration
I made sure to install both NIC interfaces (cards) in each of the Linux
machines before starting the operating system installation. This screen
should have successfully detected each of the network devices.
First, make sure that each of the
network devices are checked to [Active on boot]. The installer may
choose to not activate eth1.
Second, [Edit] both eth0 and eth1 as
follows. You may choose to use different IP addresses for both eth0 and
eth1 and that is OK. If possible, try to put eth1 (the interconnect) on
a different subnet than eth0 (the public network):
eth0:
- Check off the option to [Configure using DHCP]
- Leave the [Activate on boot] checked
- IP Address: 192.168.1.100
- Netmask: 255.255.255.0
eth1:
- Check off the option to [Configure using DHCP]
- Leave the [Activate on boot] checked
- IP Address: 192.168.2.100
- Netmask: 255.255.255.0
Continue by setting your hostname
manually. I used "linux1" for the first node and "linux2" for the
second one. Finish this dialog off by supplying your gateway and DNS
servers.
Firewall
On this screen, make sure to select [No firewall] and click [Next] to
continue. You may be prompted with a warning dialog about not setting
the firewall. If this occurs, simply hit [Proceed] to continue.
Additional
Language Support/Time Zone
The next two screens allow you to select additional language support
and time zone information. In almost all cases, you can accept the
defaults.
Set Root
Password
Select a root password and click [Next] to continue.
Package Group
Selection
Scroll down to the bottom of this screen and select [Everything] under
the "Miscellaneous" section. Click [Next] to continue.
Please note that the installation of
Oracle does not require
all Linux packages to be installed. My decision to install all packages
was for the sake of brevity. Please see section Section 15
("Check RPM Packages for Oracle 10g Release 2") for
a more detailed look at the critical packages required for a successful
Oracle installation.
Note that with some RHEL4
distributions, you will not get the "Package Group Selection" screen by
default. There, you are asked to simply "Install default software
packages" or "Customize software packages to be installed". Select the
option to "Customize software packages to be installed" and click
[Next] to continue. This will then bring up the "Package Group
Selection" screen. Now, scroll down to the bottom of this screen and
select [Everything] under the "Miscellaneous" section. Click [Next] to
continue.
About to Install
This screen is basically a confirmation screen. Click [Next] to start
the installation. During the installation process, you will be asked to
switch disks to Disk #2, Disk #3, and then Disk #4. Click [Continue] to
start the installation process.
Note that with CentOS 4.2, the
installer will ask to switch to Disk #2, Disk #3, Disk #4, Disk #1, and
then back to Disk #4.
Graphical
Interface (X) Configuration
With most RHEL4 distributions (not the case with CentOS 4.2), when the
installation is complete, the installer will attempt to detect your
video hardware. Ensure that the installer has detected and selected the
correct video hardware (graphics card and monitor) to properly use the
X Windows server. You will continue with the X configuration in the
next serveral screens.
Congratulations
And that's it. You have successfully installed CentOS Enterprise Linux
on the first node (linux1). The installer will eject the CD from the
CD-ROM drive. Take out the CD and click [Exit] to reboot the system.
When the system boots into Linux for
the first time, it will prompt you with another Welcome screen. The
following wizard allows you to configure the date and time, add any
additional users, testing the sound card, and to install any additional
CDs. The only screen I care about is the time and date (and if you are
using CentOS 4.x, the monitor/display settings). As for the others,
simply run through them as there is nothing additional that needs to be
installed (at this point anyways!). If everything was successful, you
should now be presented with the login screen.
Perform the
same installation on the second node
After completing the Linux installation on the first node, repeat the
above steps for the second node (linux2). When configuring the machine
name and networking, ensure to configure the proper values. For my
installation, this is what I configured for linux2:
First, make sure that each of the
network devices are checked to [Active on boot]. The installer will
choose not to activate eth1.
Second, [Edit] both eth0 and eth1 as
follows:
eth0:
- Check off the option to [Configure using DHCP]
- Leave the [Activate on boot] checked
- IP Address: 192.168.1.101
- Netmask: 255.255.255.0
eth1:
- Check off the option to [Configure using DHCP]
- Leave the [Activate on boot] checked
- IP Address: 192.168.2.101
- Netmask: 255.255.255.0
Continue by setting your hostname
manually. I used "linux2" for the second node. Finish this dialog off
by supplying your gateway and DNS servers.
7.
Network Configuration
Perform the following network
configuration on all nodes in the cluster!
Note:
Although we configured several of the network settings during the Linux
installation, it is important to not skip this
section as it contains critical steps that are required for the RAC
environment.
Introduction to
Network Settings
During the Linux O/S install you
already configured the IP address and host name for each of the nodes.
You now need to configure the /etc/hosts file as
well as adjust several of the network settings for the interconnect.
Each node should have one static IP
address for the public network and one static IP address for the
private cluster interconnect. The private interconnect should only be
used by Oracle to transfer Cluster Manager and Cache Fusion related
data. Although it is possible to use the public network for the
interconnect, this is not recommended as it may cause degraded database
performance (reducing the amount of bandwidth for Cache Fusion and
Cluster Manager traffic). For a production RAC implementation, the
interconnect should be at least gigabit or more and only be used by
Oracle.
Configuring Public and
Private Network
In our two-node example, you need to configure the network on both
nodes for access to the public network as well as their private
interconnect.
The easiest way to configure network
settings in RHEL4 is with the Network Configuration program. This
application can be started from the command-line as the root user
account as follows:
# su - # /usr/bin/system-config-network &
Do not use DHCP naming for the
public IP address or the interconnects; you need static IP addresses!
Using the Network Configuration
application, you need to configure both NIC devices as well as the /etc/hosts
file. Both of these tasks can be completed using the Network
Configuration GUI. Notice that the /etc/hosts
settings are the same for both nodes.
Our example
configuration will use the following settings:
| Server 1 (linux1) |
| Device |
IP Address |
Subnet |
Gateway |
Purpose |
| eth0 |
|