1. Hardware Requirements (For each node RAC1 and Rac2)
- Minimum Intel Pentium 4 Processor at 2.80GHz
- Minimum 1GB DDR SDRAM (at 333MHz)
- Minimum 40GB 7200 RPM Internal Hard Drive
- Integrated Intel 3D AGP Graphics
- Integrated 10/100 Ethernet*
* Each Linux server should contain two Ethernet LAN Card that will be used to connect to the public network and for private interconnect. Each node connected through HUB/SWITCH. We can use CAT5 crossover cable only for testing or educational purpose.
2. Install and Configure the RHEL4 (On both node RAC1 and RAC2)
Start RHEL installation on first node (RAC1) after completing the Linux installation on the first node; perform the same RHEL installation on the second node (RAC2) and the different IP addresses where appropriate.
Insert Linux OS CD and Restart The Server.
1.The first screen is the Linux boot screen. Hit ENTER to install in Graphic mode
2. Skip the media test and start the installation.
3. Welcome to enterprise Linux: Click on Next.
4. Language Selection: .
6. Installation Type: Custom.
7. Disk Partitioning Setup: Manually partition with Disk Druid.
Warning: Click on Yes to initialize each of the device – sda, sdb, sdc, sdd, and sde.
8.Disk Setup: Allocate disk space on sda drive by double-clicking on /dev/sda free space for the mount points (/ and /u01) and swap space.
Add Partition:
Mount Point: /
File System Type: ext3
Size: 8 GB
File System Type: Swap
Size: 512 MB or Double amount of RAM
Mount Point: /u01
File System Type: ext3
Size: Allocate all Available Space
9. Boot Loader Configuration: Select only the default /dev/sda1 and leave the rest unchecked.
10. Network Configuration: before starting the operating system installation, made sure to install LAN Card in each of the Linux machines.
Network Devices
Select and edit eth0
De-select Configure Using DHCP.
Select Activate on boot.
IP Address: Enter "192.168.2.131."
Netmask: Enter "255.255.255.0."
Select and edit eth1
De-select Configure Using DHCP.
Select Activate on boot.
IP Address: Enter "10.10.10.31."
Netmask: Enter "255.255.255.0."
Hostname
Select manually and enter "rac1.xyz.com."
Miscellaneous Settings
Gateway: Enter "192.168.2.1."
Primary DNS:
Secondary DNS:
11. Firewall On this screen, make sure to select [No firewall] and click [Next] to continue.
12. Additional Language Support:
13. Set Root Password:
14. Package Group Selection:
Select X Window System.
Select GNOME Desktop Environment.
Select Editors. -> Click on Details and select your preferred text editor.
Select Graphical Internet.
Select Text-based Internet.
Select Office/Productivity.
Select Sound and Video.
Select Graphics.
Select Server Configuration Tools.
Select FTP Server.
Select Legacy Network Server. -> Click on Details. Select rsh-server and telnet-server.
Select Development Tools. l.
Select Legacy Software Development.
Select Administration Tools.
Select System Tools. ->Click on Details. Select the following packages in addition to the default selected packages.
Select ocfs-2-2.6.9-42.0.0.0.1EL (driver for UP kernel), or select ocfs-2-2.6.9-42.0.0.0.1ELsmp (driver for SMP kernel).
Select ocfs2-tools.
Select ocfs2console.
Select oracle oracleasm-2.6.9-42.0.0.0.1EL (driver for UP kernel) or select oracleasm-2.6.9-42.0.0.0.1ELsmp (driver for SMP kernel).
Select sysstat.
Select Printing Support.
15. About to Install: Click on Next.
16. Required Install Media: Click on Continue.
17. At the end of the installation: Reboot the machine
Perform the same installation on the second node (RAC2):
After completing the Linux installation on the first node (RAC1), repeat the above steps for the second node (RAC2). When configuring the machine name and networking, ensure to configure the proper values.
Use following value to network Configuration on Second node (RAC2) ;
Network Devices
Select and edit eth0
De-select Configure Using DHCP.
Select Activate on boot.
IP Address: Enter "192.168.2.132."
Netmask: Enter "255.255.255.0."
Select and edit eth1
De-select Configure Using DHCP.
Select Activate on boot.
IP Address: Enter "10.10.10.32"
Netmask: Enter "255.255.255.0."
Hostname
Select manually and enter "rac2.xyz.com."
Miscellaneous Settings
Gateway: Enter "192.168.2.1."
Primary DNS:
Secondary DNS:
Modify the /etc/hosts file (On Both Node)
Note that the /etc/hosts settings are the same for both nodes.
# more /etc/hosts
127.0.0.1 localhost
192.168.2.131 rac1.xyz.com rac1
192.168.2.31 rac1-vip.xyz.com rac1-vip
10.10.10.31 rac1-priv.xyz.com rac1-priv
192.168.2.132 rac2.xyz.com rac2
192.168.2.32 rac2-vip.xyz.com rac2-vip
10.10.10.32 rac2-priv.xyz.com rac2-priv
#. The public virtual IP addresses will be configured automatically by Oracle when you run the Oracle Universal Installer, which starts Oracle's Virtual Internet Protocol Configuration Assistant (VIPCA).
# All virtual IP addresses will be activated when the srvctl start nodeapps -n
Configure the kernel parameters (On Both Node)
Use a text editor and add the lines listed below to /etc/sysctl.conf. To make the changes effective immediately, execute /sbin/sysctl –p.
# more /etc/sysctl.conf
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 1048576
net.core.rmem_max = 1048576
net.core.wmem_default = 262144
net.core.wmem_max = 262144
Configure the hangcheck timer kernel module (On Both Node)
The hangcheck timer kernel module monitors the system's health and restarts a failing RAC node. It uses two parameters, hangcheck_tick (defines the system checks frequency) and hangcheck_margin (defines the maximum hang delay before a RAC nodeis reset), to determine if a node is failing.
Add the following line in /etc/modprobe.conf to set the hangcheck kernel module parameters.
/etc/modprobe.conf
options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
To load the module immediately, execute "modprobe -v hangcheck-timer".
Check Required RPMs ( On Both Node)
The following packages (or higher versions) must be installed:
make-3.80-5
glibc-2.3.4-2.9
glibc-devel-2.3.4-2.9
glibc-headers-2.3.4-2.9
glibc-kernheaders-2.4-9.1.87
cpp-3.4.3-22.1
compat-db-4.1.25-9
compat-gcc-32-3.2.3-47.3
compat-gcc-32-c++-3.2.3-47.3
compat-libstdc++-33-3.2.3-47.3
compat-libstdc++-296-2.96-132.7.2
openmotif-2.2.3-9.RHEL4.1
setarch-1.6-1
Install Enterprise Linux software packages. The following additional packages are required for
Oracle software installation. If you have installed the 64-bit version of Enterprise Linux, the installer should have already installed these packages.
libaio-0.3.105-2.i386.rpm
openmotif21-2.1.30-11.RHEL4.6.i386.rpm
Setting the Correct Date and Time ( On Both Node)
During the installation of Oracle Clusterware and Database Software the Oracle Universal Installer (OUI) first installs the software to the local node running the installer (i.e. RAC1) and then copied remotely to all of the remaining nodes in the cluster (i.e. RAC2).
After archived and copied over remote node (i.e RAC2) , the OUI execute the UNIX "tar" command on each remote node to extract the files.
Note: Before starting any of the above noted installations, ensure that each member node of the cluster is set as closely as possible to the same date and time.
Note: Oracle strongly recommends using the Network Time Protocol feature of most operating systems for this purpose, with all nodes using the same reference Network Time Protocol server.
We set manually the date and time for the nodes in the cluster, ensure that the date and time of the node you are performing the software installations from (RAC1) is less than all other nodes in the cluster (RAC2). I generally use a 20 second difference as shown in the following example:
Setting the date and time from RAC1:
# date -s "9/13/2006 23:00:00"
Setting the date and time from RAC2:
# date -s "9/13/2006 23:00:20"
Create the oracle user ( On Both Node)
Log in as a root user and execute following steps on both node:
# groupadd oinstall
# groupadd dba
# mkdir -p /export/home/oracle /ocfs
# useradd -d /export/home/oracle -g oinstall -G dba -s /bin/ksh oracle
# chown oracle:dba /export/home/oracle /u01
# passwd oracle
New Password:
Re-enter new Password:
passwd: password successfully changed for oracle
Create the oracle user environment file (On Rac1 Node)
Log in as a orcale user and create .profile file.
/export/home/oracle/.profile
export PS1="`/bin/hostname -s`-> "
export EDITOR=vi
export ORACLE_SID=PROD1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1
export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:/bin:
/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin
umask 022
Create the oracle user environment file (On Rac2 Node)
Log in as a orcale user and create .profile file.
/export/home/oracle/.profile
export PS1="`/bin/hostname -s`-> "
export EDITOR=vi
export ORACLE_SID=PROD2
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1
export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:/bin: /usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin
umask 022
Create the directory structure (On Both Node)
Log in as a oracle user, execute on RAC1
rac1-> mkdir -p $ORACLE_BASE/admin
rac1-> mkdir -p $ORACLE_HOME
rac1-> mkdir -p $ORA_CRS_HOME
rac1-> mkdir -p /u01/oradata/PROD
Log in as a oracle user, execute on RAC2
rac2-> mkdir -p $ORACLE_BASE/admin
rac2-> mkdir -p $ORACLE_HOME
rac2-> mkdir -p $ORA_CRS_HOME
rac2-> mkdir -p /u01/oradata/PROD
Increase the shell limits for the Oracle user (On Both Node)
Use a text editor and add the lines listed below to /etc/security/limits.conf, /etc/pam.d/login, and /etc/profile.
/etc/security/limits.conf
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
/etc/pam.d/login
session required /lib/security/pam_limits.so
/etc/profile
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
Establish user equivalence with SSH (On Both Node)
During the Cluster Ready Services (CRS) and RAC installation, the Oracle Universal Installer (OUI) has to be able to copy the software as oracle to all RAC nodes without being prompted for a password. In Oracle 10g, this can be accomplished using ssh instead of rsh.
To establish user equivalence, generate the user's public and private keys as the oracle user on both nodes. Perform the following tasks on both nodes.
On RAC1,
rac1-> mkdir ~/.ssh
rac1-> chmod 700 ~/.ssh
rac1-> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/export/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /export/home/oracle/.ssh/id_rsa.
Your public key has been saved in /export/home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
87:54:4f:92:ba:ed:7b:51:5d:1d:59:5b:f9:44:da:b6 oracle@rac1.xyz.com
rac1-> ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/export/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /export/home/oracle/.ssh/id_dsa.
Your public key has been saved in /export/home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
31:76:96:e6:fc:b7:25:04:fd:70:42:04:1f:fc:9a:26 oracle@rac1.xyz.com
On RAC2,
rac2-> mkdir ~/.ssh
rac2-> chmod 700 ~/.ssh
rac2-> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/export/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Install Oracle RAC 10g on Oracle Enterprise Linux Using VMware Server http://www.oracle.com/technology/pub/articles/chan-ubl-vmware.html?...
20 of 46 4/19/2007 12:45 PM
Enter same passphrase again:
Your identification has been saved in /export/home/oracle/.ssh/id_rsa.
Your public key has been saved in /export/home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
29:5a:35:ac:0a:03:2c:38:22:3c:95:5d:68:aa:56:66 oracle@rac2.xyz.com
rac2-> ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/export/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /export/home/oracle/.ssh/id_dsa.
Your public key has been saved in /export/home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
4c:b2:5a:8d:56:0f:dc:7b:bc:e0:cd:3b:8e:b9:5c:7c oracle@rac2.xyz.com
On RAC1,
rac1-> cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
rac1-> cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
rac1-> ssh rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
The authenticity of host 'rac2 (192.168.2.132)' can't be established.
RSA key fingerprint is 63:d3:52:d4:4d:e2:cb:ac:8d:4a:66:9f:f1:ab:28:1f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'rac2,192.168.2.132' (RSA) to the list of known hosts.
oracle@rac2's password:
rac1-> ssh rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
oracle@rac2's password:
rac1-> scp ~/.ssh/authorized_keys rac2:~/.ssh/authorized_keys
oracle@rac2's password:
authorized_keys 100% 1716 1.7KB/s 00:00
Test the connection on each node. Verify that you are not prompted for password when you run the following the second time.
ssh rac1 date
ssh rac2 date
ssh rac1-priv date
ssh rac2-priv date
ssh rac1.mycorpdomain.com date
ssh rac2.mycorpdomain.com date
ssh rac1-priv.mycorpdomain.com date
ssh rac2-priv.mycorpdomain.com date
Now your node is Configured...........To be comtinued......
