Monday, January 1, 2018

Oracle Database 12c Release 1 (12.1) RAC On Oracle Linux 6 Using VirtualBox

This article describes the installation of Oracle Database 12c release 1(64 bit)  RAC on Linux (Oracle Linux 6.9 64-bit) using VirtualBox (5.1)
Introduction
System Requirements
Download Software
VirtualBox Installation
Virtual Machine Setup
Guest Operating System Installation
Check Internet Access
Oracle Installation Prerequisites
Automatic Setup
Manual Setup
Additional Setup
Install Guest Additions
Create Shared Disks
Clone the Virtual Machine
Install the Grid Infrastructure
Install the Database Software
Create a Database
Check the Status of the RAC

Introduction
If you want to get through all steps of the Oracle RAC installation and your laptop or desktop computer has 8 GB or more of RAM (16 GB is recommended), then this is entirely feasible using Oracle VirtualBox as demonstrated in this article. You can get a running RAC system which can host a small test database. The created system is not, and should not be considered, a production-ready system. It's simply to allow you to get used to installing and using RAC and test various administration procedures.
This article uses the 64-bit versions of Oracle Linux, version 6.4, and Oracle 12c Release 1, version 12.1.0.1.0. Using VirtualBox you can run multiple Virtual Machines (VMs) on a single server, allowing you to run both RAC nodes on a single machine. In addition, it allows you to set up shared virtual disks. The finished system includes two guest operating systems, two sets of Oracle Grid Infrastructure (Clusterware + ASM) and two Database instances all on a single server.

System Requirements
  • 12 GB of RAM, 16GB recommended
  • 100 GB of free space on the hard disk + staging area for Oracle Linux and RAC downloads, about 9 GB;
  • This procedure was tested on 64-bit Windows 10
Download the following software
Depending on your version of VirtualBox and Oracle Linux, there may be some slight variation in how the screen shots look.

VirtualBox Installation
First, install the VirtualBox software. On RHEL and its clones you do this with the following type of command as the root user.
# rpm -Uvh VirtualBox-4.3-4.3.16_95972_el6-1.x86_64.rpm
The package name will vary depending on the host distribution you are using. If you are doing this on windows system then, you have to just execute *.exe file. Once complete, VirtualBox is started from the menu.

Virtual Machine Setup
We must define two virtual RAC nodes. We can save time by defining one VM, then cloning it when it is installed.
Start VirtualBox and click the "New" button on the toolbar. Enter the
Name "rac121", Type "Linux" and Version "Oracle (64 bit)", and then click the "Next" button:

Enter "4096" as the base memory size, then click the "Next" button. Use more memory if you have enough physical memory on your machine as it will make the process much quicker!

Accept the default option to create a new virtual hard disk by clicking the "Create" button.

Accept the default VDI type and click the "Next" button on the Create Virtual Hard Drive screen:

Accept the default "Dynamically allocated" option by clicking the "Next" button:

Accept the default file name and set size to "40G", then click the "Create" button:

The "rac121" VM will appear on the left-hand pane. Select your machine and Click on the "Settings" link on the right side:

Make Network changes

Adapter 1- Host-only Adapter

Adapter 2- Internal Network

Adapter 3 – Bridged Adapter - This adapter is enabled for internet access in VM. We will disable this adapter, after yum update

System Changes
Move "Hard Disk" to the top of the boot order and uncheck the "Floppy" option, then click the "OK.


The virtual machine is now configured so we can start the guest operating system installation.

Guest Operating System Installation



With the new VM highlighted, click the "Start" button on the toolbar. On the "Select start-up disk" screen, choose the relevant Oracle Linux ISO image and click the "Start".





The resulting console window will contain the Oracle Linux boot screen.





Do not perform the media test. Choose "Skip" button:




On the "Welcome" screen, click the "Next" button.







Select the appropriate language, then click the "Next" button.

Select the relevant keyboard setting, then click the "Next" button.

Select the storage option "Basic Storage Devices" for the installation, then click the "Next" button.

Click on Yes, discard any data and proceed to next.

Enter a fully qualified host name - rac121.localdomain, then click the "Configure Network" button.

Highlight the relevant connection and click the "Edit" button for each connection.



Note : IPv6 Settings are “Ignored” for each connections

Select the relevant time zone by clicking on your nearest city on the map. Click on the "Next" button to proceed.

Enter a root password for the server, then click the "Next" button to proceed.

Check the partitioning type you require. If you want to amend the default partitioning layout, check the "Review and modify partitioning layout" option. Click the "Next" button.

The installer will list the default partitioning scheme for your size disk. Amend them as required and click the "Next" button, then the "Format" and "Write changes to disk" buttons on the following dialogs.




Accept the boot loader settings by clicking the "Next" button.
                          

Accept the "Basic Server" installation and check the "Customize now" option, then click the "Next" button.

The "Package Group Selection" screen allows you to select the required package groups, and individual packages within the details section. When you've made your selection, click the "Next" button. Make sure to install below packages :-
following package groups installed:
·       Base System > Base
·       Base System > Compatibility libraries
·       Base System > Hardware monitoring utilities
·       Base System > Large Systems Performance
·       Base System > Network file system client
·       Base System > Performance Tools
·       Base System > Perl Support
·       Servers> Server Platform
·       Servers> System administration tools
·       Desktops> Desktop
·       Desktops> Desktop Platform
·       Desktops> Fonts
·       Desktops> General Purpose Desktop
·       Desktops> Graphical Administration Tools
·       Desktops> Input Methods
·       Desktops> X Window System
·       Applications> Internet Browser
·       Development> Additional Development
·       Development> Development Tools

Wait for the installation to complete.

Click on Reboot.


On the "Welcome" screen, click the "Forward" button


Accept the license agreement and click the "Forward" button.

Pick the desired ULN Registration option, then click the "Forward" button. In this case we will pick the register later option. Click the "No thanks, I'll connect later." button.

Finish the software updates setup by clicking the "Forward" button

Enter the details for system user, then click the "Forward" button.

Adjust the Date and Time settings if necessary, and click the "Forward" Button.

uncheck the default settingon the Kdump screen by clicking the "Finish" button

Next, you are presented with the login screen. Log in by clicking on the user on the login screen.

Firewall

To turn off the firewall do the following as the "root" user.

# service iptables stop
# chkconfig iptables off

So your machine is ready. You are free to change the IP addresses to suit your network, but remember to stay consistent with those adjustments throughout the rest of the article.
Let’s perform some check son machine.

Check Internet Access
We will need Internet access because additional packages will be installed online. Open terminal (Main menu | Applications | System Tools | Terminal) and try to ping any Internet site, for example: 
host google.com

If ping doesn't work, troubleshoot the problem using 'ifconfig' command and making changes in Network Connections (Linux desktop Main menu | System | Preferences | Network Connections). If you made changes in Network Connections, restart interface by rebooting VM or running these two commands: 
# ifdown eth0
# ifup eth0
Then check the ping again. 

Oracle Installation Prerequisites
Perform either the Automatic Setup or the Manual Setup to complete the basic prerequisites. The Additional Setup is required for all installations.

Automatic Setup
If you plan to use the "oracle-rdbms-server-12cR1-preinstall" package to perform all your prerequisite setup, issue the following command.
# yum install oracle-rdbms-server-12cR1-preinstall -y
It is probably worth doing a full update as well, but this is not strictly speaking necessary.
# yum update -y
# yum install oracleasm
# yum install oracleasm-support
# yum install gedit
Cleanup YUM repositories:
# yum clean all

Note :-
Now you have to disable Adapter 3 (Devices >> Network >> Network Settings) and delete eth2 (System >> Preferences >> Network Connections). This step was necessary in my lab as scan were not working without this.  


Manual Setup
If you have not used the "oracle-rdbms-server-12cR1-preinstall" package to perform all prerequisites, you will need to manually perform the following setup tasks.

Add or amend the following lines to the "/etc/sysctl.conf" file.
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500

Run the following command to change the current kernel parameters.

/sbin/sysctl -p

Add the following lines to the "/etc/security/limits.d/oracle-database-server-12cR2-preinstall.conf" file.
oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc    2047
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768

In addition to the basic OS installation, the following packages must be installed whilst logged in as the root user. This includes the 64-bit and 32-bit versions of some packages.
# From Public Yum or ULN
yum install binutils -y
yum install compat-libcap1 -y
yum install compat-libstdc++-33 -y
yum install compat-libstdc++-33.i686 -y
yum install gcc -y
yum install gcc-c++ -y
yum install glibc -y
yum install glibc.i686 -y
yum install glibc-devel -y
yum install glibc-devel.i686 -y
yum install ksh -y
yum install libgcc -y
yum install libgcc.i686 -y
yum install libstdc++ -y
yum install libstdc++.i686 -y
yum install libstdc++-devel -y
yum install libstdc++-devel.i686 -y
yum install libaio -y
yum install libaio.i686 -y
yum install libaio-devel -y
yum install libaio-devel.i686 -y
yum install libXext -y
yum install libXext.i686 -y
yum install libXtst -y
yum install libXtst.i686 -y
yum install libX11 -y
yum install libX11.i686 -y
yum install libXau -y
yum install libXau.i686 -y
yum install libxcb -y
yum install libxcb.i686 -y
yum install libXi -y
yum install libXi.i686 -y
yum install make -y
yum install sysstat -y
yum install unixODBC -y
yum install unixODBC-devel -y

Create the new groups and users.
groupadd -g 54321 oinstall
groupadd -g 54322 dba
groupadd -g 54323 oper
#groupadd -g 54324 backupdba
#groupadd -g 54325 dgdba
#groupadd -g 54326 kmdba
#groupadd -g 54327 asmdba
#groupadd -g 54328 asmoper
#groupadd -g 54329 asmadmin
useradd -u 54321 -g oinstall -G dba,oper oracle
Uncomment the extra groups you require.


Additional Setup
The following steps must be performed, whether you did the manual or automatic setup.
Perform the following steps whilst logged into the "rac121" virtual machine as the root user.

Set the password for the "oracle" user.
passwd oracle

Apart form the localhost address, the "/etc/hosts" file can be left blank, but I prefer to put the addresses in for reference.

[root@rac121 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
#127.0.0.1       localhost.localdomain   localhost
192.168.56.71 rac121.localdomain rac121
# Private
192.168.10.1    rac121-priv.localdomain   rac121-priv
192.168.10.2    rac122-priv.localdomain   rac122-priv
# Public
192.168.56.71    rac121.localdomain        rac121
192.168.56.72    rac122.localdomain        rac122
# Virtual
192.168.56.81    rac121-vip.localdomain    rac121-vip
192.168.56.82    rac122-vip.localdomain    rac122-vip
# SCAN
192.168.56.91    rac-scan.localdomain    rac-scan
192.168.56.92    rac-scan.localdomain    rac-scan
192.168.56.93    rac-scan.localdomain    rac-scan
 

With this in place and the DNS configured the SCAN address is being resolved to all three IP addresses.
[root@rac121 ~]# nslookup rac-scan
Server:         127.0.0.1
Address:        127.0.0.1#53
Name:   rac-scan.localdomain
Address: 192.168.56.93
Name:   rac-scan.localdomain
Address: 192.168.56.91
Name:   rac-scan.localdomain
Address: 192.168.56.92

Change the setting of SELinux to permissive by editing the "/etc/selinux/config" file, making sure the SELINUX flag is set as follows.
SELINUX=permissive

Either configure NTP, or make sure it is not configured so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes. In this case we will deconfigure NTP.
# service ntpd stop
Shutting down ntpd:                                        [  OK  ]
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid

If your RAC is going to be permanently connected to your main network and you want to use NTP, you must add the "-x" option into the following line in the "/etc/sysconfig/ntpd" file.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

Then restart NTP.
# service ntpd restart

Create the directories in which the Oracle software will be installed.
mkdir -p  /u01/app/12.1.0.2/grid
mkdir -p /u01/app/oracle/product/12.1.0.2/db_1
chown -R oracle:oinstall /u01
chmod -R 775 /u01/

Log in as the "oracle" user and add the following lines at the end of the "/home/oracle/.bash_profile" file.
# Oracle Settings
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=rac121.localdomain
export ORACLE_UNQNAME=racdb
export ORACLE_BASE=/u01/app/oracle
export GRID_HOME=/u01/app/12.1.0.2/grid
export DB_HOME=$ORACLE_BASE/product/12.1.0.2/db_1
export ORACLE_HOME=$DB_HOME
export ORACLE_SID=racdb1
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
alias grid_env='. /home/oracle/grid_env'
alias db_env='. /home/oracle/db_env'

Create a file called "/home/oracle/grid_env" with the following contents.
export ORACLE_SID=+ASM1
export ORACLE_HOME=$GRID_HOME
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

Create a file called "/home/oracle/db_env" with the following contents.
export ORACLE_SID=racdb1
export ORACLE_HOME=$DB_HOME
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

Let's do a reboot of the VM at this point to make sure all the changes have taken effect.
# shutdown -r now 


Install Guest Additions

Click on the "Devices > Install Guest Additions" menu option at the top of the VM screen.
            





Add the "oracle" user into the "vboxsf" group so it has access to shared drives.
# usermod -G vboxsf,dba oracle
# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf)
#

Create a shared folder (Devices > Shared Folders) on the virtual machine, pointing to the directory on the host where the Oracle software was unzipped. Check the "Auto-mount" and "Make Permanent" options before clicking the "OK" button.

The VM will need to be restarted for the guest additions to be used properly. The next section requires a shutdown so no additional restart is needed at this time. Once the VM is restarted, the shared folder called "/media/sf_12.1.0.2" will be accessible by the "oracle" user.

Create Shared Disks
Shut down the "rac121" virtual machine using the following command.

# shutdown -h now

On the host server, create 4 sharable virtual disks and associate them as virtual media using the following commands. You can pick a different location.

cd C:\Program Files\Oracle\VirtualBox\
# Create the disks and associate them with VirtualBox as virtual media.
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm1.vdi" --size 3072 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm2.vdi" --size 3072 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm3.vdi" --size 3072 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm4.vdi" --size 3072 --format VDI --variant Fixed

# Connect them to the VM.
VBoxManage storageattach rac121 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm1.vdi" --mtype shareable
VBoxManage storageattach rac121 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm2.vdi" --mtype shareable
VBoxManage storageattach rac121 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm3.vdi" --mtype shareable
VBoxManage storageattach rac121 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm4.vdi" --mtype shareable

# Make shareable.
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm1.vdi" --type shareable
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm2.vdi" --type shareable
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm3.vdi" --type shareable
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm4.vdi" --type shareable

Start the "rac121" virtual machine by clicking the "Start" button on the toolbar. When the server has started, log in as the root user so you can configure the shared disks. The current disks can be seen by issuing the following commands.
# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdc  sdd  sde
#

Use the "fdisk" command to partition the disks sdb to sde.

[root@rac121 dev]# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-652, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652):
Using default value 652

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#

In each case, the sequence of answers is "n", "p", "1", "Return", "Return" and "w".

Once all the disks are partitioned, the results can be seen by repeating the previous "ls" command.
# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdb1  sdc  sdc1  sdd  sdd1  sde  sde1
#

Configure your UDEV rules.

Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted. Create the file if it doesn't already exist.

options=-g

The SCSI ID of my disks are displayed below.
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sdb
1ATA_VBOX_HARDDISK_VB5a97a6d6-443d513c
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sdc
1ATA_VBOX_HARDDISK_VBebd1447e-07dfd77f
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sdd
1ATA_VBOX_HARDDISK_VB6d4e6597-6510505c
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sde
1ATA_VBOX_HARDDISK_VB15e37173-5a00f121

Using these values, edit the "/etc/udev/rules.d/99-oracle-asmdevices.rules" file adding the following 4 entries. All parameters for a single entry must be on the same line.

KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB5a97a6d6-443d513c",NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBebd1447e-07dfd77f",NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB6d4e6597-6510505c",NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB15e37173-5a00f121",NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"

Load updated block device partition tables.

# /sbin/partprobe /dev/sdb1
# /sbin/partprobe /dev/sdc1
# /sbin/partprobe /dev/sdd1
# /sbin/partprobe /dev/sde1

Test the rules are working as expected.
# /sbin/udevadm test /block/sdb/sdb1

Reload the UDEV rules and start UDEV.
# /sbin/udevadm control --reload-rules
# /sbin/start_udev

The disks should now be visible and have the correct ownership using the following command. If they are not visible, your UDEV configuration is incorrect and must be fixed before you proceed.
[root@rac121 dev]# ls -al /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Dec 28 19:40 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 33 Dec 28 19:40 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Dec 28 19:40 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Dec 28 19:40 /dev/asm-disk4

The shared disks are now configured for the grid infrastructure.

Clone the Virtual Machine
Later versions of VirtualBox allow you to clone VMs, but these also attempt to clone the shared disks, which is not what we want. Instead we must manually clone the VM.

Shut down the "rac121" virtual machine using the following command.
# shutdown -h now

Manually clone the "rac121.vdi" disk using the following commands on the host server.
cd C:\Program Files\Oracle\VirtualBox\
C:\Program Files\Oracle\VirtualBox>VBoxManage clonehd "C:\My Drive\Virtual Disks\rac121.vdi" "C:\My Drive\Virtual Disks\rac122.vdi"

Create the "rac122" virtual machine in VirtualBox in the same way as you did for "rac121", with the exception of using an existing "rac122.vdi" virtual hard drive.

When the VM is created, attach the shared disks to this VM.
cd C:\Program Files\Oracle\VirtualBox\
VBoxManage storageattach rac122 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm1.vdi" --mtype shareable
VBoxManage storageattach rac122 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm2.vdi" --mtype shareable
VBoxManage storageattach rac122 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm3.vdi" --mtype shareable
VBoxManage storageattach rac122 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm4.vdi" --mtype shareable
Start the "rac122" virtual machine by clicking the "Start" button on the toolbar. Ignore any network errors during the startup.

Log in to the "rac122"  virtual machine as the "root" user so we can reconfigure the network settings to match the following.

Amend the hostname in the "/etc/sysconfig/network" file.
NETWORKING=yes
HOSTNAME=rac122.localdomain


Start "Network Connections" tool (Main menu | System | Preferences | Network Connections). Edit eth0 and eth1 interfaces and set in IPv4 addresses 192.168.56.72 and 192.168.10.2 correspondingly.



Now we need to change MAC address for all three interfaces. At the moment we have two VMs with the same set of MAC addresses. We can run one machine or another, but not both of them at the same time because MAC address must be unique. No changes will be made to rac1, we will pick up three new unused addresses and set them for eth0, eth1, and eth2 in rac2. The easiest way to do that is to change just last two characters of the address. We are going to change them to '00'. If the last two characters are already '00', then change to something else, '01', for example. Just make sure that these addresses don't collide with the MAC addresses of rac1. In running rac2 node, open "Network Connections" and edit MAC address in the "Wired" tab. The screenshot below shows where to set MAC address. Don't forget to change MAC addresses for all three interfaces. Please note that your setup will have a different set of MAC addresses because they are random-generated by VirtualBox.


Write down the new MAC addresses for all three interfaces. Save new settings pressing "Apply" button, then shutdown the machine. After shutdown, return to the VirtualBox Manager, select rac122 VM and edit "Network" settings. Make same changes to the MAC addresses. Don't forget to change MAC addresses for all three adapters.


Restart the virtual machines.
# shutdown -r now

Edit the "/home/oracle/.bash_profile" file on the "ol6-121-rac2" node to correct the ORACLE_SID and ORACLE_HOSTNAME values.
export ORACLE_SID=racdb1
export ORACLE_HOSTNAME=rac122.localdomain

Also, amend the ORACLE_SID setting in the "/home/oracle/db_env" and "/home/oracle/grid_env" files.

Restart the "rac122" virtual machine and start the "rac121" virtual machine. When both nodes have started, check they can both ping all the public and private IP addresses using the following commands.
ping -c 3 rac121
ping -c 3 rac121-priv
ping -c 3 rac122
ping -c 3 rac122-priv

Check the SCAN address is still being resolved properly on both nodes
[root@rac121 dev]# nslookup rac-scan
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   rac-scan.localdomain
Address: 192.168.56.93
Name:   rac-scan.localdomain
Address: 192.168.56.91
Name:   rac-scan.localdomain
Address: 192.168.56.92

[root@rac122 ~]# nslookup rac-scan
Server:         127.0.0.1
Address:        127.0.0.1#53
Name:   rac-scan.localdomain
Address: 192.168.56.93
Name:   rac-scan.localdomain
Address: 192.168.56.91
Name:   rac-scan.localdomain
Address: 192.168.56.92

Check the UDEV rules are working on both machines. In previous versions of OL6 the "/etc/udev/rules.d/99-oracle-asmdevices.rules" file copied between servers during the clone without any issues. For some reason, this doesn't seem to happen on my OL6 installations, so you may need to repeat the UDEV configuration on the second node if the output of the following command is not consistent on both nodes.
[root@rac122 ~]# ls -al /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Dec 28 20:36 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 33 Dec 28 20:36 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Dec 28 20:36 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Dec 28 20:34 /dev/asm-disk4

Install the Grid Infrastructure
Make sure both virtual machines are started. Install the following package from the Oracle grid media as the root user.
[root@rac122 rpm]# cd /media/sf_54/grid_sf/grid/rpm
[root@rac122 rpm]# rpm -Uvh cvuqdisk*
Login to "rac121" as the "oracle" user (better to avoid using "su -" command, might have problems) and start the Oracle installer.
$ cd media/sf_54/grid_sf/grid/
$ ./runInstaller
Select the "Install and Configure Oracle Grid Infrastructure for a Cluster" option, then click the "Next" button.

Accept the "Configure a Standard cluster" option by clicking the "Next" button.

Select the "Typical Installation" option, then click the "Next" button.

On the "Specify Cluster Configuration" screen, enter the correct SCAN Name and click the "Add" button.

Enter the details of the second node in the cluster, then click the "OK" button.

Click the "SSH Connectivity..." button and enter the password for the "oracle" user. Click the "Setup" button to configure SSH connectivity, and the "Test" button to test it once it is complete. Once the test is complete, click the "Next" button.





On the "Specify Network Interface Usage" screen check the Public and Private networks are specified correctly. Press the "Next" button.

Enter "/u01/app/12.1.0.2/grid" as the software location and "Automatic Storage Manager" as the cluster registry storage type. Enter the ASM password, select "dba" as the group and click "Next".

Set the redundancy to "External", click the "Change Discovery Path" button and set the path to "/dev/asm*". Return the main screen and select all 4 disks and click the "Next" button.


Accept the default inventory directory by clicking the "Next" button.

If you want the root scripts to run automatically, enter the relevant credentials. I prefer to run them manually. Click the "Next" button.

Wait while the prerequisite checks complete. If you have any issues use the "Fix & Check Again" button. Once possible fixes are complete, check the "Ignore All" checkbox and click the "Next" button.It is likely the "SwapMemory" and "Device Checks for ASM" tests will fail for this type of installation. This is OK.

If you are fine with the summary information, click the "Install" button.

Wait while the installation takes place and When prompted, run the configuration scripts on each node.

Wait for the configuration assistants to complete. If any of the configuration steps fail you should check the specified log to see if the error is a show-stopper or not.

The grid infrastructure installation is now complete. We can check the status of the installation using the following commands.
[oracle@rac122 ~]$ grid_env
[oracle@rac122 ~]$ crsctl stat res -t

Install the Database Software
Make sure both virtual machines are started, then login to "rac121" as the oracle user and start the Oracle installer. Check that all services are up using "crsctl stat res -t", as described
[oracle@rac122 ~]$ cd /media/sf_54/db_sf/database
[oracle@rac122 database]$ ./runInstaller

Uncheck the security updates checkbox and click the "Next" button and "Yes" on the subsequent warning dialog.

Select the "Install database software only" option, then click the "Next" button

Accept the "Oracle Real Application Clusters database installation" option by clicking the "Next" button.

Make sure both nodes are selected, then click the "Next" button.

Select the "Language" and "Enterprise Edition" option, then click the "Next" button.

Enter "/u01/app/oracle" as the Oracle base and "/u01/app/oracle/product/12.1.0.2/db_1" as the software location, then click the "Next" button.

Select the desired operating system groups, then click the "Next" button.

Wait for the prerequisite check to complete. If there are any problems either click the "Fix & Check Again" button, or check the "Ignore All" checkbox and click the "Next" button.

If you are happy with the summary information, click the "Install" button.

Wait while the installation takes place and When prompted, run the configuration script on each node. When the scripts have been run on each node, click the "OK" button

Click the "Close" button to exit the installer.


Create a Database
Make sure both virtual machines are started, then login to "rac121" as the oracle user and start the Database Creation Asistant (DBCA).
$dbca


Check "Create a database with default configuration"; type in "Global Dtabase Name" the desired name; we have selected AL32UTF8 character set, make your choice as needed; also supply a password for SYS. We are going to create a reqular (pre-12c type) database, the "Create As Container Database is unchecked. Press Next button:

Wait for the prerequisite check to complete. The only reported problem was all familiar "Swap Size". Check "Ignore All" checkbox and click the "Next" button. Press "Finish" button in the Summary screen:







The RAC database creation is now complete.

Check the Status of the RAC

[oracle@rac121 bin]$ srvctl config database -d racdb
Database unique name: racdb
Database name: racdb
Oracle home: /u01/app/oracle/product/12.1.0.2/db_1
Oracle user: oracle
Spfile: +DATA/RACDB/PARAMETERFILE/spfile.293.963776861
Password file: +DATA/RACDB/PASSWORD/pwdracdb.276.963776547
Domain: localdomain
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: DATA
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: oper
Database instances: racdb1,racdb2
Configured nodes: rac121,rac122
Database is administrator managed
[oracle@rac121 bin]$ srvctl status database -d racdb
Instance racdb1 is running on node rac121
Instance racdb2 is running on node rac122
SQL> SELECT inst_name FROM v$active_instances;

INST_NAME
--------------------------------------------------------------------------------
rac121.localdomain:racdb1
rac122.localdomain:racdb2


Here your RAC database "racdb" is ready with two instances "racdb1"  and "racdb2" on nodes "rac121" and "rac122".
If you see any issues while doing above tasks, feel free to post them in comment section. 


No comments:

Post a Comment