Tuesday, January 30, 2018

Recovery Scenarios | Loss Of SPFILE

SPFILE is a small but very important file and if we lose the spfile, we cannot start the database even if all the other database files are present and intact.You might need to restore the spfile for one of several reasons:
• You accidentally deleted your server parameter file.
• You want to view an old copy of the spfile.
• You can’t start your instance with the current spfile.

If you’re using an spfile, you can have it automatically backed up for you by enabling the autobackup of the control file.By default, automatic control file backups are disabled. Even when the auto backup feature is disabled, RMAN will back up the current control file and the server parameter file whenever any backup command includes data file 1 from the data files that belong to the target database. In an Oracle database, data file 1 is always part of the system tablespace, which contains the data dictionary. You can configure RMAN to automatically back up the control file
following every backup and any database structural change by using the configure command.

When does the SPFILE gat backed up with AUTOBACKUP tuned on?
Once you configure automatic control file backup, RMAN will automatically back up your target database control file, as well as the current server parameter file, when any of the following events occurs:
• Successful completion of either a backup or the copy command
• After a create catalog command from the RMAN prompt is successfully completed
• Any structural changes to the database modify the contents of the control file

After a backup or copy command completes and the recovery catalog—if you are using one—is successfully updated, RMAN will then back up the control file to its own backup piece. In addition, any changes to the physical structure of your database, even if they are made through SQL*Plus, will trigger a control file auto backup. (For example, the following actions will trigger an auto backup of the control file: adding a tablespace or data file, dropping a data file, placing a tablespace offline or online, adding an online redo log, and renaming a data file.)

When automatic backup is triggered by a structural change, an Oracle server process (not an RMAN process) will automatically create the auto backup of your control file.

Check Autobackup Status 






Enable Autobackup of controlfile











Where does the SPFILE autobackup goes?
You’ve just enabled the auto backup of the control file feature, but you don’t know where the files are physically being written. You want to ensure that these critical backups are being written to a location you know about so that you can maintain and monitor that location.

1. If you have enabled a fast area as well as the auto backup of the control file, then RMAN will write the backup to the directory defined for the fast area. By default, RMAN creates these files as Oracle managed files.

2. If you haven’t enabled a fast recovery area, RMAN will create the auto backups in an operating system–specific location ($ORACLE_HOME/dbs on Unix and %ORACLE_HOME%\database on Windows).

3. You can also configure the auto backup to back up the control file to an automatic storage management (ASM) disk group, as shown in the following example:
RMAN> configure controlfile autobackup format for device type disk to '+DATA/%F';

4. You can override where RMAN will write the auto backup control file and its name using the configure command.
RMAN> CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '/u01/12.1.0.2/backup/%F';

Note : The format variable %F yields a unique combination of the database ID, day, month, year, and sequence.

5. To set the directory and file format back to the default value, run this command:
RMAN> configure controlfile autobackup format for device type disk clear;

6. To list backup of spfile use command  : "list backup of spfile;"

Recovery scenarios involving loss of SPFILE

Case 1 : Using a Recovery Catalog
If you’re using a recovery catalog, then restoring the spfile is fairly straightforward.

$ rman target / catalog rman/xxxx@racdb
RMAN> startup nomount;
RMAN> restore spfile;
RMAN> startup force; # startup using restored spfile

If you receive an error such as this when running the restore command:
RMAN-20001: target database not found in recovery catalog

Then first set the DBID
RMAN> set dbid 961393455;

Case 2 : Not Using a Recovery Catalog, RMAN Auto Backup in Default Location
In this case we assume that auto backups of the spfile are going to the default location.
The default location depends on your operating system. For Linux/Unix, the default location is ORACLE_HOME/dbs.On Windows systems, it’s usually ORACLE_HOME\database.

$ rman target /
RMAN> startup force nomount; # start instance for retrieval of spfile
RMAN> set dbid 961393455;
RMAN> restore spfile from autobackup;
RMAN> startup force; # startup using restored spfile

Case 3 : Not Using a Recovery Catalog, RMAN Auto Backup Not in Default Location

If you’re either using an FRA or have the auto backup of your control file configured to a nondefault location, then the spfile will not be backed up to what Oracle calls the default location. In these situations, you have to specifically tell RMAN where to retrieve the backup from.

If you’re using an FRA, your spfiles will be backed up in an auto backup directory in the FRA. You’ll have to find that directory and backup piece name before you can restore your spfile.You’ll also need to know your database identifier before you can proceed.

$ rman target /
RMAN> set dbid 961393455;
RMAN> startup force nomount; # start instance for retrieval of spfile
RMAN> restore spfile from '/u01/12.1.0.2/backup/c-961393455-20180130-01';
RMAN> startup force; # startup using restored spfile

Suppose you are not sure which backup file contains the SPFILE backup. So to work around this, we tried to fool RMAN by creating the directory structure when using a FRA. We create the directory structure RACDB/autobackup/2018_01_30 under the to level location /u01/12.1.0.2/backup/ and copy the backup pieces to this location.

$ cd /u01/12.1.0.2/backup/
$mkdir -p RACDB/autobackup/2018_01_30
$ mv /u01/12.1.0.2/backup/o1* /u01/12.1.0.2/backup/RACDB/autobackup/2018_01_30

Now we can restore spfile using command :-
restore spfile from autobackup db_recovery_file_dest='/u01/12.1.0.2/backup/' db_name='RACDB';

Monday, January 29, 2018

ORA-1114 For An Unknown File Id

Most of the time DBAs get issues on ORA-1114 but the file id mentioned in error message becomes an unknown file id. Error message looks like :-
ORA-01114: IO error writing block to file 201 (block # 458754)
SVR4 Error: 28: No space left on device
What Oracle Doc says on this error :-
Cause:
The device on which the file resides is probably offline. If the file is a temporary file, then it is also possible that the device has run out of space. This could happen because disk space of temporary files is not necessarily allocated at file creation time.
Action: Restore access to the device or remove unnecessary files to free up space.

But in my case file id reported by error is unknown file id. First we'll check "DBA_DATA_FILES" to find out if issue is with any datafile.
SYS@racdb1> select file_id, RELATIVE_FNO, file_name from dba_data_files where file_id='201';
no rows selected
SYS@racdb1> select file_id, RELATIVE_FNO, file_name from dba_data_files where RELATIVE_FNO=201;
no rows selected

This error is being reported on a temp file. For TEMPFILES, the file numbering starts after the value of db_files. Run the following query to get the file name.
select file#, name from v$tempfile where file#=<file id reported in ora-1114 error >-<value of db_files>;

In this case :-
SYS@racdb1> show parameter db_files
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_files                             integer     200
SYS@racdb1> select file#, name from v$tempfile where file#=(201-200);
FILE# NAME
---------- --------------------------------------------------
1 +DATA/RACDB/TEMPFILE/temp.287.963776695
So i got issue with my tempfile 1. We need to check this tempfile and take actions accordingly.

Monday, January 22, 2018

How to failover a service during instance shutdown Oracle RAC12c

We can use srvctl to failover the running services to another instance, before shutting down the instance. In this demonstration we'll use "-failover" option to move services to another node before shutting down the node.

Step1 : Check the status of database
[oracle@rac121 ~]$ srvctl status database -d racdb
Instance racdb1 is running on node rac121
Instance racdb2 is running on node rac122

Step2 : Check status of service
[oracle@rac121 ~]$ srvctl status service -d racdb
Service temp_serv is running on instance(s) racdb1

Step 3: Stop the instance but use "-failover" option
[oracle@rac121 ~]$ srvctl stop instance -d racdb -i racdb1 -failover

Step 4: Check status of service, now my service is running on another node
[oracle@rac121 ~]$ srvctl status service -d racdb
Service temp_serv is running on instance(s) racdb2

Step 5: Check database status
[oracle@rac121 ~]$  srvctl status database -d racdb
Instance racdb1 is not running on node rac121
Instance racdb2 is running on node rac122

So without "-failover" option, you will not be able to stop instance if any service is running on that instance. In next post we'll see more details on service failover.

Sunday, January 21, 2018

Service Failover 11g/12c Oracle RAC

In this post we'll try to distinguish the rules of service failover to another instance.

If using sqlplus to shutdown instance
  • In pre-11.2.0.4 (11.2.0.2), the service  will failover to available instance
  • In 11.2.0.4 or above (12.1.*), the service will not failover but stop because SQLPLUS abort is detected by the database agent as PLANNED_OFFLINE with label "Instance Shutdown, Stable", thus clusterware does not failover the services. For a real crash, it would be OFFLINE with "Abnormal Termination".
If shutdown instance using srvctl
  • In pre-11.2, the service will failover to available instance.
  • In 11.2, the service will not failover but stop
  • In 11.2, you need to use "-f " option if you want service failover to available instance when stopping instance using srvctl.
  • You may relocate service to another instance before stopping instance through srvctl. e.g."srvctl relocate service -d racdb -s temp_serv -i racdb1 -t racdb2"
  • In 12c, you need to use "-failover" option to failover service to available instance.
New Changes in 12c
1) If stopping instance without -force or -failover option while you have service running on the stoping instance, errors (PRCD-1315,PRCR-1014, PRCR-1065, CRS-2529) are reported
[oracle@rac121 ~]$ srvctl stop instance -d racdb -i racdb1
PRCD-1315 : failed to stop instances for database racdb
PRCR-1014 : Failed to stop resource ora.racdb.db
PRCR-1065 : Failed to stop resource ora.racdb.db
CRS-2529: Unable to act on 'ora.racdb.db' because that would require stopping or relocating 'ora.racdb.temp_serv.svc', but the force option was not specified

2) If you want to stop both instance AND services running on the instance, use '-force' option.

[oracle@rac121 ~]$ srvctl stop instance -d racdb -i racdb1 -force 

3) If you want to stop instance but failover the services to another instance, you need to use '-failover' option instead '-force'. Click Here to get more details on this.

Tuesday, January 16, 2018

Temporary Undo

You might be in a situation where you are administering a database that supports an OLTP workload. An application regularly creates global temporary tables and a large number of transactions are performed on them. You notice that performance is degraded because of excessive generation of undo due to a large number of transactions on the global temporary tables.

Oracle database 12c Release 1 (12.1) introduced the concept of temporary undo, with this the undo segments for global temporary tables can be stored in the temporary tablespace.
By default we used to store undo segments of global temporary tables in regular undo tablespace, which is protected by REDO. 
This can cause multiple issues :-
  • Database should be in read-write mode to writing into undo tablespace. So global temporary tables can't be used in read-only databases and physical standby databases. 
  • Transient data ( contained by global temporary tables) is not required in database recovery scenarios. So protecting them with redo represents an unnecessary additional load on the system. 
  • The undo associated with global temporary tables adds to the total space needed to meet the undo retention period.

Key Points :-
  • When temporary undo is enabled, it might be necessary to increase the size of the temporary tablespaces to account for the undo records. 
  • When a session uses temporary objects for the first time, the current value of the TEMP_UNDO_ENABLED initialization parameter is set for the rest of the session. Therefore, if temporary undo is enabled for a session and the session uses temporary objects,then temporary undo cannot be disabled for the session. Similarly, if temporary undo is disabled for a session and the session uses temporary objects, then temporary undo cannot be enabled for the session. 
  • The temporary undo functionality is only available if the COMPATIBLE parameter is set to 12.0.0 or higher. 
  • Temporary undo is the default for standby databases, so any alteration to the TEMP_UNDO_ENABLED parameter is ignored on standby databases. 
  • Temporary undo is enabled by default for a physical standby database with the Oracle Active Data Guard option. 
  • You can use the TEMP_UNDO_ENABLED initialization parameter to separate undo for temporary tables from undo for persistent tables. When this parameter is set to TRUE, the undo for temporary tables is called temporary undo.
  • Temporary undo records are stored in the database's temporary tablespaces and thus are not logged in the redo log. When temporary undo is enabled, some of the segments used by the temporary tablespaces store the temporary undo, and these segments are called temporary undo segments.

Enable/Disable Temporary Undo :-
In Oracle 12c, undo segments for global temporary tables are still managed using the conventional undo tablespace by default. Temporary undo is enabled/disabled using the TEMP_UNDO_ENABLED parameter at session or system level.
--Session Level
ALTER SESSION SET TEMP_UNDO_ENABLED=TRUE/FALSE;
--System Level
ALTER SYSTEM SET TEMP_UNDO_ENABLED=TRUE/FALSE

Monitor Temporary Undo :-
  • V$UNDOSTAT -Regular undo activity
  • V$TEMPUNDOSTAT -temporary undo activity
Hands-On :-
Step 1 : Create Temporary Table
Create a global temporary table for testing.
SYS@racdb1> CREATE GLOBAL TEMPORARY TABLE test_table (
  id           NUMBER,
  details        VARCHAR2(10)
)
ON COMMIT DELETE ROWS;
Table created.
Make sure the user can access the V$TEMPUNDOSTAT view.Here I'm using SYS else you need to execute below grant statement
GRANT SELECT ON v_$tempundostat TO <user>;

Step 2 : Load data into GTT

Load some data into the GTT, then checks the redo and undo usage. This example is not using temporary undo.
SYS@racdb1> SET AUTOTRACE ON STATISTICS;
SYS@racdb1> INSERT INTO test_table
WITH data AS (
  SELECT 10 AS id
  FROM   dual
  CONNECT BY level < 20000
)
SELECT rownum, TO_CHAR(rownum)
FROM   data a, data b
WHERE  rownum <= 2000000; 
2000000 rows created.
Statistics
----------------------------------------------------------
         58  recursive calls
      32706  db block gets
       5012  consistent gets
         32  physical reads
    5930248  redo size
        855  bytes sent via SQL*Net to client
        984  bytes received via SQL*Net from client
          3  SQL*Net roundtrips to/from client
          3  sorts (memory)
          0  sorts (disk)
    2000000  rows processed

Step 3 : Verify UNDO generated by transaction
SYS@racdb1> SET AUTOTRACE OFF
SYS@racdb1> -- Check undo used by transaction.
SYS@racdb1> SELECT t.used_ublk,
       t.used_urec
FROM   v$transaction t,
       v$session s
WHERE  s.saddr = t.ses_addr
AND    s.audsid = SYS_CONTEXT('USERENV', 'SESSIONID'); 
 USED_UBLK  USED_UREC
---------- ----------
       614      12767
So this insert produces 614 blocks of undo, which is why the statistics show a redo size of 5.9 million bytes of redo, generated to protect the undo.

Now we'll enable temporary undo,checks the redo and undo usage.
SYS@racdb1> ALTER SESSION SET TEMP_UNDO_ENABLED = TRUE;
Session altered.
SYS@racdb1> SET AUTOTRACE ON STATISTICS;
SYS@racdb1> INSERT INTO test_table
WITH data AS (
  SELECT 10 AS id
  FROM   dual
  CONNECT BY level < 20000
)
SELECT rownum, TO_CHAR(rownum)
FROM   data a, data b
WHERE  rownum <= 2000000;   
2000000 rows created.
Statistics
----------------------------------------------------------
         48  recursive calls
      32760  db block gets
       4979  consistent gets
         31  physical reads
        980  redo size
        845  bytes sent via SQL*Net to client
        984  bytes received via SQL*Net from client
          3  SQL*Net roundtrips to/from client
          3  sorts (memory)
          0  sorts (disk)
    2000000  rows processed
SYS@racdb1> SET AUTOTRACE OFF
SYS@racdb1> -- Check undo used by transaction.
SYS@racdb1> SELECT t.used_ublk,
       t.used_urec
FROM   v$transaction t,
       v$session s
WHERE  s.saddr = t.ses_addr
AND    s.audsid = SYS_CONTEXT('USERENV', 'SESSIONID'); 
 USED_UBLK  USED_UREC
---------- ----------
         1          1
SYS@racdb1> -- Check temporary undo usage.
SYS@racdb1> SET LINES 1000
SYS@racdb1> ALTER SESSION SET NLS_DATE_FORMAT='DD-MON-YYYY HH24:MI:SS';
Session altered.
SYS@racdb1> SELECT *
FROM   v$tempundostat
WHERE  end_time >= SYSDATE - INTERVAL '1' MINUTE;  2    3
BEGIN_TIME           END_TIME                UNDOBLKCNT   
-------------------- -------------------- ---------- ---------- --------------
16-JAN-2018 19:35:54 16-JAN-2018 19:45:57         648  

Here we can see only a single byte of conventional undo is generated, which results in approximately 980 bytes of redo. The UNDOBLKCNT of the V$TEMPUNDOSTAT view shows us 648 blocks of temporary undo have been used.

So here with help of "Temporary Undo" we can address the issue causing by excessive generation of undo due to a large number of transactions on the global temporary tables.

Saturday, January 13, 2018

Data Recovery Advisor

The Data Recovery Advisor tool was added to RMAN in Oracle Database 11g.This tool automatically detects problems with your database that may require you to perform restore and recovery operations. The Data Recovery Advisor will list failures, give you advice on how to resolve issues, and allow you to push the button that instructs RMAN to run the commands required to fix the problem.

Below RMAN commands are associated with the Data Recovery Advisor:
  • list failure
  • advise failure
  • repair failure

Listing Failures

RMAN> list failure;
Here is some sample output:
List of Database Failures
=========================
Failure ID Priority Status Time Detected Summary
---------- -------- --------- ------------- -------
2 HIGH OPEN 13-JAN-18 One or more non-system data files are missing

The prior output indicates that the database has at least one data file that is missing. At this point you should next run the advise failure. To get more details on this failure:-
RMAN> list failure 52 detail;

The Data Recovery Advisor stores its information outside the database in the Automatic Diagnostic Repository (ADR). This allows you to run the list failure command even when your database is started in nomount mode and is also not dependent on the availability of the recovery catalog (if using one). You can determine the location of the base directory for the ADR by viewing the diagnostic_dest initialization parameter.

If you suspect there is a problem with your database and the Data Recovery Advisor is not reporting a failure, you can proactively initiate a database health check by running the following RMAN command:
RMAN> validate database;

If you want to see all failures that have been resolved and closed, then use the closed clause:
RMAN> list failure closed;

Advise Failure

You’ve experienced a media failure. You want to get advice from the Data Recovery Advisor about how to restore and recover your database.
 RMAN> advise failure;

OR
 RMAN> advise failure 52;

Depending on your situation, the advise failure output will contain one or both of the following sections:

  • Manual Checklist
  • Automated Repair Options

The Manual Checklist section gives you advice for manually resolving the issue.

The Automated Repair Options section will list the location and name of a repair script that contains RMAN commands to resolve the problem. At this point, you may want to open another terminal session and inspect the contents of the repair script (with a text editor such as vi or Notepad).

By analyzing the script you can gain a greater understanding of the failure and how RMAN intends to resolve the problem. If you want RMAN to automatically repair the failure, then run the repair failure.

Repair Failure

By running list failure and advise failure, you will gain an understanding as to what the problem is and how to fix it. You can use the repair failure command to run the repair script generated by the Data Recovery Advisor:
RMAN> repair failure;

It’s recommend that you run the repair failure command only after you have run the list failure and advise failure commands. Repairing the problem should be the last step performed. You should use the Data Recovery Advisor to repair failures only after you thoroughly understand what the failure is and what commands will be run to repair the failure.

If you want to inspect what the repair failure command will do without running the commands, then use the preview clause:
RMAN> repair failure preview;

Monday, January 1, 2018

Oracle Database 12c Release 1 (12.1) RAC On Oracle Linux 6 Using VirtualBox

This article describes the installation of Oracle Database 12c release 1(64 bit)  RAC on Linux (Oracle Linux 6.9 64-bit) using VirtualBox (5.1)
Introduction
System Requirements
Download Software
VirtualBox Installation
Virtual Machine Setup
Guest Operating System Installation
Check Internet Access
Oracle Installation Prerequisites
Automatic Setup
Manual Setup
Additional Setup
Install Guest Additions
Create Shared Disks
Clone the Virtual Machine
Install the Grid Infrastructure
Install the Database Software
Create a Database
Check the Status of the RAC

Introduction
If you want to get through all steps of the Oracle RAC installation and your laptop or desktop computer has 8 GB or more of RAM (16 GB is recommended), then this is entirely feasible using Oracle VirtualBox as demonstrated in this article. You can get a running RAC system which can host a small test database. The created system is not, and should not be considered, a production-ready system. It's simply to allow you to get used to installing and using RAC and test various administration procedures.
This article uses the 64-bit versions of Oracle Linux, version 6.4, and Oracle 12c Release 1, version 12.1.0.1.0. Using VirtualBox you can run multiple Virtual Machines (VMs) on a single server, allowing you to run both RAC nodes on a single machine. In addition, it allows you to set up shared virtual disks. The finished system includes two guest operating systems, two sets of Oracle Grid Infrastructure (Clusterware + ASM) and two Database instances all on a single server.

System Requirements
  • 12 GB of RAM, 16GB recommended
  • 100 GB of free space on the hard disk + staging area for Oracle Linux and RAC downloads, about 9 GB;
  • This procedure was tested on 64-bit Windows 10
Download the following software
Depending on your version of VirtualBox and Oracle Linux, there may be some slight variation in how the screen shots look.

VirtualBox Installation
First, install the VirtualBox software. On RHEL and its clones you do this with the following type of command as the root user.
# rpm -Uvh VirtualBox-4.3-4.3.16_95972_el6-1.x86_64.rpm
The package name will vary depending on the host distribution you are using. If you are doing this on windows system then, you have to just execute *.exe file. Once complete, VirtualBox is started from the menu.

Virtual Machine Setup
We must define two virtual RAC nodes. We can save time by defining one VM, then cloning it when it is installed.
Start VirtualBox and click the "New" button on the toolbar. Enter the
Name "rac121", Type "Linux" and Version "Oracle (64 bit)", and then click the "Next" button:

Enter "4096" as the base memory size, then click the "Next" button. Use more memory if you have enough physical memory on your machine as it will make the process much quicker!

Accept the default option to create a new virtual hard disk by clicking the "Create" button.

Accept the default VDI type and click the "Next" button on the Create Virtual Hard Drive screen:

Accept the default "Dynamically allocated" option by clicking the "Next" button:

Accept the default file name and set size to "40G", then click the "Create" button:

The "rac121" VM will appear on the left-hand pane. Select your machine and Click on the "Settings" link on the right side:

Make Network changes

Adapter 1- Host-only Adapter

Adapter 2- Internal Network

Adapter 3 – Bridged Adapter - This adapter is enabled for internet access in VM. We will disable this adapter, after yum update

System Changes
Move "Hard Disk" to the top of the boot order and uncheck the "Floppy" option, then click the "OK.


The virtual machine is now configured so we can start the guest operating system installation.

Guest Operating System Installation



With the new VM highlighted, click the "Start" button on the toolbar. On the "Select start-up disk" screen, choose the relevant Oracle Linux ISO image and click the "Start".





The resulting console window will contain the Oracle Linux boot screen.





Do not perform the media test. Choose "Skip" button:




On the "Welcome" screen, click the "Next" button.







Select the appropriate language, then click the "Next" button.

Select the relevant keyboard setting, then click the "Next" button.

Select the storage option "Basic Storage Devices" for the installation, then click the "Next" button.

Click on Yes, discard any data and proceed to next.

Enter a fully qualified host name - rac121.localdomain, then click the "Configure Network" button.

Highlight the relevant connection and click the "Edit" button for each connection.



Note : IPv6 Settings are “Ignored” for each connections

Select the relevant time zone by clicking on your nearest city on the map. Click on the "Next" button to proceed.

Enter a root password for the server, then click the "Next" button to proceed.

Check the partitioning type you require. If you want to amend the default partitioning layout, check the "Review and modify partitioning layout" option. Click the "Next" button.

The installer will list the default partitioning scheme for your size disk. Amend them as required and click the "Next" button, then the "Format" and "Write changes to disk" buttons on the following dialogs.




Accept the boot loader settings by clicking the "Next" button.
                          

Accept the "Basic Server" installation and check the "Customize now" option, then click the "Next" button.

The "Package Group Selection" screen allows you to select the required package groups, and individual packages within the details section. When you've made your selection, click the "Next" button. Make sure to install below packages :-
following package groups installed:
·       Base System > Base
·       Base System > Compatibility libraries
·       Base System > Hardware monitoring utilities
·       Base System > Large Systems Performance
·       Base System > Network file system client
·       Base System > Performance Tools
·       Base System > Perl Support
·       Servers> Server Platform
·       Servers> System administration tools
·       Desktops> Desktop
·       Desktops> Desktop Platform
·       Desktops> Fonts
·       Desktops> General Purpose Desktop
·       Desktops> Graphical Administration Tools
·       Desktops> Input Methods
·       Desktops> X Window System
·       Applications> Internet Browser
·       Development> Additional Development
·       Development> Development Tools

Wait for the installation to complete.

Click on Reboot.


On the "Welcome" screen, click the "Forward" button


Accept the license agreement and click the "Forward" button.

Pick the desired ULN Registration option, then click the "Forward" button. In this case we will pick the register later option. Click the "No thanks, I'll connect later." button.

Finish the software updates setup by clicking the "Forward" button

Enter the details for system user, then click the "Forward" button.

Adjust the Date and Time settings if necessary, and click the "Forward" Button.

uncheck the default settingon the Kdump screen by clicking the "Finish" button

Next, you are presented with the login screen. Log in by clicking on the user on the login screen.

Firewall

To turn off the firewall do the following as the "root" user.

# service iptables stop
# chkconfig iptables off

So your machine is ready. You are free to change the IP addresses to suit your network, but remember to stay consistent with those adjustments throughout the rest of the article.
Let’s perform some check son machine.

Check Internet Access
We will need Internet access because additional packages will be installed online. Open terminal (Main menu | Applications | System Tools | Terminal) and try to ping any Internet site, for example: 
host google.com

If ping doesn't work, troubleshoot the problem using 'ifconfig' command and making changes in Network Connections (Linux desktop Main menu | System | Preferences | Network Connections). If you made changes in Network Connections, restart interface by rebooting VM or running these two commands: 
# ifdown eth0
# ifup eth0
Then check the ping again. 

Oracle Installation Prerequisites
Perform either the Automatic Setup or the Manual Setup to complete the basic prerequisites. The Additional Setup is required for all installations.

Automatic Setup
If you plan to use the "oracle-rdbms-server-12cR1-preinstall" package to perform all your prerequisite setup, issue the following command.
# yum install oracle-rdbms-server-12cR1-preinstall -y
It is probably worth doing a full update as well, but this is not strictly speaking necessary.
# yum update -y
# yum install oracleasm
# yum install oracleasm-support
# yum install gedit
Cleanup YUM repositories:
# yum clean all

Note :-
Now you have to disable Adapter 3 (Devices >> Network >> Network Settings) and delete eth2 (System >> Preferences >> Network Connections). This step was necessary in my lab as scan were not working without this.  


Manual Setup
If you have not used the "oracle-rdbms-server-12cR1-preinstall" package to perform all prerequisites, you will need to manually perform the following setup tasks.

Add or amend the following lines to the "/etc/sysctl.conf" file.
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500

Run the following command to change the current kernel parameters.

/sbin/sysctl -p

Add the following lines to the "/etc/security/limits.d/oracle-database-server-12cR2-preinstall.conf" file.
oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc    2047
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768

In addition to the basic OS installation, the following packages must be installed whilst logged in as the root user. This includes the 64-bit and 32-bit versions of some packages.
# From Public Yum or ULN
yum install binutils -y
yum install compat-libcap1 -y
yum install compat-libstdc++-33 -y
yum install compat-libstdc++-33.i686 -y
yum install gcc -y
yum install gcc-c++ -y
yum install glibc -y
yum install glibc.i686 -y
yum install glibc-devel -y
yum install glibc-devel.i686 -y
yum install ksh -y
yum install libgcc -y
yum install libgcc.i686 -y
yum install libstdc++ -y
yum install libstdc++.i686 -y
yum install libstdc++-devel -y
yum install libstdc++-devel.i686 -y
yum install libaio -y
yum install libaio.i686 -y
yum install libaio-devel -y
yum install libaio-devel.i686 -y
yum install libXext -y
yum install libXext.i686 -y
yum install libXtst -y
yum install libXtst.i686 -y
yum install libX11 -y
yum install libX11.i686 -y
yum install libXau -y
yum install libXau.i686 -y
yum install libxcb -y
yum install libxcb.i686 -y
yum install libXi -y
yum install libXi.i686 -y
yum install make -y
yum install sysstat -y
yum install unixODBC -y
yum install unixODBC-devel -y

Create the new groups and users.
groupadd -g 54321 oinstall
groupadd -g 54322 dba
groupadd -g 54323 oper
#groupadd -g 54324 backupdba
#groupadd -g 54325 dgdba
#groupadd -g 54326 kmdba
#groupadd -g 54327 asmdba
#groupadd -g 54328 asmoper
#groupadd -g 54329 asmadmin
useradd -u 54321 -g oinstall -G dba,oper oracle
Uncomment the extra groups you require.


Additional Setup
The following steps must be performed, whether you did the manual or automatic setup.
Perform the following steps whilst logged into the "rac121" virtual machine as the root user.

Set the password for the "oracle" user.
passwd oracle

Apart form the localhost address, the "/etc/hosts" file can be left blank, but I prefer to put the addresses in for reference.

[root@rac121 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
#127.0.0.1       localhost.localdomain   localhost
192.168.56.71 rac121.localdomain rac121
# Private
192.168.10.1    rac121-priv.localdomain   rac121-priv
192.168.10.2    rac122-priv.localdomain   rac122-priv
# Public
192.168.56.71    rac121.localdomain        rac121
192.168.56.72    rac122.localdomain        rac122
# Virtual
192.168.56.81    rac121-vip.localdomain    rac121-vip
192.168.56.82    rac122-vip.localdomain    rac122-vip
# SCAN
192.168.56.91    rac-scan.localdomain    rac-scan
192.168.56.92    rac-scan.localdomain    rac-scan
192.168.56.93    rac-scan.localdomain    rac-scan
 

With this in place and the DNS configured the SCAN address is being resolved to all three IP addresses.
[root@rac121 ~]# nslookup rac-scan
Server:         127.0.0.1
Address:        127.0.0.1#53
Name:   rac-scan.localdomain
Address: 192.168.56.93
Name:   rac-scan.localdomain
Address: 192.168.56.91
Name:   rac-scan.localdomain
Address: 192.168.56.92

Change the setting of SELinux to permissive by editing the "/etc/selinux/config" file, making sure the SELINUX flag is set as follows.
SELINUX=permissive

Either configure NTP, or make sure it is not configured so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes. In this case we will deconfigure NTP.
# service ntpd stop
Shutting down ntpd:                                        [  OK  ]
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid

If your RAC is going to be permanently connected to your main network and you want to use NTP, you must add the "-x" option into the following line in the "/etc/sysconfig/ntpd" file.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

Then restart NTP.
# service ntpd restart

Create the directories in which the Oracle software will be installed.
mkdir -p  /u01/app/12.1.0.2/grid
mkdir -p /u01/app/oracle/product/12.1.0.2/db_1
chown -R oracle:oinstall /u01
chmod -R 775 /u01/

Log in as the "oracle" user and add the following lines at the end of the "/home/oracle/.bash_profile" file.
# Oracle Settings
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=rac121.localdomain
export ORACLE_UNQNAME=racdb
export ORACLE_BASE=/u01/app/oracle
export GRID_HOME=/u01/app/12.1.0.2/grid
export DB_HOME=$ORACLE_BASE/product/12.1.0.2/db_1
export ORACLE_HOME=$DB_HOME
export ORACLE_SID=racdb1
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
alias grid_env='. /home/oracle/grid_env'
alias db_env='. /home/oracle/db_env'

Create a file called "/home/oracle/grid_env" with the following contents.
export ORACLE_SID=+ASM1
export ORACLE_HOME=$GRID_HOME
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

Create a file called "/home/oracle/db_env" with the following contents.
export ORACLE_SID=racdb1
export ORACLE_HOME=$DB_HOME
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

Let's do a reboot of the VM at this point to make sure all the changes have taken effect.
# shutdown -r now 


Install Guest Additions

Click on the "Devices > Install Guest Additions" menu option at the top of the VM screen.
            





Add the "oracle" user into the "vboxsf" group so it has access to shared drives.
# usermod -G vboxsf,dba oracle
# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf)
#

Create a shared folder (Devices > Shared Folders) on the virtual machine, pointing to the directory on the host where the Oracle software was unzipped. Check the "Auto-mount" and "Make Permanent" options before clicking the "OK" button.

The VM will need to be restarted for the guest additions to be used properly. The next section requires a shutdown so no additional restart is needed at this time. Once the VM is restarted, the shared folder called "/media/sf_12.1.0.2" will be accessible by the "oracle" user.

Create Shared Disks
Shut down the "rac121" virtual machine using the following command.

# shutdown -h now

On the host server, create 4 sharable virtual disks and associate them as virtual media using the following commands. You can pick a different location.

cd C:\Program Files\Oracle\VirtualBox\
# Create the disks and associate them with VirtualBox as virtual media.
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm1.vdi" --size 3072 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm2.vdi" --size 3072 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm3.vdi" --size 3072 --format VDI --variant Fixed
VBoxManage createhd --filename "C:\My Drive\Virtual Disks\asm4.vdi" --size 3072 --format VDI --variant Fixed

# Connect them to the VM.
VBoxManage storageattach rac121 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm1.vdi" --mtype shareable
VBoxManage storageattach rac121 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm2.vdi" --mtype shareable
VBoxManage storageattach rac121 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm3.vdi" --mtype shareable
VBoxManage storageattach rac121 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm4.vdi" --mtype shareable

# Make shareable.
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm1.vdi" --type shareable
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm2.vdi" --type shareable
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm3.vdi" --type shareable
VBoxManage modifyhd "C:\My Drive\Virtual Disks\asm4.vdi" --type shareable

Start the "rac121" virtual machine by clicking the "Start" button on the toolbar. When the server has started, log in as the root user so you can configure the shared disks. The current disks can be seen by issuing the following commands.
# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdc  sdd  sde
#

Use the "fdisk" command to partition the disks sdb to sde.

[root@rac121 dev]# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-652, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652):
Using default value 652

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#

In each case, the sequence of answers is "n", "p", "1", "Return", "Return" and "w".

Once all the disks are partitioned, the results can be seen by repeating the previous "ls" command.
# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdb1  sdc  sdc1  sdd  sdd1  sde  sde1
#

Configure your UDEV rules.

Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted. Create the file if it doesn't already exist.

options=-g

The SCSI ID of my disks are displayed below.
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sdb
1ATA_VBOX_HARDDISK_VB5a97a6d6-443d513c
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sdc
1ATA_VBOX_HARDDISK_VBebd1447e-07dfd77f
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sdd
1ATA_VBOX_HARDDISK_VB6d4e6597-6510505c
[root@rac121 dev]# /sbin/scsi_id -g -u -d /dev/sde
1ATA_VBOX_HARDDISK_VB15e37173-5a00f121

Using these values, edit the "/etc/udev/rules.d/99-oracle-asmdevices.rules" file adding the following 4 entries. All parameters for a single entry must be on the same line.

KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB5a97a6d6-443d513c",NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBebd1447e-07dfd77f",NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB6d4e6597-6510505c",NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB15e37173-5a00f121",NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"

Load updated block device partition tables.

# /sbin/partprobe /dev/sdb1
# /sbin/partprobe /dev/sdc1
# /sbin/partprobe /dev/sdd1
# /sbin/partprobe /dev/sde1

Test the rules are working as expected.
# /sbin/udevadm test /block/sdb/sdb1

Reload the UDEV rules and start UDEV.
# /sbin/udevadm control --reload-rules
# /sbin/start_udev

The disks should now be visible and have the correct ownership using the following command. If they are not visible, your UDEV configuration is incorrect and must be fixed before you proceed.
[root@rac121 dev]# ls -al /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Dec 28 19:40 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 33 Dec 28 19:40 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Dec 28 19:40 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Dec 28 19:40 /dev/asm-disk4

The shared disks are now configured for the grid infrastructure.

Clone the Virtual Machine
Later versions of VirtualBox allow you to clone VMs, but these also attempt to clone the shared disks, which is not what we want. Instead we must manually clone the VM.

Shut down the "rac121" virtual machine using the following command.
# shutdown -h now

Manually clone the "rac121.vdi" disk using the following commands on the host server.
cd C:\Program Files\Oracle\VirtualBox\
C:\Program Files\Oracle\VirtualBox>VBoxManage clonehd "C:\My Drive\Virtual Disks\rac121.vdi" "C:\My Drive\Virtual Disks\rac122.vdi"

Create the "rac122" virtual machine in VirtualBox in the same way as you did for "rac121", with the exception of using an existing "rac122.vdi" virtual hard drive.

When the VM is created, attach the shared disks to this VM.
cd C:\Program Files\Oracle\VirtualBox\
VBoxManage storageattach rac122 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm1.vdi" --mtype shareable
VBoxManage storageattach rac122 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm2.vdi" --mtype shareable
VBoxManage storageattach rac122 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm3.vdi" --mtype shareable
VBoxManage storageattach rac122 --storagectl "SATA" --port 4 --device 0 --type hdd --medium "C:\My Drive\Virtual Disks\asm4.vdi" --mtype shareable
Start the "rac122" virtual machine by clicking the "Start" button on the toolbar. Ignore any network errors during the startup.

Log in to the "rac122"  virtual machine as the "root" user so we can reconfigure the network settings to match the following.

Amend the hostname in the "/etc/sysconfig/network" file.
NETWORKING=yes
HOSTNAME=rac122.localdomain


Start "Network Connections" tool (Main menu | System | Preferences | Network Connections). Edit eth0 and eth1 interfaces and set in IPv4 addresses 192.168.56.72 and 192.168.10.2 correspondingly.



Now we need to change MAC address for all three interfaces. At the moment we have two VMs with the same set of MAC addresses. We can run one machine or another, but not both of them at the same time because MAC address must be unique. No changes will be made to rac1, we will pick up three new unused addresses and set them for eth0, eth1, and eth2 in rac2. The easiest way to do that is to change just last two characters of the address. We are going to change them to '00'. If the last two characters are already '00', then change to something else, '01', for example. Just make sure that these addresses don't collide with the MAC addresses of rac1. In running rac2 node, open "Network Connections" and edit MAC address in the "Wired" tab. The screenshot below shows where to set MAC address. Don't forget to change MAC addresses for all three interfaces. Please note that your setup will have a different set of MAC addresses because they are random-generated by VirtualBox.


Write down the new MAC addresses for all three interfaces. Save new settings pressing "Apply" button, then shutdown the machine. After shutdown, return to the VirtualBox Manager, select rac122 VM and edit "Network" settings. Make same changes to the MAC addresses. Don't forget to change MAC addresses for all three adapters.


Restart the virtual machines.
# shutdown -r now

Edit the "/home/oracle/.bash_profile" file on the "ol6-121-rac2" node to correct the ORACLE_SID and ORACLE_HOSTNAME values.
export ORACLE_SID=racdb1
export ORACLE_HOSTNAME=rac122.localdomain

Also, amend the ORACLE_SID setting in the "/home/oracle/db_env" and "/home/oracle/grid_env" files.

Restart the "rac122" virtual machine and start the "rac121" virtual machine. When both nodes have started, check they can both ping all the public and private IP addresses using the following commands.
ping -c 3 rac121
ping -c 3 rac121-priv
ping -c 3 rac122
ping -c 3 rac122-priv

Check the SCAN address is still being resolved properly on both nodes
[root@rac121 dev]# nslookup rac-scan
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   rac-scan.localdomain
Address: 192.168.56.93
Name:   rac-scan.localdomain
Address: 192.168.56.91
Name:   rac-scan.localdomain
Address: 192.168.56.92

[root@rac122 ~]# nslookup rac-scan
Server:         127.0.0.1
Address:        127.0.0.1#53
Name:   rac-scan.localdomain
Address: 192.168.56.93
Name:   rac-scan.localdomain
Address: 192.168.56.91
Name:   rac-scan.localdomain
Address: 192.168.56.92

Check the UDEV rules are working on both machines. In previous versions of OL6 the "/etc/udev/rules.d/99-oracle-asmdevices.rules" file copied between servers during the clone without any issues. For some reason, this doesn't seem to happen on my OL6 installations, so you may need to repeat the UDEV configuration on the second node if the output of the following command is not consistent on both nodes.
[root@rac122 ~]# ls -al /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Dec 28 20:36 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 33 Dec 28 20:36 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Dec 28 20:36 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Dec 28 20:34 /dev/asm-disk4

Install the Grid Infrastructure
Make sure both virtual machines are started. Install the following package from the Oracle grid media as the root user.
[root@rac122 rpm]# cd /media/sf_54/grid_sf/grid/rpm
[root@rac122 rpm]# rpm -Uvh cvuqdisk*
Login to "rac121" as the "oracle" user (better to avoid using "su -" command, might have problems) and start the Oracle installer.
$ cd media/sf_54/grid_sf/grid/
$ ./runInstaller
Select the "Install and Configure Oracle Grid Infrastructure for a Cluster" option, then click the "Next" button.

Accept the "Configure a Standard cluster" option by clicking the "Next" button.

Select the "Typical Installation" option, then click the "Next" button.

On the "Specify Cluster Configuration" screen, enter the correct SCAN Name and click the "Add" button.

Enter the details of the second node in the cluster, then click the "OK" button.

Click the "SSH Connectivity..." button and enter the password for the "oracle" user. Click the "Setup" button to configure SSH connectivity, and the "Test" button to test it once it is complete. Once the test is complete, click the "Next" button.





On the "Specify Network Interface Usage" screen check the Public and Private networks are specified correctly. Press the "Next" button.

Enter "/u01/app/12.1.0.2/grid" as the software location and "Automatic Storage Manager" as the cluster registry storage type. Enter the ASM password, select "dba" as the group and click "Next".

Set the redundancy to "External", click the "Change Discovery Path" button and set the path to "/dev/asm*". Return the main screen and select all 4 disks and click the "Next" button.


Accept the default inventory directory by clicking the "Next" button.

If you want the root scripts to run automatically, enter the relevant credentials. I prefer to run them manually. Click the "Next" button.

Wait while the prerequisite checks complete. If you have any issues use the "Fix & Check Again" button. Once possible fixes are complete, check the "Ignore All" checkbox and click the "Next" button.It is likely the "SwapMemory" and "Device Checks for ASM" tests will fail for this type of installation. This is OK.

If you are fine with the summary information, click the "Install" button.

Wait while the installation takes place and When prompted, run the configuration scripts on each node.

Wait for the configuration assistants to complete. If any of the configuration steps fail you should check the specified log to see if the error is a show-stopper or not.

The grid infrastructure installation is now complete. We can check the status of the installation using the following commands.
[oracle@rac122 ~]$ grid_env
[oracle@rac122 ~]$ crsctl stat res -t

Install the Database Software
Make sure both virtual machines are started, then login to "rac121" as the oracle user and start the Oracle installer. Check that all services are up using "crsctl stat res -t", as described
[oracle@rac122 ~]$ cd /media/sf_54/db_sf/database
[oracle@rac122 database]$ ./runInstaller

Uncheck the security updates checkbox and click the "Next" button and "Yes" on the subsequent warning dialog.

Select the "Install database software only" option, then click the "Next" button

Accept the "Oracle Real Application Clusters database installation" option by clicking the "Next" button.

Make sure both nodes are selected, then click the "Next" button.

Select the "Language" and "Enterprise Edition" option, then click the "Next" button.

Enter "/u01/app/oracle" as the Oracle base and "/u01/app/oracle/product/12.1.0.2/db_1" as the software location, then click the "Next" button.

Select the desired operating system groups, then click the "Next" button.

Wait for the prerequisite check to complete. If there are any problems either click the "Fix & Check Again" button, or check the "Ignore All" checkbox and click the "Next" button.

If you are happy with the summary information, click the "Install" button.

Wait while the installation takes place and When prompted, run the configuration script on each node. When the scripts have been run on each node, click the "OK" button

Click the "Close" button to exit the installer.


Create a Database
Make sure both virtual machines are started, then login to "rac121" as the oracle user and start the Database Creation Asistant (DBCA).
$dbca


Check "Create a database with default configuration"; type in "Global Dtabase Name" the desired name; we have selected AL32UTF8 character set, make your choice as needed; also supply a password for SYS. We are going to create a reqular (pre-12c type) database, the "Create As Container Database is unchecked. Press Next button:

Wait for the prerequisite check to complete. The only reported problem was all familiar "Swap Size". Check "Ignore All" checkbox and click the "Next" button. Press "Finish" button in the Summary screen:







The RAC database creation is now complete.

Check the Status of the RAC

[oracle@rac121 bin]$ srvctl config database -d racdb
Database unique name: racdb
Database name: racdb
Oracle home: /u01/app/oracle/product/12.1.0.2/db_1
Oracle user: oracle
Spfile: +DATA/RACDB/PARAMETERFILE/spfile.293.963776861
Password file: +DATA/RACDB/PASSWORD/pwdracdb.276.963776547
Domain: localdomain
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: DATA
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: oper
Database instances: racdb1,racdb2
Configured nodes: rac121,rac122
Database is administrator managed
[oracle@rac121 bin]$ srvctl status database -d racdb
Instance racdb1 is running on node rac121
Instance racdb2 is running on node rac122
SQL> SELECT inst_name FROM v$active_instances;

INST_NAME
--------------------------------------------------------------------------------
rac121.localdomain:racdb1
rac122.localdomain:racdb2


Here your RAC database "racdb" is ready with two instances "racdb1"  and "racdb2" on nodes "rac121" and "rac122".
If you see any issues while doing above tasks, feel free to post them in comment section.