Download ClusterWare and DB Software 12.1.0.2 

On Grid :-

Important: Validate ORACLE INVENTORY in both nodes . 

Check GRID Infrastructure software version and Cluster ware status:

cd $CRS_HOME/bin

bash-3.2$ ./crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [11.2.0.4.0]

bash-3.2$ ./crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

bash-3.2$ ./crs_stat -t

Check Database status and configuration :

bash-3.2$ ./srvctl status database -d PROD

Instance PROD1 is running on node racnode1

Instance PROD2 is running on node racnode2

bash-3.2$ ./srvctl config database -d PROD

Perform local backup of OCR 

cd $CRS_HOME/bin/

[root@racpb1 bin]# ./ocrconfig -export /home/oracle/ocrfile_bkp

Move the 12c GRID Software to the server and unzip the software :

Run cluvfy utility to pre-check  any errors :

cd /home/oracle/grid

./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /oracle/grid/11g -dest_crshome /oracle/grid/12cr1 -dest_version 12.1.0.2.0 -verbose

Issue: 

Solution:

[root@racnode1 grid]# systemctl status  avahi-daemon

[[root@racnnode1 grid]# systemctl stop avahi-daemo

[[root@racnode1 grid ~]# systemctl disable avahi-daemon

Upgrade 12c by executing runInstaller :

go to <software Dump Directory>

./runInstaller.sh

Issue:

Failed to write the key pair (SYSTEM.rootcrs.checkpoints.firstnode:FAIL) to OCR

2023/02/21 08:34:35 CLSRSC-115: Start of resource 'ora.cssd' failed

2023/02/21 08:34:35 CLSRSC-117: Failed to start Oracle Clusterware stack

2023/02/21 08:34:35 CLSRSC-244: Failed to start Cluster Synchronization Service in clustered mode

Died at /oracle/grid/12.1/crs/install/crsupgrade.pm line 794.

The command '/oracle/grid/12.1/perl/bin/perl -I/oracle/grid/12.1/perl/lib -I/oracle/grid/12.1/crs/install /oracle/grid/12.1/crs/install/rootcrs.pl  -upgrade' execution failed

Solution:

go to the $GRID_HOME/crs/install directory and execute the sh rootcrs.sh –downgrade script on failed node or all nodes in a cluster

• When prompted, run the rootupgrade.sh script on each node in the cluster that you want to upgrade.

• Run the script on the local node first. 

• The script shuts down the earlier release installation, replaces it with the new Oracle Clusterware release, and starts the new Oracle Clusterware installation.

After running rootupgrade.sh script,Click the OK button.

Issue: 

1. rootupgrdae.sh failed 

2. validate the Log File: $GRID_HOME/cfgtoollogs/crsconfig/rootcrs_xxxx.log

fix the issue and rerun rootupgrade.sh -force

Check the Clusterware upgrade version:

cd /oracle/grid/12cr1/bin/

./crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [12.1.0.2.0]

srvctl status database -d PROD

Instance PROD1 is running on node racnode1

Instance PROD2 is running on node racnode2

Validate the services after CRS upgrade:

./crsctl status resource -t

Then check the status of the clusterware globally as follows:

# crsctl check cluster -all

**************************************************************

racnode1:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

racnode2:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

Install RAC database from 11gR2 to 12cR1

Check the preupgrade status 

cd /home/oracle/grid

./runcluvfy.sh stage -pre dbinst -upgrade -src_dbhome /oracle/db/11g -dest_dbhome /oracle/db/12cr1 -dest_version 12.1.0.2.0

go to the 12c database dump directory 

cd /home/oracle/database

./runInstaller 

Upgrade RAC database

Run the DBUA to start the database upgrade

export ORACLE_HOME=/oracle/db/12cr1

export PATH=$ORACLE_HOME/bin:$PATH

export ORACLE_SID=PROD1

invoke  —---->> dbua

Check Database status and configuration :

bash-3.2$ ./srvctl status database -d PROD

Instance PROD1 is running on node racnode1

Instance PROD2 is running on node racnode2

bash-3.2$ ./srvctl config database -d PROD

Instance PROD1 is running on node racnode1

Instance PROD2 is running on node racnode2

[oracle@racnode1 bin]$ ./srvctl config database -d PROD

Database unique name: PROD

Database name: PROD

Oracle home: /oracle/db/12cr1

Oracle user: oracle

Spfile: +DATA/prod/spfileprod.ora

Password file:

Domain:

Start options: open

Stop options: immediate

Database role: PRIMARY

Management policy: AUTOMATIC

Server pools:

Disk Groups: DATA

Mount point paths:

Services:

Type: RAC

Start concurrency:

Stop concurrency:

OSDBA group: dba

OSOPER group:

Database instances: PROD1,PROD2

Configured nodes: racnode1,racnode2

Database is administrator managed

./crs_stat -t

Name           Type           Target    State     Host

------------------------------------------------------------

ora.DATA.dg    ora....up.type ONLINE    ONLINE    racnode1

ora....ER.lsnr ora....er.type ONLINE    ONLINE    racnode1

ora....N1.lsnr ora....er.type ONLINE    ONLINE    racnode2

ora....N2.lsnr ora....er.type ONLINE    ONLINE    racnode1

ora....N3.lsnr ora....er.type ONLINE    ONLINE    racnode1

ora.MGMTLSNR   ora....nr.type ONLINE    ONLINE    racnode1

ora.OCR.dg     ora....up.type ONLINE    ONLINE    racnode1

ora.asm        ora.asm.type   ONLINE    ONLINE    racnode1

ora.cvu        ora.cvu.type   ONLINE    ONLINE    racnode2

ora.mgmtdb     ora....db.type ONLINE    ONLINE    racnode1

ora....network ora....rk.type ONLINE    ONLINE    racnode1

ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    racnode2

ora.ons        ora.ons.type   ONLINE    ONLINE    racnode1

ora.prod.db    ora....se.type ONLINE    ONLINE    racnode1

ora....SM1.asm application    ONLINE    ONLINE    racnode1

ora....E1.lsnr application    ONLINE    ONLINE    racnode1

ora....de1.ons application    ONLINE    ONLINE    racnode1

ora....de1.vip ora....t1.type ONLINE    ONLINE    racnode1

ora....SM2.asm application    ONLINE    ONLINE    racnode2

ora....E2.lsnr application    ONLINE    ONLINE    racnode2

ora....de2.ons application    ONLINE    ONLINE    racnode2

ora....de2.vip ora....t1.type ONLINE    ONLINE    racnode2

ora.scan1.vip  ora....ip.type ONLINE    ONLINE    racnode2

ora.scan2.vip  ora....ip.type ONLINE    ONLINE    racnode1

ora.scan3.vip  ora....ip.type ONLINE    ONLINE    racnode1

[oracle@racnode1 bin]$

./crsctl stop cluster -all

./crsctl start cluster -all