Saturday, February 1, 2014

Unix Shell Scripting-Conditional Code

- When every UNIX command completes, it invisibly returns a value to the program that started it (usually the shell) informing that program of the “status” of completion of the command.

- That value is a number, and is known as the “exit” status of the command

- Typically an exit status of 0 means that the program completed without any errors, while an exit status of some other value (other than 0) means errors has occurred

- The exit status is stored in a build-in variable called “?” and can be examined at any time with the command -   echo $?  as below

- The contents of this variable(?) as updated every time a program is run (including the echo command)

[oracle@rac1 ~]$ ls dffsdf
ls: cannot access dffsdf: No such file or directory


[oracle@rac1 ~]$ echo $?
2

[oracle@rac1 ~]$ echo $?
0  <--  The value 0 in here means that the above echo command has run successfully without any errors

- It is often useful for shell programming to think of exit status of 0 & 1 as below

0 – True – Executed without any errors

1 – False – Executed with errors

[oracle@rac1 ~]$ true
[oracle@rac1 ~]$ echo $?
0


[oracle@rac1 ~]$ false
[oracle@rac1 ~]$ echo $?
1

Wednesday, January 29, 2014

Unix Shell Scripting - Line Command

- It is possible to run two or more Unix commands on the same line in a shell script, by separating the commands with ; (semicolon) character

[oracle@rac1 ~]$ echo Todays date is ;date
Todays date is
Thu Jan 30 08:45:08 IST 2014

- For aesthetic reasons if you wish to split the command into multiple lines, we can use single quotes, double quotes or back slash

Using Double Quotes:

[oracle@rac1 ~]$ echo "This is an example
> of command splitting
> across multiple
> lines"
This is an example
of command splitting
across multiple
lines

Using Single Quotes:

[oracle@rac1 ~]$ echo 'This is an example
> of command splitting
> across multiple
> lines'
This is an example
of command splitting
across multiple
lines

Using Backslash:

[oracle@rac1 ~]$ echo This is an example \
> of command splitting \
> across multiple \
> lines
This is an example of command splitting across multiple lines

Notice the difference for splitting the command using backslash, that’s the specialty of Backslash

Unix Shell Scripting – Grouping Commands ()

- Occasionally in Unix Shell scripting we may need to group multiple commands together and treat them as single command

Why that is needed ?  Lets validate the requirements examples for doing so

- We may need to run the command in background (via the & command)

- We may need to redirect the output of the command to a file or other program (or command)  (using > or |)

- If you wish to apply the same action (in the example below output to other file using >) to several consecutive programs(in the example below echo & date programs), it is possible to group them together and apply the action to the group as below

Without Grouping-

[oracle@rac1 ~]$ echo Todays date is > DateOutput
[oracle@rac1 ~]$ date >> DateOutput
[oracle@rac1 ~]$ more DateOutput
Todays date is
Thu Jan 30 08:24:23 IST 2014

With Grouping – Checking the parentheses used to group commands echo & date, > before the date in second line(highlighted in yellow) shows that the you have opened the parentheses for grouping and waiting for other commands.

Check the output is same from the above(in which programs as executed individually) and below (programs are grouped together using parentheses () )

[oracle@rac1 ~]$ (echo Todays date is
> date ) > DateOutput
[oracle@rac1 ~]$ cat DateOutput
Todays date is
Thu Jan 30 08:29:03 IST 2014

Instead you can use the ; (Line COntroller) character to separate the commands and mention the commands in the same line as below

[oracle@rac1 ~]$ (echo User Logged is $LOGNAME today at ;date) > UserLogged

[oracle@rac1 ~]$ cat UserLogged
User Logged is oracle today at
Thu Jan 30 08:48:04 IST 2014

Unix Shell Scripting - Trouble with Quotes



Trouble with Quotes in Unix shell Scripting:

- In Unix shell scripting, there are 3 types of quotes
    - single quote -  '
    - Double quote -  "
    - Back quote   - `

- Single Quotes - The text enclosed in single quote is considered to be normal text including the special characters like - $, #, *, [, space character(space is a special character used to identify the list of variables), newline character except for another single quote(')
[oracle@rac1 ~]$ echo 'The user logged in is $LOGNAME'
The user logged in is $LOGNAME

- Double Quotes - The double quote will maintain the special characters as is, like $, `
[oracle@rac1 ~]$ echo "The user logged in is $LOGNAME"
The user logged in is oracle

To remove the special meaning of characters used in double quotes those need to be escaped using backslash \
[oracle@rac1 ~]$ echo "The user logged in is \$LOGNAME"
The user logged in is $LOGNAME


- Back Quotes - 
- The commands mentioned in the back quotes are treated as Unix commands and executed in a new shell and output is displayed on to the shell from which the command is run

[oracle@rac1 ~]$ echo "Todays date is `date`"
Todays date is Thu Jan 30 08:03:16 IST 2014

- The output generated by the commands in back quotes will not have the newlines, multiple spaces or tabs are replaced with single space.

[oracle@rac1 ~]$ who | sort
oracle   pts/0        2014-01-27 07:47 (:0.0)
oracle   tty1         2014-01-27 07:45 (:0)

[oracle@rac1 ~]$ list=`who | sort`
[oracle@rac1 ~]$ echo $list
oracle pts/0 2014-01-27 07:47 (:0.0) oracle tty1 2014-01-27 07:45 (:0)

Tuesday, January 14, 2014

Blocker Sessions in Oracle

In this post basically I will try to explain all about session blockers(or locks), how to identify them and finally how to resolve them.(In Production after getting the confirmation from the application development teams.)

For identifying the blockers, all the details are present in the dynamic views

1. v$session
2. v$lock
3. v$process

In the above dynamic views the columns which are useful to identify the details are as below

1. In v$session

- SID – Session Identifier.(This is a sequence number generated from audses$ sequence)
- SERIAL# – Session serial number. Used to identify a session’s objects.( SID & Serial# uniquely identify the session to be killed after the blocker session is identified).
- PADDR – Address of OS level process that owns the session
- USERNAME – Oracle User Name
- COMMAND – Command in progress (last statement parsed). We can find the sql command using the query - SELECT command_name FROM v$sqlcommand WHERE command_type = n;
- LOCKWAIT – Address of the lock the session is waiting for; NULL if none
- STATUS – Status of the session;
        ACTIVE – Session currently working executing the SQL statements.
        INACTIVE - Session which is inactive and either has no configured limits or has not yet exceeded the configured limits
        KILLED - Session marked to be killed
        CACHED - Session temporarily cached for use by Oracle*XA
        SNIPED - An inactive session that has exceeded some configured limits.
- SCHEMANAME - Schema User name
- OSUSER - Operating System client user name
- PROCESS - Operating system client process ID
- MACHINE - Operating system machine name
- TERMINAL - Operating system Terminal name
- PROGRAM - Operating system program name
- TYPE - Session Type
- SQL_ID - SQL identifier of the SQL statement that is currently being executed
- MODULE - Name of the currently executing module as set by calling the DBMS_APPLICATION_INFO.SET_MODULE procedure
- ROW_WAIT_OBJ# - Object ID of the table containing the row specified in ROW_WAIT_ROW#
- ROW_WAIT_FILE# - Identifier of the data file containing the row specified in ROW_WAIT_ROW#
- ROW_WAIT_BLOCK# - Identifier for the block containing the row specified in ROW_WAIT_ROW#
- ROW_WAIT_ROW# - Current Row being locked
- LAST_CALL_ET - If the session status is ACTIVE, then the value represents the elapsed time(in seconds) since the session has become active.
                 If the session status is INACTIVE, then the value represents the elapsed time(in seconds) since the session has become inactive.
- BLOCKING_SESSION - Session identifier of the blocking session.
- EVENT - Resource or event for which the session is waiting
- STATE - Wait State
            - WAITING - Session is currently waiting
            - WAITED UNKNOWN TIME - Duration of the last wait is unknown
            - WAITED SHORT TIME - Last wait was less than hundredth of a second
            - WAITED KNOWN TIME - Duration of the last wait is specified in the WAIT_TIME column

Creating the Blocking Sessions (For Demonstration):

Let us now create the row blockers for demonstration purpose. Consider the HR sample schemas which come with Oracle Installation.

For creating the row level locks, let us have two sessions

Session I – Try to updating Region_ID value of India country to 1 (currently it is 3 as shown below) in HR schema table – COUNTRIES. And please note that after the update statement the COMMIT statement is not executed making the update statement grab the exclusive row lock on table – COUNTRIES

captured_Image.png 
SQL> update HR.COUNTRIES set REGION_ID = 1 where COUNTRY_NAME = 'India';

1 row updated.

SQL
> --Note that the changes are not COMMITTED--

Session II – Open a new session by connecting to the database using user account SH and try updating the same row(Country India, Region Id value to say 10) in HR.COUNTRIES table.


SQL> show user
USER is "SH"
SQL
>
SQL
> update HR.COUNTRIES set REGION_ID=10 where COUNTRY_NAME='India';

--Note that the SQL> prompt is not returned as this session is waiting above HR session update to get committed or rollback--

Now lets check the details on which session is blocking which one and gathered the necessary details by querying the v$session, v$lock & v$process dynamic views.


Lets connect to database as sys dba to check the details about locks executing the below query on v$session dynamic view.


SQL> set linesize 10000
column STATE format a20
column USERNAME format a10
column EVENT format a30
select SID, SERIAL#, USERNAME, STATUS, STATE, EVENT, BLOCKING_SESSION from v$session where USERNAME is not null;

SID SERIAL# USERNAME STATUS STATE EVENT BLOCKING_SESSION
---------- ---------- ---------- -------- -------------------- ------------------------------ ----------------
249 5487 HR INACTIVE WAITING SQL*Net message from client
262 1889 SH ACTIVE WAITING enq: TX - row lock contention 249
377 1135 SYS ACTIVE WAITED SHORT TIME SQL*Net message to client

Below are the inference we can make from the above query details::
1. It can be noticed from the above query output that the session with SID – 249 (Update query executed by HR user) is basically blocking the session with SID – 262 (the update query executed by SH user).
2. Note the Event column of SID – 262 (SH User), “enq: TX – row lock contention” mean that the lock currently existing row level lock.
3. SID – 249 is in INACTIVE status and it is basically WAITING for other messages to be received from the client(in our case sql*plus client), perhaps the commit/rollback or other queries.
4. SID – 262 is in ACTIVE status and it is basically WAITING for the SID – 249 (HR User) to commit/rollback the transaction in which he is trying to update the row(Country India, REGION_ID = 1)
5. SID – 377 is the session from SYS login, where we have executed the query to find the details about the locks.
 
Now lets find the query details which is basically leading to row contention(lock). For this we need to look for sql_id column value in v$session and then sql_text value in v$sql dynamic view.
Modified query with the sql_id column included is below (please note that my DB session has disconnected and when tried the row lock creation after connecting to DB, system created different SID’s but the concept wise it is all same)
 
set linesize 10000
column STATE format a20
column USERNAME format a10
column EVENT format a30
select SID, SERIAL#, USERNAME, STATUS, STATE, EVENT, SQL_ID, BLOCKING_SESSION from v$session where USERNAME is not null;SQL> SQL> SQL> SQL>

SID SERIAL# USERNAME STATUS STATE EVENT SQL_ID BLOCKING_SESSION
---------- ---------- ---------- -------- -------------------- ------------------------------ ------------- ----------------
26 5 HR INACTIVE WAITING SQL*Net message from client 50tbf7vd82htx
254 14391 SYS ACTIVE WAITED SHORT TIME SQL*Net message from client 3tyc5ht9g7ynw
259 7621 SH ACTIVE WAITING enq: TX - row lock contention 63r8m52wk2uut 26
Now querying the v$sql view for the sql_id will give the details of the query causing the row contention.
 
select DISTINCT sq.sql_id, s.PROCESS "OS_ProcessID", s.SID, s.serial#, s.USERNAME "Executed_By_User", sq.sql_text from v$sql sq
JOIN v$session s ON (s.sql_id = sq.sql_id) where s.USERNAME is not null;

SQL_ID OS_ProcessID SID SERIAL# Executed_By_User SQL_TEXT
------------- ------------------------ ---------- ---------- ------------------------------ ------------------------------
50tbf7vd82htx 3799 26 5 HR update HR.COUNTRIES set REGION
_ID
= 1 where COUNTRY_NAME = '
India
'

9xbzgbk9znfkd
6288 254 14391 SYS select DISTINCT sq.sql_id, s.P
ROCESS "OS_ProcessID", s.SID,
s.serial#, s.USERNAME "
Execute
d_By_User", sq.sql_text
from v
$sql sq
JOIN v$session s ON (s
.sql_id
= sq.sql_id) where s.U
SERNAME
is not null

63r8m52wk2uut
27508 259 7621 SH update HR.COUNTRIES set REGION
_ID
=10 where COUNTRY_NAME='Ind
ia
'
 
 
Now that you have identified the queries causing the contention, we need to decide on killing them or committing or rollback based on the application need. Once the development team confirms that we can go ahead and kill the culprit session.  There are two ways we can kill the offensive session 
1. Using the database – ALTER SYSTEM KILL SESSION ‘SID, SERIAL#’;
2. Using the OS level Kill command – KILL –9 <OS_ProcessID>

SQL> !ps -ef | grep '3799\|6288\|27508' | grep -v grep
oracle
3799 3027 0 Jan14 pts/0 00:00:00 sqlplus as sysdba
oracle
6288 6221 0 Jan14 pts/2 00:00:00 sqlplus
oracle
27508 7698 0 07:51 pts/3 00:00:00 sqlplus

Monday, November 18, 2013

RAC - Validating ASM Instance


- For ASM storage, we need two components 1. ASM Drivers 2. Grid Infrastructure

- ASM Drivers is a optional nice to have - configures ASM disks, assigns ASM drivers, configuration at the node bootup

- To install oracleasm driver we need to install - 1. oracleasmlib(chipset specific) 2. oracleasm-support(chipset specific) 3. oracleasm(specific to OS)

- oracleasm driver will be installed in /etc/init.d location

[root@rac1 init.d]# pwd
/etc/init.d
[root@rac1 init.d]# ls -lart oracl*
-rwxr-xr-x. 1 root root 7124 Feb  9  2013 oracleasm
-rwx------. 1 root root 1450 Apr 24  2013 oracle-rdbms-server-12cR1-preinstall-firstboot
[root@rac1 init.d]# ./oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes

- Grid Infrastructure installation installs 1. ASM software 2. Clusterware software(CRS, CSS, SSH) 3. Installs & configures ASM instance

- After installing the GI, check for ASM process running

[root@rac1 init.d]# ps -ef | grep smon
oracle    1959     1  0 00:49 ?        00:00:00 ora_smon_MyDB1
oracle    2008     1  0 00:49 ?        00:00:00 mdb_smon_-MGMTDB
oracle    2690     1  0 Nov18 ?        00:00:01 asm_smon_+ASM1  <-- This confirms that the GI installation has installed and configured ASM instance.
root      2726     1  2 Nov18 ?        00:14:27 /u01/app/12.1.0/grid/bin/osysmond.bin
root      4461  3453  0 01:38 pts/0    00:00:00 grep smon

As part of GI installation ASM entry is made in /etc/oratab file

[root@rac1 init.d]# more /etc/oratab
#Backup file is  /u01/app/oracle/product/12.1.0/dbhome_1/srvm/admin/oratab.bak.rac1 line added by Agent
#

# This file is used by ORACLE utilities.  It is created by root.sh
# and updated by either Database Configuration Assistant while creating
# a database or ASM Configuration Assistant while creating ASM instance.

# A colon, ':', is used as the field terminator.  A new line terminates
# the entry.  Lines beginning with a pound sign, '#', are comments.
#
# Entries are of the form:
#   $ORACLE_SID:$ORACLE_HOME:<N|Y>:
#
# The first and second fields are the system identifier and home
# directory of the database respectively.  The third field indicates
# to the dbstart utility that the database should , "Y", or should not,
# "N", be brought up at system boot time.
#
# Multiple entries with the same $ORACLE_SID are not allowed.
#
#
+ASM1:/u01/app/12.1.0/grid:N: # line added by Agent
-MGMTDB:/u01/app/12.1.0/grid:N: # line added by Agent
MyDB:/u01/app/oracle/product/12.1.0/dbhome_1:N: # line added by Agent

Apart from above process, oratab entry, GI installation will create the two network called - vip & scan network

[root@rac1 init.d]# ping rac1-vip
PING rac1-vip.localdomain (192.168.56.81) 56(84) bytes of data.
64 bytes from rac1-vip.localdomain (192.168.56.81): icmp_seq=1 ttl=64 time=0.026 ms
64 bytes from rac1-vip.localdomain (192.168.56.81): icmp_seq=2 ttl=64 time=0.029 ms
64 bytes from rac1-vip.localdomain (192.168.56.81): icmp_seq=3 ttl=64 time=0.066 ms
^C
--- rac1-vip.localdomain ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2824ms
rtt min/avg/max/mdev = 0.026/0.040/0.066/0.018 ms

[root@rac1 init.d]# ping rac-scan
PING rac-scan.localdomain (192.168.56.91) 56(84) bytes of data.
64 bytes from rac-scan.localdomain (192.168.56.91): icmp_seq=1 ttl=64 time=1.48 ms
64 bytes from rac-scan.localdomain (192.168.56.91): icmp_seq=2 ttl=64 time=0.405 ms
64 bytes from rac-scan.localdomain (192.168.56.91): icmp_seq=3 ttl=64 time=0.477 ms
^C
--- rac-scan.localdomain ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2846ms
rtt min/avg/max/mdev = 0.405/0.788/1.484/0.493 ms

- Manually asm disk groups configuration can be performed using - asmca

- To check the details of spfile(binary) of ASM instance, create a pfile from spfile by logging into sqlplus as sysasm user. 

[oracle@rac1 disks]$ . oraenv
ORACLE_SID = [+ASM1] ? +ASM1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1 disks]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Tue Nov 19 01:53:53 2013

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> create pfile='/home/oracle/init+ASM1.ora' from spfile;

File created.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
[oracle@rac1 disks]$ cd ~
[oracle@rac1 ~]$ ls -lart init*
-rw-r--r-- 1 oracle oinstall 272 Nov 19 01:55 init+ASM1.ora
[oracle@rac1 ~]$ cat init+ASM1.ora
+ASM1.__oracle_base='/u01/app/oracle'#ORACLE_BASE set from in memory value
+ASM2.__oracle_base='/u01/app/oracle'#ORACLE_BASE set from in memory value
*.asm_diskstring='/dev/oracleasm/disks'
*.asm_power_limit=1
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'

Sunday, November 17, 2013

RAC - Oracle Cluster Registry (OCR), OLR & VOTING Disk Locations


In RAC cluster, 2 disks (to be presise files) are important
- 1. Voting Disk - Heartbeat of the RAC components
- 2. OCR - Oracle Cluster Registry - It has all list of RAC components (it is just like windows registry)
           It has all the components of RAC as DB Instances, ASM Instances, Listeners, SCAN listeners, RAC nodes

Voting disk checks for the heart beat of all the RAC nodes and if it can't receive the heart beat from a particular node, then voting disk evicts that particular node from RAC cluster.
Voting disk gets the details of all the nodes it is supposed to check the alive heartbeat from OCR disk.

Utilities to determine the location of OCR 
- OLR (Oracle Local Registry) - registry specific to a node
          OCR - Registry for overall RAC
          OLR - Registry for specific node
- OCRCHECK
- ASMCMD

- To work in OCR, the environment need to be set to ASM instance (+ASM1/2) or GI

--------------------------------------------------------------------------------------------
- To know which environment we need to mention in the rac cluster on executing . oraenv, we need to check the details in the /etc/oratab

[oracle@rac1 ~]$ cat /etc/oratab

#Backup file is  /u01/app/oracle/product/12.1.0/dbhome_1/srvm/admin/oratab.bak.rac1 line added by Agent
#
# This file is used by ORACLE utilities.  It is created by root.sh
# and updated by either Database Configuration Assistant while creating
# a database or ASM Configuration Assistant while creating ASM instance.

# A colon, ':', is used as the field terminator.  A new line terminates
# the entry.  Lines beginning with a pound sign, '#', are comments.
#
# Entries are of the form:
#   $ORACLE_SID:$ORACLE_HOME:<N|Y>:
#
# The first and second fields are the system identifier and home
# directory of the database respectively.  The third field indicates
# to the dbstart utility that the database should , "Y", or should not,
# "N", be brought up at system boot time.
#
# Multiple entries with the same $ORACLE_SID are not allowed.
#
#
+ASM1:/u01/app/12.1.0/grid:N: # line added by Agent
-MGMTDB:/u01/app/12.1.0/grid:N: # line added by Agent
MyDB:/u01/app/oracle/product/12.1.0/dbhome_1:N: # line added by Agent


- On entering the ORACLE_SID value while setting the environment variable, oracle RAC shouldn't prompt us with any other parameter to enter, that confirms we have set the environment correctly.

[oracle@rac2 ~]$ . oraenv
ORACLE_SID = [oracle] ? +ASM1 <Entering wrong ASM instance value +ASM1 instead of +ASM2, environment asked for ORACLE_HOME value to enter as below>
ORACLE_HOME = [/home/oracle] ? 

[oracle@rac2 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? +ASM2 <On entering the correct ASM instance +ASM2 on node 2, system didn't prompt 
for any variable value to be entered. It successfully set the environment as below>
The Oracle base has been changed from exit to /u01/app/oracle
[oracle@rac2 ~]$ 
----------------------------------------------------------------------------------------------------------------------------------
Find location of OCR Local disk file:
To find the location of the olr disk(or file), use ocrcheck with local clause as below.
Per the below details, the olr binary file is located at - /u01/app/12.1.0/grid/cdatarac1.olr

- On node 1 (rac1)

[root@rac1 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[root@rac1 ~]# ocrcheck -local
Status of Oracle Local Registry is as follows :
 Version                  :          4
 Total space (kbytes)     :     409568
 Used space (kbytes)      :        968
 Available space (kbytes) :     408600
 ID                       :  882531601
 Device/File Name         : /u01/app/12.1.0/grid/cdata/rac1.olr
                                    Device/File integrity check succeeded

 Local registry integrity check succeeded

 Logical corruption check succeeded


- On node 2 (rac2)

[root@rac2 ~]# . oraenv
ORACLE_SID = [+asm2] ? +ASM2
The Oracle base has been changed from /home/oracle to /u01/app/oracle
[root@rac2 ~]# ocrcheck -local
Status of Oracle Local Registry is as follows :
Version                  :          4
Total space (kbytes)     :     409568
Used space (kbytes)      :        976
Available space (kbytes) :     408592
ID                       :  873311553
Device/File Name         : /u01/app/12.1.0/grid/cdata/rac2.olr
                                    Device/File integrity check succeeded

Local registry integrity check succeeded

Logical corruption check succeeded



Alternatively one can find the location of the olr file from olr.loc file present in /etc/oracle
- On node 1 (rac1)

[root@rac1 oracle]# pwd
/etc/oracle
[root@rac1 oracle]# ls -lart
total 2876
drwxr-xr-x    3 root oinstall    4096 Nov  8 06:12 scls_scr
drwxrwxr-x    5 root oinstall    4096 Nov  8 06:12 oprocd
-rws--x---    1 root oinstall 2903931 Nov  8 06:12 setasmgid
-rw-r--r--    1 root root           0 Nov  8 06:12 olr.loc.orig
-rw-r--r--    1 root oinstall      80 Nov  8 06:12 olr.loc
-rw-r--r--    1 root root           0 Nov  8 06:12 ocr.loc.orig
-rw-r--r--    1 root oinstall      37 Nov  8 06:12 ocr.loc
drwxr-xr-x    6 root oinstall    4096 Nov  8 06:12 .
drwxrwx---    2 root oinstall    4096 Nov  8 06:29 lastgasp
drwxrwxrwt    2 root oinstall    4096 Nov 16 23:30 maps
drwxr-xr-x. 123 root root       12288 Nov 17 18:56 ..
[root@rac1 oracle]# more olr.loc
olrconfig_loc=/u01/app/12.1.0/grid/cdata/rac1.olr
crs_home=/u01/app/12.1.0/grid

- On node 2 (rac2)

[root@rac2 ~]# cd /etc/oracle
[root@rac2 oracle]# ls -lart
total 2876
drwxr-xr-x    3 root oinstall    4096 Nov  8 06:36 scls_scr
drwxrwxr-x    5 root oinstall    4096 Nov  8 06:36 oprocd
-rws--x---    1 root oinstall 2903931 Nov  8 06:36 setasmgid
-rw-r--r--    1 root root           0 Nov  8 06:36 ocr.loc.orig
-rw-r--r--    1 root oinstall      37 Nov  8 06:36 ocr.loc
-rw-r--r--    1 root root           0 Nov  8 06:36 olr.loc.orig
-rw-r--r--    1 root oinstall      80 Nov  8 06:36 olr.loc
drwxr-xr-x    6 root oinstall    4096 Nov  8 06:36 .
drwxrwx---    2 root oinstall    4096 Nov  8 06:42 lastgasp
drwxrwxrwt    2 root oinstall    4096 Nov 16 23:30 maps
drwxr-xr-x. 123 root root       12288 Nov 17 18:56 ..
[root@rac2 oracle]# more olr.loc
olrconfig_loc=/u01/app/12.1.0/grid/cdata/rac2.olr
crs_home=/u01/app/12.1.0/grid


Find location of OCR disk file:
- OCR & Voting disk must reside in the ASM disk group
- OCR & Voting disks works hand-in-hand to determine what are the nodes currently active and responding to heartbeat
- In RAC cluster all nodes should have the same server timestamp, else if they are off by even few secs, voting disks considers those nodes as not active and evicts them from the cluster.  To maintain the same timestamp across the cluster nodes, NTP(Network Time Protocol) is used.

Using ocrcheck one can find the location of the ASM disk group in which OCR file is located

- [root@rac1 bin]# ocrcheck
Status of Oracle Cluster Registry is as follows :
 Version                              :          4
 Total space (kbytes)        :     409568
 Used space (kbytes)        :       1528
 Available space (kbytes) :     408040
 ID                       :  121423581
 Device/File Name         :      +DATA
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

 Cluster registry integrity check succeeded

 Logical corruption check succeeded

- Location of the OCR disk file in ASM disk group

[oracle@rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? +ASM1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1 ~]$ asmcmd
ASMCMD> ls
DATA/
ASMCMD> cd data
ASMCMD> ls
ASM/
MYDB/
_MGMTDB/
orapwasm
rac-cluster/   <-- This is the cluster name given during the GI installation
ASMCMD> cd rac-cluster
ASMCMD> ls
ASMPARAMETERFILE/
OCRFILE/
ASMCMD> cd ocrfile
ASMCMD> ls
REGISTRY.255.830932005 <--OCR file is located in ASM disk grip
                     +data/rac-cluster/ocrfile/registry.255.830932005 (Binary file)




Find location of Voting Disk:
- Typically CRSCTL utility commands should be issued from the Grid Infrastructure environment(+ASM1 or +ASM2 ...)
  SRVCTL utility commands should be issued from Database environment (+<DBInstance>1, +<DBInstance>2, ..)

- To know the location of the voting disk file, execute the below command in GI environment as below

[oracle@rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? +ASM1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1 ~]$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   7427f69521004f49bf9221af584d4e6e (/dev/oracleasm/disks/DISK1) [DATA]
Located 1 voting disk(s).