Interview Q and A for Oracle ASM Part - 4

91. ASM instance is not discovering disks when asm_diskstring is 'ORCL:*'

You have reviewed the previous five points and still the disks are not discovered using the string 'ORCL:*' and only discover the disks if using the native path '/dev/sdX', '/dev/emcpowerxy' or '/dev/oracleasm/disks/XXX'. If you are in 10gR2 this is possible because from this version, Oracle can

execute IO referencing the block device and the devices under /dev/oracleasm/disks are block devices, linked to the physical device.

SOLUTION

We can simulate the disk discovery from the operating system level, using tool kfod. ($ORACLE_HOME/bin). The execution syntax is:

[usupport@jfrac1 bin]$ kfod asm_diskstring='ORCL:*' disks=all

--------------------------------------------------------------------------------

Disk Size Path

================================================================================

1: 954 Mb ORCL:ASMLIB1

2: 955 Mb ORCL:ASMLIB2

--------------------------------------------------------------------------------

ORACLE_SID ORACLE_HOME

================================================================================

+ASM2 /oracle/10gR2/asm

+ASM1 /oracle/10gR2/asm

That is the normal output when executed by oracle user. Sometimes, the command reports (discover) the disks when executed as root. This is an indication of an access problem in one of the files or directories under /opt/oracle. Make sure the permissions are 755 for all the directories and the

files under /opt/oracle.

If after verifying the permissions the disks are still not discovered, it is probably a faulty installation of the oracleasm rpms. The symptoms for this problem are:

The filesystem /dev/oracleasm is mounted

ASMLIB disks can be created using /etc/init.d/oracleasm createdisk command, and the block devices exist under /dev/oracleasm/disks

Commands like /etc/init.d/oracleasm listdisks or /etc/init.d/oracleasm querydisks return the expected results.

The first 5 points referenced in this note have been validated. The rpms are installed, at least what is rpm -qa oracleasm reports. At this point, the recommendation is reinstalling the rpms for ASMLIB. Use the --force flag.

92. ASMLIB disks are bound to the individual path device and not to the pseudo device created by the multipath software.

Multipathing allows to stablish multiple I/O access paths for an individual disk, providing features like load balancing, automatic failover. There is a pseudo device, which is created under /dev and it can be referenced by ASM. Examples are /dev/emcpowerxx (EMC PowerPath), /dev/vpath (IBM

SDD), /dev/md-x (Linux MD), /dev/dm (Linux DM).

ASMLIB disks can be created referencing this pseudo devices when /etc/init.d/oracleasm createdisk is used.

There is a situation during scandisks operation where the ASMLIB is binded to the single/individual path and not to the pseudo device. You can verify this running an ls -l /dev/oracleasm/disks and checking the major,minor numbers of the devices.

[usupport@jfrac1 bin]$ ls -l /dev/oracleasm/disks

total 0

brw-rw---- 1 usupport dba 8, 17 Jul 11 17:08 ASMLIB1

brw-rw---- 1 usupport dba 8, 18 Jul 11 17:08 ASMLIB2

In this particular case, ASMLIB1 is binded to disk identified with major,minor 8,17. File /proc/partitions contains the mapping of the major,minor and the name of the device.

Although some multipath vendors will trap the IO even if the individual path is referenced, it is prefered to bind the ASMLIB disks with the pseudo device created by the multipath layer.

SOLUTION

Modify parameter ORACLEASM_SCANORDER on file /etc/sysconfig/oracleasm and set the string associated with the pseudodevices.

Examples are:

ORACLEASM_SCANORDER=emcpower

ORACLEASM_SCANORDER=dm

ORACLEASM_SCANORDER=vpath

ORACLEASM_SCANORDER=md

If you are running a cluster, make sure to modify the file in all the nodes and restart ASMLIB.

93. ASMLIB Logging

ASMLIB provides additional logging for the following areas:

ENTRY /* func call entry */

EXIT /* func call exit */

DISK /* Disk information */

REQUEST /* I/O requests */

BIO /* bios backing I/O */

IOC /* asm_iocs */

ABI /* ABI entry points */

The settings are recorded in file /proc/fs/oracleasm, where those are the default values:

ENTRY deny

EXIT deny

DISK off

REQUEST off

BIO off

IOC off

ABI off

ERROR allow

NOTICE allow

There are three possible values:

deny

off

allow

To change the values, run this simple command:

echo "DISK allow" > /proc/fs/oracleasm/log_mask

This will change the particular value for the DISK entry without affecting others values. It is recommended to run the command for each change individually or you can use the following shell script:

1.log_mask="/proc/fs/oracleasm/log_mask"

2.echo " ***************** Current values ********************"

3.echo

4.cat $log_mask

5.echo REQUEST > /tmp/logmask.out

6.echo DISK >> /tmp/logmask.out

7. cat /tmp/logmask.out | (

8. while read bit status; do

9. # $1 is "allow" or "off

10. echo "$bit $1" > $log_mask

11. done

12. )

13.echo

14.echo "****************** New Values ***********************"

15.cat $log_mask

and execute sh x.sh <new value>.

Ex: x.sh allow

Setting the value to off will disable the extra logging.

This changes does not require restarting ASMLIB or restarting ASM Instance.

94. What is the default memory allocation for ASM?

In 10g the default SGA size is 1G in 11g it is set to 256M and in 12c ASM it is set back to 1G.

95. How do your backup ASM Metadata?

You can use md_backup to restore the ASM diskgroup configuration in-case of ASM diskgroup storage loss.

96. What files can be stored in the ASM diskgroup?

In 11g the following files can be stored in ASM diskgroups.

·       Datafiles

·       Redo logfiles

·       Spfiles

In 12c the files below can also new be stored in the ASM Diskgroup

·       Password file

97. What it the ASM POWER_LIMIT?

This is the parameter which controls the number of Allocation units the ASM instance will try to rebalance at any given time. In ASM versions less than 11.2.0.3 the default value is 11 however it has been changed to unlimited in later versions.

98. What are different types of redundancies in ASM & explain?
External redundancy,
Normal redundancy,
High redundancy.

99. What is stripping and mirroring.
Striping is spreading data across multiple disks so that IO is spread across multiple disks and hence increase in throughput. It provides read/write performance but fail over support.
ASM offers two types of striping, with the choice depending on the type of database file. Coarse striping uses a stripe size of 1MB, and you can use coarse striping for every file in your database, except for the control files, online redo log files, and flashback files. Fine striping uses a stripe size of 128KB. You can use fine striping for control files, online redo log files, and flashback files.
 Mirroring means redundancy. It may add performance benefit for read operations but overhead for write operations. It's basic purpose is to provide fail over support.
There are three ASM mirroring options:
High Redundancy - In this configuration, for each primary extent, there are two mirrored extents. For Oracle Database Appliance this means, during normal operations there would be three extents (one primary and two secondary) containing the same data, thus providing “high” level of protection. Since ASM distributes the partnering extents in a way that prevents all extents to be unable due to a component failure in the IO path, this configuration can sustain at least two simultaneous disk failures on Oracle Database Appliance (which should be rare but is possible).
Normal Redundancy - In this configuration, for each primary extent, there is one mirrored (secondary) extent. This configuration protects against at least one disk failure. Note that in the event a disk fails in this configuration, although there is typically no outage or data loss, the system operates in a vulnerable state, should a second disk fail while the old failed disk replacement has not completed. Many Oracle Database Appliance customers thus prefer the High Redundancy configuration to mitigate the lack of additional protection during this time.
External Redundancy - In this configuration there are only primary extents and no mirrored extents. This option is typically used in traditional non-appliance environments when the storage sub-system may have existing redundancy such as hardware mirroring or other types of third-party mirroring in place. Oracle Database Appliance does not support External Redundancy.

100. How does ASM provides Redundancy?
When you create a disk group, you specify an ASM disk group type based on one of the following three redundancy levels:
Normal for 2-way mirroring - When ASM allocates an extent for a normal redundancy file, ASM allocates a primary copy and a secondary copy. ASM chooses the disk on which to store the secondary copy in a different failure group other than the primary copy.
High for 3-way mirroring. In this case the extent is mirrored across 3 disks.
External to not use ASM mirroring. This is used if you are using Third party Redundancy mechanism like RAID, Storage arrays.

101. Can ASM instance and database (rdbms) be on different servers?
ASM instance and Database (rdbms) have to be present on same server. Otherwise it will not work.

102. Can we see the files stored in the ASM instance using standard unix commands.
No, you cannot see the files using standard unix commands like ls. You need to use utility called asmcmd to do this. This is present in 10.2 and above.e.g
/home/oracle>asmcmd
Asmcmd>
You can use help command to see the options.

103. What is SYSASM role?
Starting from Oracle 11g, SYSASM role can be used to administer the ASM instances. You can continue using SYSDBA role to connect to ASM but it will generate following warning messages at time of startup/shutdown, create Diskgroup/add disk ,etc
<span style="font-size: small; font-family: arial,helvetica,sans-serif;">Alert entry
WARNING: Deprecated privilege SYSDBA for command 'STARTUP'</span>

Starting from Oracle 11g, SYSASM role can be used to administer the ASM instances. You can continue using SYSDBA role to connect to ASM but it will generate following warning messages at time of startup/shutdown, create Diskgroup/add disk, etc

Alert entry

WARNING: Deprecated privilege SYSDBA for command 'STARTUP'

104. Is it mandatory to use disks of same size and characteristics for Diskgroups?
No, it is not mandatory to use the disks of same size and characteristics for Diskgroups though it is a Recommended Practice.
Same size disk for Failuregroups in Normal/High redundancy will prevent issues like ORA-15041 as the file extents needs to be mirrored across the disks. Also as Oracle distributes data based on capacity, so larger disk will have more data stored in it and which will result in higher I/O to disk and eventually can lead to sub-optimal performance.
Moreover having disks of different characteristic like varying disk speed can impact the performance.
When managing disks with different size and performance capabilities, best practice is to group them into disk groups according to their characteristics. So you can use higher speed disks for your database files while other disks can be part of Diskgroup used for Flash Recovery Area.

105. Do we need to install ASM and Oracle Database Software in different ORACLE_HOME?
No. Again installing ASM and Oracle Database Software in different ORACLE_HOME is not mandatory but a best practice. This is useful in cases when we need to have multiple databases using same ASM instance and you need to patch only one of them. E.g You need to apply a CBO patch to one of 10.2 database while your other 10.1 database using different installation does not require it. In this case having a ASM_HOME separate from 10.2 ORACLE_HOME will allow your 10.1 database to keep running. Thus this approach is useful for High Availability.

105. What is the maximum size of Disk supported by ASM?
ASM supports disks upto 2Tb, so you need to ensure that lun size should be less then 2Tb. 10.2.0.4 and 11g database will give error if you try to create a diskgroup with ASM disks having disk size >2Tb.

106. I have created Oracle database using DBCA and having a different home for ASM and Oracle Database. I see that listener is running from ASM_HOME. Is it correct?
This is fine. When using different home for ASM, you need to run the listener from ASM_HOME instead of ORACLE_HOME.

107. How do we identify if we are connected to Normal Instance or ASM instance?

Issue following command to identify this

SQL> show parameter instance_type

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

instance_type string asm

In case you are connected to ASM instance, it will display value as asm, otherwise it will display value as RDBMS.

108. Can we change the Redundancy for Diskgroup after its creation?

No, we cannot modify the redundancy for Diskgroup once it has been created. To alter it we will be required to create a new Diskgroup and move the files to it. This can also be done by restoring full backup on the new Diskgroup. Following metalink note describes the steps

Note.438580.1 – How To Move The Database To Different Diskgroup (Change Diskgroup  Redundancy)

109. Can we use ASM for storing Voting Disk/OCR in a RAC instance?

In oracle 11gR1 and below, you cannot use ASM for storing the voting disk and OCR. It is due to the fact that Clusterware starts before ASM instance and it should be able to access these files which is not possible if you are storing it on ASM. You will have to use raw devices or OCFS or any other shared storage.

In Oracle 11gR2 we can store them in ASM.

110. Does ASM instance automatically rebalances and takes care of hot spots?

No. This is a myth and ASM does not do it. It will initiate automatic rebalance only when a new disk is added to Diskgroup or we drop a disk from existing Diskgroup.

111. Can we use block devices for ASM Disks?

Yes. Starting from Oracle Database 10.2 block devices can be used directly for ASM Disks in Linux. This is not true for other Unix based systems where block devices are not supported yet.

Along with this it is recommended to use a Device mapping functionality so that disk mapping is preserved after disk failure. This is important when you have devices as /dev/sda,/dev/sdb,/dev/sdc and due to some reason the devices are not detected at next reboot (say /dev/sdb), the system will map the incorrect device (i.e /dev/sdc will be marked as /dev/sdb). You can use following methods for preserving disk names

-udev – the role of udev is to provide device persistency and naming consistency.This is especially important for the Oracle Cluster Registry (OCR) and Voting disks required by Oracle Clusterware.

-ASMLIB – ASMLIB will provide device management specifically for ASM disk devices.

 112. Describe about ASM architecture.

Automatic Storage Management (ASM) instance

Instance that manages the diskgroup metadata

Disk Groups

Logcal grouping of disks
Determines file mirroring options
ASM Disks
LUNs presented to ASM
ASM Files
Files that are stored in ASM disk groups are called ASM files, this includes database files

Notes:

Many databases can connect as clients to single ASM instances

ASM instance name should only be +ASM only

One diskgroup can serve many databases