ASM disk header
ASM disk header is probably the best known
piece of ASM metadata. Chances are you learned about
it when it was damaged or lost and hopefully Oracle Support was able to get you
up and running. In this post I will try to
explain why ASM disk header is important and what it contains.
Block zero
ASM disks are formatted into Allocation Units. Some Allocation units contain ASM metadata and some contain database data. Allocation units that contain ASM metadata are formatted into ASM metadata blocks. Allocation unit 0 is at the beginning of an ASM disk and it always contain ASM metadata. The very first block (block 0) of Allocation Unit 0 contains the ASM disk header.
ASM disk header contents
Most of the data in the ASM disk header is of interest to that disk only. But some information in the ASM disk header is relevant to the whole disk group and some is even relevant to the whole cluster!
Block zero
ASM disks are formatted into Allocation Units. Some Allocation units contain ASM metadata and some contain database data. Allocation units that contain ASM metadata are formatted into ASM metadata blocks. Allocation unit 0 is at the beginning of an ASM disk and it always contain ASM metadata. The very first block (block 0) of Allocation Unit 0 contains the ASM disk header.
ASM disk header contents
Most of the data in the ASM disk header is of interest to that disk only. But some information in the ASM disk header is relevant to the whole disk group and some is even relevant to the whole cluster!
[oracle@rac1 log]$ ls /dev/oracleasm/disks/
DISK1 DISK2 DISK3
DISK4 DISK5
[oracle@rac1 log]$ kfed read /dev/oracleasm/disks/DISK1
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002:
KFBTYP_DISKHEAD
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 0 ; 0x004: T=0 NUMB=0x0
kfbh.block.obj: 2147483648 ; 0x008: TYPE=0x8 NUMB=0x0
kfbh.check: 2101694786 ; 0x00c:
0x7d455142
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr:
ORCLDISKDISK1 ; 0x000: length=13
kfdhdb.driver.reserved[0]: 1263749444 ; 0x008: 0x4b534944
kfdhdb.driver.reserved[1]: 49 ; 0x00c: 0x00000031
kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000
kfdhdb.compat: 186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 1 ; 0x026:
KFDGTP_EXTERNAL
kfdhdb.hdrsts: 3 ; 0x027:
KFDHDR_MEMBER
kfdhdb.dskname: DISK1 ; 0x028: length=5
kfdhdb.grpname: OCR ; 0x048: length=3
kfdhdb.fgname: DISK1 ; 0x068: length=5
kfdhdb.capname: ; 0x088: length=0
kfdhdb.crestmp.hi:
32984819 ; 0x0a8: HOUR=0x13
DAYS=0x17 MNTH=0x3 YEAR=0x7dd
kfdhdb.crestmp.lo: 3511297024 ; 0x0ac: USEC=0x0
MSEC=0x289 SECS=0x14 MINS=0x34
kfdhdb.mntstmp.hi: 32984844 ; 0x0b0: HOUR=0xc
DAYS=0x18 MNTH=0x3 YEAR=0x7dd
kfdhdb.mntstmp.lo: 2744026112 ; 0x0b4: USEC=0x0
MSEC=0x3a1 SECS=0x38 MINS=0x28
kfdhdb.secsize: 512 ; 0x0b8: 0x0200
The result of the above kfed command shows
us that this ASM block has two types of data - block header data - prefixed with kfbh, and ASM disk header data - prefixed with kfdhdb. In fact, every ASM metadata block will have the block header data plus the data specific to its block type.
Important ASM metadata block 0 header data
kfbh.endian: 1 ; 0x000: 0x01
kfbh.type: 1 ; 0x002:
KFBTYP_DISKHEAD
kfbh.block.blk: 0 ; 0x004: T=0 NUMB=0x0
ASM disk header backup
Oracle 11g
md_backup
md_backup
is ran from the ASMCMD> prompt. It very simply creates a file that is a
backup of your metadata, for all of your disk groups, or just a few if you want
to specify. By default, it writes to the current directory and backs up all of
the mounted disk groups header files.
Arguments:
-b and –g
-g
Specifies the disk group to back up
Example: ASMCMD> md_backup –b /mydir/somebackupfile –g DG1 –g DG3
md_restore
md_restore is a bit more involved than md_backup. As you may have
guessed, it restores previously backed up metadata to disk groups you specifiy.
Arguments:
-b –i –t –f –g –o
-b
Specifies the backup file to read
-i
Ignores errors. (Normally, if md_restore hits an error, it stops. This argument
suppressed that.
-t
Specifies the type of disk group to be created/restored. “full” creates a disk
group and restores the metadata. “nodg” restores metadata only, and “newdg”
creates a new disk group with a different name using in conjunction with the –o
argument.
-o
Renames the disk group
-f
writes the SQL commands to be performed in the restore to a file instead of
executing them (sql_script_file).
-g
selects the disk groups to be restored. If none are specified, it restores all
of them.
Example: ASMCMD> md_restore –t full –g DG1 –i backup_file
This
will restore disk group DG1 using the specified file.
Example: ASMCMD> md_restore –t nodg –g DG1 –i backup_file
This
will restore an existing disk group’s metadata.
Oracle
10g
When implementing ASM backups should not only be made of the
data within ASM but the disk headers should also be backed up. Oracle 10g does
not provide a way out of the box to do this but Oracle 11g provides two new
commands in the asmcmd utility to backup disk headers. The header of a disk is
comprised of the first 4096 bytes. While not supported it is possible to use
the UNIX dd to read and write these bytes to a backup file.
dd if=device name of=unique header file bs=4096 count=1
That will capture the first 4k block of the ASM file. You would do this for each file, creating a separate backup for each. A simple dd will write it back in to the header, if you need to restore from your backup. Example:
dd if=/dev/rdisk/asmvol1 of=/u01/ora_backup/asmvo1_header.bak bs=4096 count=1
This would capture the first 4k of the given ASM file. Example:
dd if=/u01/ora_backup/asmvol1_header.bak of=/dev/rdisk/asmvol1
This would write in the backed up header back into the file it was backed up from.
It is always a good idea to keep backups of your disk group metadata, because no one can know when the corruption gremlin might strike.
dd if=device name of=unique header file bs=4096 count=1
That will capture the first 4k block of the ASM file. You would do this for each file, creating a separate backup for each. A simple dd will write it back in to the header, if you need to restore from your backup. Example:
dd if=/dev/rdisk/asmvol1 of=/u01/ora_backup/asmvo1_header.bak bs=4096 count=1
This would capture the first 4k of the given ASM file. Example:
dd if=/u01/ora_backup/asmvol1_header.bak of=/dev/rdisk/asmvol1
This would write in the backed up header back into the file it was backed up from.
It is always a good idea to keep backups of your disk group metadata, because no one can know when the corruption gremlin might strike.
I talked about this in my post on kfed, but let's do that again - get those
values from the block header and calculate the second last block number in
allocation unit 1:
$ ausize=`kfed read /dev/oracleasm/disks/ASMD1 | grep
ausize | tr -s ' ' | cut -d' ' -f2`
$ blksize=`kfed read /dev/oracleasm/disks/ASMD1 | grep blksize | tr -s ' ' | cut -d' ' -f2`
$ let n=$ausize/$blksize-2
$ echo $n
254
$ kfed read /dev/oracleasm/disks/ASMD1 aun=1 blkn=254
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 0 ; 0x004: T=0 NUMB=0x0
kfbh.block.obj: 2147483648 ; 0x008: TYPE=0x8 NUMB=0x0
kfbh.check: 473773689 ; 0x00c: 0x1c3d3679
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr: ORCLDISKASMD1 ; 0x000: length=13
...
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname: ASMD1 ; 0x028: length=5
kfdhdb.grpname: DATA ; 0x048: length=4
kfdhdb.fgname: ASMD1 ; 0x068: length=5
...
$ blksize=`kfed read /dev/oracleasm/disks/ASMD1 | grep blksize | tr -s ' ' | cut -d' ' -f2`
$ let n=$ausize/$blksize-2
$ echo $n
254
$ kfed read /dev/oracleasm/disks/ASMD1 aun=1 blkn=254
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 0 ; 0x004: T=0 NUMB=0x0
kfbh.block.obj: 2147483648 ; 0x008: TYPE=0x8 NUMB=0x0
kfbh.check: 473773689 ; 0x00c: 0x1c3d3679
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr: ORCLDISKASMD1 ; 0x000: length=13
...
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname: ASMD1 ; 0x028: length=5
kfdhdb.grpname: DATA ; 0x048: length=4
kfdhdb.fgname: ASMD1 ; 0x068: length=5
...
So we see the same contents as in block 0 in allocation
unit 0.
This can be very handy when the disk header is damaged or
lost. All we have to do is run kfed repair [disk_name], and specify the
allocation unit size if the value is not default (1MB). But as I said in the
kfed post, please do not do this on your own - seek Oracle Support assistance
if you suspect problems with ASM disk header.
ASM disk header in Exadata
ASM disks in Exadata are not exposed to the OS via device names. Instead they can be accessed via special name - "o/[IP address]/[disk name]". The kfed understands that syntax, so we can still use it in Exadata.
Let's have a look at the ASM disk header on an Exadata disk:
$ kfed read o/192.168.10.9/DBFS_DG_CD_03_exadatacel01
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
...
kfdhdb.dskname:DBFS_DG_CD_03_EXADATACEL01 ; 0x028: length=26
kfdhdb.grpname: DBFS_DG ; 0x048: length=7
kfdhdb.fgname: EXADATACEL01 ; 0x068: length=12
...
kfdhdb.ausize: 4194304 ; 0x0bc: 0x00400000
...
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
...
kfdhdb.dskname:DBFS_DG_CD_03_EXADATACEL01 ; 0x028: length=26
kfdhdb.grpname: DBFS_DG ; 0x048: length=7
kfdhdb.fgname: EXADATACEL01 ; 0x068: length=12
...
kfdhdb.ausize: 4194304 ; 0x0bc: 0x00400000
...
Some Exadata specific values in the ASM disk header are as follows:
ASM disk name that consists of the disk group name
(DBFS_DG), cell disk label (CD), cell disk number (3) and the storage cell name
(exadatacel01)
Failgroup name is the same as the storage cell name
Default allocation unit size in Exadata is 4 MB
Very useful blog, thank you. -Iftequar Mohammed
ReplyDelete