ASM Filter Driver — simple test on filtering

Franck Pachot's picture

Here is a simple test on ASM Filter Driver showing that when filtering is enabled the disks presented to ASM are protected from external writes.

On a 12.2 Grid Infrastructure installation I have my disks labeled with ASM Filter Driver (AFD):

ASMCMD> afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'VM106'
ASMCMD> afd_lsdsk
----------------------------------------------
Label Filtering Path
==============================================
VDI100M0001 ENABLED /dev/sdh
VDI100M0002 ENABLED /dev/sdg
VDI100M0003 ENABLED /dev/sdi
VDI100M0004 ENABLED /dev/sdj
VDI1G0001 ENABLED /dev/sde
VDI1G0002 ENABLED /dev/sdf
VDI1G0003 ENABLED /dev/sdd
VDI1G0004 ENABLED /dev/sdc
VDI1G0005 ENABLED /dev/sdk
VDI1G0006 ENABLED /dev/sdl
...

lsblk is a good way to see all block devices, those mounted as a filesystem, but also those labeled for ASM:

lsblk --paths -o NAME,KNAME,FSTYPE,LABEL,MOUNTPOINT,SIZE,OWNER,GROUP,MODE,ALIGNMENT,MIN-IO,OPT-IO,PHY-SEC,LOG-SEC,ROTA,SCHED,RQ-SIZE,WSAME

I have created a diskgroup, named FRANCK, with only one disk: /dev/sde labeled as VDI1G0001

ASMCMD> lsdsk -G FRANCK
Path
AFD:VDI1G0001

Then I’ve created a tablespace with its datafile on this diskgroup, and a table in this tablespace, inserting ‘Hello Franck!’ which is a string easy to recognize with no need to check the extents block offset:

create tablespace FRANCK datafile '+FRANCK';
create table FRANCK (x varchar2(30)) tablespace FRANCK;
insert into FRANCK values ('Hello Franck!');
commit;
alter system checkpoint;

Nothing is encrypted, the disk is small (1GB) and then I can simply use grep to find my character string:

[root@VM106 ~]# grep --byte-offset --only-matching --text "Hello Franck" /dev/sde
97574895:Hello Franck

I displayed the offset, and can get more detail about the binary data stored around here with od:

[root@VM106 ~]# od -t c --skip-bytes=$(( 97574895 - 10 )) --read-bytes=40 --width=80  /dev/sde
564157745  \0  \0  \0  \0  \0  \0   , 001 001  \r   H   e   l   l   o       F   r   a   n   c   k   ! 001 006   ) 334 006 242  \0  \0 207  \0 200  \a   ( 334   $  \0  \0

As I have the offset, I can easily corrupt the file with dd, writing 5 chr(0) in the middle:

[root@VM106 ~]# dd of=/dev/sde if=/dev/zero seek=$(( 97574895 + 5 )) bs=1 count=5 conv=notrunc
5+0 records in
5+0 records out
5 bytes (5 B) copied, 0.000164038 s, 30.5 kB/s

No errors here. The five write() calls were successful. However we can see the following in /var/log/messages

[root@VM106]# Aug 31 23:19:43 VM106 kernel: Buffer I/O error on device sde, logical block 23821
Aug 31 23:19:43 VM106 kernel: lost page write due to I/O error on sde
Aug 31 23:19:43 VM106 kernel: sd 4:0:2:0: [sde] Incomplete mode parameter data
Aug 31 23:19:43 VM106 kernel: sd 4:0:2:0: [sde] Assuming drive cache: write through
Aug 31 23:19:43 VM106 kernel: sde: unknown partition table

This is the kernel informing us that the writes were lost. This is where AFD has filtered the writes() which were not recognized as what Oracle processes are expected to do. The 23821 logical block is where my offset is: 97574895/4096=23821.99…

If I check back my file, nothing has changed:

[root@VM106 ~]# od -t c --skip-bytes=$(( 97574895 - 10 )) --read-bytes=40 --width=80  /dev/sde
564157745  \0  \0  \0  \0  \0  \0   , 001 001  \r   H   e   l   l   o       F   r   a   n   c   k   ! 001 006   ) 334 006 242  \0  \0 207  \0 200  \a   ( 334   $  \0  \0
564160015

Here I tested with 5 small writes. I got only one set of messages on /var/log/messages. I get the same if I try to create a partition, or a filesystem, on this disk device. This is the goal of ASM Filter Driver: be sure that nobody thinks that the device is not used because they don’t know about ASM, and tries to format it.

Now I disable the filtering to show the normal behaviour:

ASMCMD> afd_filter -d --all
ASMCMD> afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'DISABLED' on host 'VM106'
ASMCMD> afd_lsdsk
----------------------------------------------
Label Filtering Path
==============================================
VDI100M0001 DISABLED /dev/sdh
VDI100M0002 DISABLED /dev/sdg
VDI100M0003 DISABLED /dev/sdi
VDI100M0004 DISABLED /dev/sdj
VDI1G0001 DISABLED /dev/sde
...

and run the same writes:

[root@VM106 ~]# dd of=/dev/sde if=/dev/zero seek=$(( 97574895 + 5 )) bs=1 count=5 conv=notrunc
5+0 records in
5+0 records out
5 bytes (5 B) copied, 0.000124124 s, 40.3 kB/s

Here is what I have on /var/log/messages:

Sep  1 22:03:19 VM106 kernel: sd 4:0:2:0: [sde] Incomplete mode parameter data
Sep 1 22:03:19 VM106 kernel: sd 4:0:2:0: [sde] Assuming drive cache: write through
Sep 1 22:03:19 VM106 kernel: sde: unknown partition table

Nothing about I/O error and page write lost.

Without AFD filtering, the diskgroup is corrupt:

[root@VM106 ~]# od -t c --skip-bytes=$(( 97574895 - 10 )) --read-bytes=40 --width=80  /dev/sde
564157745 \0 \0 \0 \0 \0 \0 , 001 001 \r H e l l o \0 \0 \0 \0 \0 c k ! 001 006 ) 334 006 242 \0 \0 207 \0 200 \a ( 334 $ \0 \0

And, of course, those unexpected \0 are detected as corruption:

SQL> select * from franck;
select * from franck
*
ERROR at line 1:
ORA-01578: ORACLE data block corrupted (file # 30, block # 133)
ORA-01110: data file 30: '+FRANCK/CDB1/DATAFILE/franck.256.985734495'

To prevent automated spam submissions leave this field empty.