Web lists-archives.com

Re: Auto-assembled SW RAID in rescue mode




At boot, I was prompted with a BIOS message saying that there was no boot device.


No, a BIOS upgrade doesn't modify fstab. I believe that EFI has anti-tampering mechanisms that might have been triggered by the BIOS upgrade. (At least it's currently my best guess. It is in line with the fact that a clean install after this BIOS upgrade has no problem.)


If you think of Debian installer rescue mode as a command promt, then I agree that it won't change fstab by itself.

However, Debian's installer rescue mode does indeed contain functionality that can modify fstab for you:


When entering rescue mode, you are prompted to "Select the partitions to assemble into a RAID array. If you select 'Automatic', then all devices containing RAID physical volumes will be scanned and assembled. Note that a RAID partition at the end of a disk may sometimes cause that disk to be mistakenly detected as containing a RAID physical volume. In that case, you should select the appropriate partitions individually." Here I have selected "Automatic" and in the subsequent screens I have chosen to mount /boot/efi and selected /dev/md0 as root partition. I don't remember what I did after this, but I think that the ensuing option to write new grub failed which caused me to try other ways.


Either way, at least the menu option for graphical rescue mode lets you navigate to a listing of possible operations that correspond to the regular installation sequence. One of these operations is partitioning. Manual partitioning presents a correct listing of my RAID volumes, although as "do not use" (and hence without a mount point). When assigning and writing "ext4" and mount point / for /dev/md0, swap for /dev/md1, and "ext4" /storage for /dev/md2 I could finally write new grub. I suspect that it is this step that somehow transmogrified my single-partition /dev/md2 (up to this point referred to as /dev/md2) into a /dev/md2 RAID gpt volume with a partition now referred to as /dev/md2p1.


The output from fdisk -l and parted -l:


fdisk -l:

Disk /dev/sdc: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B2E029B3-65EE-44F8-9EE6-A363E25AA8D0

Device        Start        End    Sectors  Size Type
/dev/sdc1      2048     391167     389120  190M Microsoft basic data
/dev/sdc2    391168   78516223   78125056 37.3G Linux RAID
/dev/sdc3  78516224 7814035455 7735519232  3.6T Linux RAID


Disk /dev/sdf: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 8D67E618-9BF8-45F5-91F9-1B41779274D6

Device        Start        End    Sectors  Size Type
/dev/sdf1      2048     391167     389120  190M Microsoft basic data
/dev/sdf2    391168   78516223   78125056 37.3G Linux RAID
/dev/sdf3  78516224 7814035455 7735519232  3.6T Linux RAID


Disk /dev/sde: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 66151BB0-196C-49F4-AF7A-594FE4BF98AF

Device        Start        End    Sectors  Size Type
/dev/sde1      2048     391167     389120  190M Microsoft basic data
/dev/sde2    391168   78516223   78125056 37.3G Linux RAID
/dev/sde3  78516224 7814035455 7735519232  3.6T Linux RAID


Disk /dev/sdd: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B532E854-8DFE-4DCD-B887-BC3FD13F8755

Device        Start        End    Sectors  Size Type
/dev/sdd1      2048     391167     389120  190M Microsoft basic data
/dev/sdd2    391168   78516223   78125056 37.3G Linux RAID
/dev/sdd3  78516224 7814035455 7735519232  3.6T Linux RAID


Disk /dev/sdb: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 5910E337-8CDD-4DD2-9457-084A617CA599

Device        Start        End    Sectors  Size Type
/dev/sdb1      2048     391167     389120  190M EFI System
/dev/sdb2    391168   78516223   78125056 37.3G Linux RAID
/dev/sdb3  78516224 7814035455 7735519232  3.6T Linux RAID


Disk /dev/sda: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 39F8743A-E279-4DF2-9775-4ABF3742E013

Device        Start        End    Sectors  Size Type
/dev/sda1      2048     391167     389120  190M EFI System
/dev/sda2    391168   78516223   78125056 37.3G Linux RAID
/dev/sda3  78516224 7814035455 7735519232  3.6T Linux RAID


Disk /dev/md1: 37.2 GiB, 39966474240 bytes, 78059520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/md0: 37.2 GiB, 39966474240 bytes, 78059520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/md2: 18 TiB, 19802258145280 bytes, 38676285440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 524288 bytes / 2621440 bytes
Disklabel type: gpt
Disk identifier: 41440B8B-C8DC-489C-9F03-E85D8D309302

Device     Start         End     Sectors Size Type
/dev/md2p1  2048 38676283391 38676281344  18T Linux filesystem


Disk /dev/sdg: 7.2 GiB, 7761035264 bytes, 15158272 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x3e847943

Device     Boot Start     End Sectors  Size Id Type
/dev/sdg1  *        0 7743263 7743264  3.7G  0 Empty
/dev/sdg2       21468   22299     832  416K ef EFI (FAT-12/16/32)

---

parted -l:

Model: ATA HGST HDN724040AL (scsi)
Disk /dev/sda: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  200MB   199MB   fat32              boot, esp
 2      200MB   40.2GB  40.0GB                     raid
 3      40.2GB  4001GB  3961GB                     raid


Model: ATA HGST HDN724040AL (scsi)
Disk /dev/sdb: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  200MB   199MB   fat32              boot, esp
 2      200MB   40.2GB  40.0GB                     raid
 3      40.2GB  4001GB  3961GB                     raid


Model: ATA HGST HDN724040AL (scsi)
Disk /dev/sdc: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  200MB   199MB   fat16              msftdata
 2      200MB   40.2GB  40.0GB                     raid
 3      40.2GB  4001GB  3961GB                     raid


Model: ATA HGST HDN724040AL (scsi)
Disk /dev/sdd: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  200MB   199MB   fat16              msftdata
 2      200MB   40.2GB  40.0GB                     raid
 3      40.2GB  4001GB  3961GB                     raid


Model: ATA HGST HDN724040AL (scsi)
Disk /dev/sde: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  200MB   199MB   fat16              msftdata
 2      200MB   40.2GB  40.0GB                     raid
 3      40.2GB  4001GB  3961GB                     raid


Model: ATA HGST HDN724040AL (scsi)
Disk /dev/sdf: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  200MB   199MB   fat16              msftdata
 2      200MB   40.2GB  40.0GB                     raid
 3      40.2GB  4001GB  3961GB                     raid


Model: Linux Software RAID Array (md)
Disk /dev/md2: 19.8TB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  19.8TB  19.8TB  ext4


Model: Linux Software RAID Array (md)
Disk /dev/md0: 40.0GB
Sector size (logical/physical): 512B/4096B
Partition Table: loop
Disk Flags:

Number  Start  End     Size    File system  Flags
 1      0.00B  40.0GB  40.0GB  ext4


Warning: The driver descriptor says the physical block size is 2048 bytes,
but Linux says it is 512 bytes.


Ignore/Cancel? I
Model: SanDisk Cruzer Blade (scsi)
Disk /dev/sdg: 31.0GB
Sector size (logical/physical): 2048B/512B
Partition Table: mac
Disk Flags:

Number  Start   End     Size   File system  Name   Flags
 1      2048B   6143B   4096B               Apple
 2      11.0MB  11.4MB  426kB               EFI


Model: Linux Software RAID Array (md)
Disk /dev/md1: 40.0GB
Sector size (logical/physical): 512B/4096B
Partition Table: loop
Disk Flags:

Number  Start  End     Size    File system     Flags
 1      0.00B  40.0GB  40.0GB  linux-swap(v1)






From: Pascal Hambourg <pascal@xxxxxxxxxxxxxxx>
Sent: Friday, May 11, 2018 9:15 AM
To: debian-user@xxxxxxxxxxxxxxxx
Subject: Re: Auto-assembled SW RAID in rescue mode
 
Le 10/05/2018 à 23:30, Niclas Arndt a écrit :
>
> I received unofficial BIOS files from Gigabyte with new versions of Intel ME and CPU microcode. After upgrade, several Stretch machines no longer boot. (Fresh installation works without any problem.)

What happened at boot time ?
I doubt that all you describe below is related to a motherboard firmware
upgrade.

> I booted from USB in rescue mode, did auto-assembly of RAID arrays, mounted /boot/efi and rewrote grub.
>
> This made it possible to boot, but /dev/md2 was removed from fstab. I also for the first time noticed that there is an additional /dev/md2p1 entry when running blkid (although not in fstab).

Neither a BIOS upgrade nor Debian installer's rescue mode modify fstab.

> Now /dev/md2 has a PTUUID of PTTYPE="gpt" and
>
> /dev/md2p1 has a UUID of TYPE="ext4" with a PARTUUID.

It seems to indicate that the RAID array /dev/md2 has a GPT partition
table defining a partition md2p1 containing an ext4 filesystem. This is
not the usual setup, but software RAID has supported partitioned arrays
for a long time.

Can you post the output of "fdisk -l" or "parted -l" ?

> Manual mount of /dev/md2 works, but the machine won't boot if I add an fstab entry with the /dev/md2 PTUUID. With the /dev/md2p1 UUID in fstab it is mounted at startup and the machine boots.

It is extremely surprising that mounting /dev/md2 works because
according to the above, /dev/md2 contains a partition table, not a
filesystem.

Also, the filesystem should be identified in fstab with its UUID or
label because a RAID device name is not stable.

> mdadm refers to the volume as /dev/md2. (I don't use lvm.)

mdadm manages only RAID arrays, not partitions inside an array. It just
marks the array as partitioned and the kernel handles the partition
table as it does with any other partitioned block device.