Web lists-archives.com

Re: Sync two disks and hot swap

On 11/9/17 2:01 AM, David Christensen wrote:
On 11/08/17 17:44, Bob Weber wrote:
On 11/8/17 5:59 PM, David Christensen wrote:
I have read articles about building a RAID 1 with three drives, migrating in
data, pulling one drive and placing it off-site, operating in degraded mode on
two drives, and then periodically re-installing the third drive, resilvering,
pulling one drive and placing it off-site, and returning to degraded
operations on two drives.  But STFW just now, I see a lot of posts with titles
indicating this is a bad idea.
But what I really want is some form of snapshot technology (rsync/hard link,
LVM, btrfs, ZFS) with all the goodies -- realtime compression, realtime
de-duplication, and encryption.  I need a more powerful backup server (many
core processor with AES-NI, 16+ GB RAM, SSD caches, etc.).

I have used raid 1 to make a drive I can take off site for backup.  You just
grow the raid 1 array by one disk and add the disk you want to take out (even on
a usb/sata connection ... but slow).   Of course the disk or partition(s) need
to be the same size as the array.  Let it sync and then boot to a live cd and
you can fail and remove that drive.   Or just power down and remove the drive.
That way the embedded file system will be unmounted correctly.  I have then
taken that one drive and connected it to another system and been able to run the
raid 1 in degraded mode and mount the embedded file system(s) to get to the
files.  To make the original raid happy just grow the array again setting the
number of drives back to what it was originally (you can grow to a smaller
number).  The syncing can be slow since every byte on the drive needs to be
"synced" instead of just the space the files take up.

Okay.  What RAID technology were you using -- LVM, mdadm, btrfs, ZFS, other?

I use software raid with mdadm.  Its pretty forgiving with powering down and removing a drive (after sync) and growing the array back down to the original size.  I mainly do this on my backup server running backuppc.  The files are compressed and hard linked between backups if they have not changed.  This makes any other type of offsite backup pretty hard ... rsync just ran out of memory.  So adding a drive to the raid 1 and syncing is easy in comparison.  Having grub make the drive bootable  (since the os is also on a raid 1 partition) makes the drive very easy to just install in new hardware and get going again (assuming the original backup system was destroyed).