Web lists-archives.com

Re: Sync two disks and hot swap




On 11/8/17 5:59 PM, David Christensen wrote:
On 11/08/17 02:49, Dominik George wrote:
Hi,

I have the following scenario:

  * A server with two hard drives in removable cases
  * A backup process writes data to both disks, making up a live backup server
  * A third disk is to be kept off-site
  * On a ergular basis, I want to hot-swap one of the disks, as in, remove
    one of the two synced disks and replace it with the stale off-site copy,
    and put the now recent copy off-site

I figure that a simple software RAID 1 would do the trick, but it is not
really made for it and would need some complex manual intervention in
order to not break the state on the removed disk.

Any ideas on how to achieve this, or arguments that RAID 1 would indeed
be a good solution?

Are the two drives in RAID (1?) or do they each have their own file system?


I have read articles about building a RAID 1 with three drives, migrating in data, pulling one drive and placing it off-site, operating in degraded mode on two drives, and then periodically re-installing the third drive, resilvering, pulling one drive and placing it off-site, and returning to degraded operations on two drives.  But STFW just now, I see a lot of posts with titles indicating this is a bad idea.


I have three drives in mobile dock drawers, each with LUKS and ext4. One is on-line in my backup server, one is near-site, and one is off-site.  Periodically, I put the near-site drive into the backup server, rsync the on-line drive to the near-site drive, remove the near-site drive, and then swap the near-site and off-site drives. Admittedly the wear is uneven, but it's KISS and it works.


But what I really want is some form of snapshot technology (rsync/hard link, LVM, btrfs, ZFS) with all the goodies -- realtime compression, realtime de-duplication, and encryption.  I need a more powerful backup server (many core processor with AES-NI, 16+ GB RAM, SSD caches, etc.).


David


I have used raid 1 to make a drive I can take off site for backup.  You just grow the raid 1 array by one disk and add the disk you want to take out (even on a usb/sata connection ... but slow).   Of course the disk or partition(s) need to be the same size as the array.  Let it sync and then boot to a live cd and you can fail and remove that drive.   Or just power down and remove the drive.   That way the embedded file system will be unmounted correctly.  I have then taken that one drive and connected it to another system and been able to run the raid 1 in degraded mode and mount the embedded file system(s) to get to the files.  To make the original raid happy just grow the array again setting the number of drives back to what it was originally (you can grow to a smaller number).  The syncing can be slow since every byte on the drive needs to be "synced" instead of just the space the files take up.

I tried to use btrfs in several VMs running debian but I kept having to delete snapshots to make sure I had enough free space.

--


...Bob