• Block devices connected over the network (drbd or nbd devices) are not covered here.
• Alternative aproaches include the syncing of regular (possibly network mounted) filesystems (unison, ChironFS) or replicating file systems (GlusterFS, OpenAFS, coda, intermezzo, ...).

HotplugRaid

A replicating raid (redundant array of independent disks) that holds the user data is a generally useful setup. A hotplugable raid with one ore more external drives is especially usefull for home or office laptop users. This page provides a How-To and a basic test case.

Home directories on HotplugRaid

Layman's Description:

Consider you are doing critical work on a laptop computer. At home, or when on AC, you plug in one (or possibly several) of your USB hard disks containing mirror partitions. Upon insertion the lights of those drives will stay lit during idle times for a little while. This is the mirror partition being synced in the background automatically. As long as those drives stay attached any work you save will be written simultaneously to the laptops internal disk, as well as to the attached mirror partitions.

If the internal disk fails, crashes down the stairway together with your laptop when you are on the road, or it drives away in that fake taxi drivers trunk, well hopefully you have at least one up-to-date mirror partition on that drive in your backpack, or on the one you kept at home.

Technical Description:

It's an installation with a filesystem residing on a multi-disk (md) raid array that contains hot-plugable devices like removable USB and Firewire drives or (possibly external / bay-mounted) (S)ATA harddrives.

It is possible to have the entire root file system (/) on a hot-plugable raid or as in the example installation below just the home directories (/home).

Beginning with ubuntu 9.10 the arrays are properly assembled using udev rules and the --incremental option. Recovery of temporarily disconected (external) drives should now work out of the box.

Installation Instructions

1. Attach all external drives that you want to hold raid partitions.
2. Begin the install process by booting from the alternate-cd.
3. Install using manual partitioning.

   • (The basic process is described more elaborately at https://help.ubuntu.com/community/Installation/SoftwareRAID)

   • Since this setup and test case assumes that the installation is done on a laptop machine the system we'll also set up encryption. (This can be left out if encryption is not desired.) First create a rather small (primary) partition that will hold the /boot files. (500M is more than needed.) Configure it to be an ext2 file system and be mounted as /boot. Then create a (logical) partition that will hold the rootfs and swap partition on the internal drive. Configure it to be used for dm-crypt with luks header. Enter the encryption setup to set the passphrase for it. Then configure the resulting _crypt device to be used as a physical volume for lvm. Enter the lvm setup to create a volume group (in this case vg_internal_hostname) and the logical volumes for the rootfs and swap (lv_root, lv_swap). Then configure lv_root to be mounted as / and lv_swap to be used as swap. Now setup equally sized (logical) partitions on the internal drive and each external drive to be used as physical raid partitions for the home directories. Choose to configure raid (multi-disks) from the menu and create a raid 1 (mirroring) device from the partitions you just created. (md0 will be assumed here.) The multi-disk raid partitions should start syncing in the background. Go on to create a partition on the md0 device and configure it to be used for dm-crypt with luks header. Enter the encryption setup to set the passphrase for it. Finally create an ext4 partition on the md0_crypt device and configure it to be mounted as /home. (Note that unlike with the first *_crypt device there is no lvm used on top of the second *_crypt device. If you want it to hold multiple logical volumes it is of course possible to create a lvm volume group (vg_mirrored_hostname) containing logical volumes (lv_home, lv_svn, ...).) The partitioning of this example can be summarized as:

     [boot partition] plain device -> ext2 (/boot)
     [system partition] first_crypt device -> lvm {ext4 (/) and swap}
     [multi disk member partitions] -> md0 device -> second_crypt device -> ext3 (/home)

     (During subsequent install attempts or when the disks already contain some lvm, raid or luks partitions you may encounter problems with the installer not being able to delete or use preexisting devices (with data) or inadvertently scraping previous setups. If you encounter this please file/report on appropriate bugs.)

4. If the HotplugRaid is used for the home directories (i.e. not too much i/o, the little slow down does not matter) as in this example the re-syncing can be accelerated using an "internal bitmap". If the array has not been set up with the "-b internal" option, you can "grow" one afterwards with "mdadm --grow /dev/mdX -b internal")

   • Note: If you have multiple external members and want to use them conneced one at a time (rolling backup style) you may like to have a bitmap for each member. For this you need to stack md arrays of two members. The first array (i.e. md0) consists of an internal member and one external member. The second array (i.e. md1) consists of md0 and the second external member.

5. Booting with a degraded array (see BootDegradedRaid).

   • When you detach a removable device with a raid member partition from the machine when it is powered down, the degraded array may not come up automatically during boot.

     With 9.10: Do a "dpkg-reconfigure mdadm" after the install and set boot degraded to yes again. (462258) The bootscripts need to wait for devices and time out after a while to start degraded arrays. To start the array manually if you are droped to a console:

     mdadm --incremental --run /dev/sde5 # Assemble and run the device (in this case sde5) as a degraded raid.
     cryptsetup luksOpen /dev/mdX my-rescued-mirror # Open the encrypted md device using its luks header.
     mount /dev/mapper/my-rescued-mirror /mountpoint # Finally mount your data.

6. Resyncing reconnected drives (readding raid member partitions to its array) when they are re-attached.
   • Since 9.10 uses "mdadm --incremental" in the udev rules this now works automatically. If a reconnected partitions that got marked faulty still needs to (do they?) be re-added manually: "mdadm --add /dev/md0 $dev" will manually re-add the partition to the raid and start the syncing. A custom udev rule may be similar to this one:

     ACTION=="add", BUS=="usb", SUBSYSTEM=="block", DEVPATH=="*[0-9]" SYSFS{serial}=="5B6B1B916C31", RUN+="mdadm --add /dev/md0 $DEVNAME" 

   • But the above rule actually identifies the USB device by its serial number and not the raid partition by superblock/uuid (which you may have moved to another device).

7. Check if the system will survive the sleep when you suspend the computer. 230671

To check the status of the arrays: cat /proc/mdstat

Further Improvements

• Maybe enabling the mdadm options --write-mostly --write-behind for external drives.
• Having the root file system and swap on one encrypted lvm volume-group and /home on a second encrypted raid device, means you have to enter passphrases twice on each boot and resume. With fast devices like external SATA Drives you can include the full root file system and swap partition into the raid mirror.
• You may keep your /boot partition on a raid and have it mirrored to external drives,
• maybe including a read only medium as a preferred mirror, showing discrepancies if the internal drive has been tempered with. Maybe even allowing you to boot from that external drive, possibly re-syncing the internal /boot partition and master boot records (MBR).

References

https://help.ubuntu.com/community/Installation/SoftwareRAID https://features.launchpad.net/distros/ubuntu/+spec/udev-mdadm https://blueprints.launchpad.net/ubuntu/+spec/udev-mdadm https://blueprints.edge.launchpad.net/ubuntu/+spec/boot-degraded-raid https://blueprints.launchpad.net/ubuntu/+spec/udev-lvm-mdadm-evms-gutsy

CategoryBootAndPartition CategoryUsb CategoryRecovery