DRBD 8.3.0

DRBD 8.3.0 installation & prerequisites

For drbd testing, two working Ubuntu Server Jaunty installations are needed. This can be done with ubuntu-vm-builder or tests can be done on real hardware. Both servers need to be able to reach each other over the network.

Each server needs one additional partition (additional to the one for the system). The size of partition isn't relevant for testing, but both partitions on servers need to be of the same size.

For the purpose of this tutorial, I'll name them drbd-1 and drbd-2. Servers need to be able to resolve each others hostnames, so you should either have DNS or enter hostnames to /etc/hosts manually. Since drbd can start before dhcp client gets an IP, you should set up both servers with static IPs.

/etc/hosts:

127.0.0.1 localhost
192.168.0.1 drbd-1
192.168.0.2 drbd-2

From this point you should do everything as root (sudo -i).

Next, install drbd8-utils package.

Configuration

drbd has one single configuration file - /etc/drbd.conf. This file should be identical on both servers. This file should look like this:

global { usage-count no; }
common { syncer { rate 100M; } }
resource r0 {
        protocol C;
        startup {
                wfc-timeout  15;
                degr-wfc-timeout 60;
        }
        net {
                cram-hmac-alg sha1;
                shared-secret "test";
                allow-two-primaries;
        }
        on drbd-1 {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.0.1:7788;
                meta-disk internal;
        }
        on drbd-2 {
                device /dev/drbd0;
                disk /dev/sdb1;
                address 192.168.0.2:7788;
                meta-disk internal;
        }
}

Once you've created this file on both servers, on both of them run command:

drbdadm create-md r0

This will create r0 resource on both servers. Then, choose one server to be your primary server (drbd-1), and mark it:

drbdadm -- --overwrite-data-of-peer primary all

As soon as you run this command, the other server (drbd-2) will start syncing data. You can check the progress with:

cat /proc/drbd

You should get something like this:

version: 8.3.0 (api:88/proto:86-89)
GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by ivoks@ubuntu, 2009-01-17 07:49:56
 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r---
    ns:76524 nr:0 dw:0 dr:84352 al:0 bm:4 lo:1 pe:11 ua:245 ap:0 ep:1 wo:b oos:429772
       [==>.................] sync'ed: 16.2% (429772/505964)K
       finish: 0:00:22 speed: 19,048 (19,048) K/sec

Once syncing is done, this will change to:

GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by ivoks@ubuntu, 2009-01-17 07:49:56
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
    ns:505964 nr:0 dw:0 dr:505964 al:0 bm:31 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

Tests

A: ext3 in primary/secondary

Create ext3 fileystem on primary server (drbd-1):

mkfs.ext3 /dev/drbd0

Mount special device /dev/drbd0 to /mnt/drbd and copy a file to it:

mkdir /mnt/drbd
mount -t ext3 /dev/drbd0 /mnt/drbd
cp /etc/hosts /mnt/drbd

Unmount the partition and degrade primary to secondary (drbd-1):

umount /mnt/drbd
drbdadm secondary r0

On the other server (drbd-2), promote it to primary, mount drbd device and check copied filed:

drbdadm primary r0
mkdir /mnt/drbd
mount -t ext3 /dev/drbd0 /mnt/drbd
ls -d /mnt/drbd/*

NOTE: At this point, primary is no longer drbd-1, but drbd-2, and secondary is drbd-1.

B: secondary off

NOTE: Run test A first.

Shutdown secondary server (drbd-1). Check /proc/drbd:

cat /proc/drbd

You should see that the other server is unavailable:

version: 8.3.0 (api:88/proto:86-89)
GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by ivoks@ubuntu, 2009-01-17 07:49:56
 0: cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown C r---
    ns:11 nr:530744 dw:530767 dr:561 al:2 bm:31 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:24

Create couple of files on primary server (drbd-2):

cp /etc/hosts.* /etc/group /mnt/

Now, power up secondary server (drbd-1). As soon as it boots, it will start syncing data. If you are testing will small partitions, sync will be faster. On primary server (drbd-2), /proc/drbd should look like this:

GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by ivoks@ubuntu, 2009-01-17 07:49:56
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
    ns:63 nr:530744 dw:530788 dr:614 al:2 bm:33 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

C: primary off

NOTE: Run test A first. Running test B isn't needed for this one, but it doesn't hurt.

Shutdown primary server (drbd-2) device /dev/drbd0 mounted. /proc/drbd on secondary (drbd-1), after few seconds, should look like this:

version: 8.3.0 (api:88/proto:86-89)
GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by ivoks@ubuntu, 2009-01-17 07:49:56
 0: cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r---
    ns:0 nr:53 dw:53 dr:0 al:0 bm:2 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

Make it primary:

drbdadm primary r0

NOTE: From this point, primary is again drbd-1, and secondary is again drbd-2.

Now, you should be able to mount /dev/drbd0 on primary (drbd-1):

mount -t ext3 /dev/drbd0 /mnt/drbd

Once secondary (drbd-2) boots up again, it won't connect to primary, since it was primary before the power failure and they can't decide which will be primary again. This is called split-brain situation. On secondary (drbd-2) run this commands:

drbdadm secondary r0
drbdadm -- --discard-my-data connect r0

And on the primary (drbd-1) run this command:

drbdadm connect r0

This will bring them back in sync. cat /proc/drbd:

version: 8.3.0 (api:88/proto:86-89)
GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by ivoks@ubuntu, 2009-01-17 07:49:56
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
    ns:8192 nr:0 dw:62 dr:8598 al:2 bm:4 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

Test results