Ubuntu Wiki

This is an extract from SecurityTeam/BuildEnvironment that tries to make it as quick and easy as possible to reduce your pbuilder / PPA / distro failure rate and improve your build & turnaround time, as well as allow you to debug the builds in an environment close to the Ubuntu builders.

This has only been tested on current Saucy.

The original wiki page deals with much more details (especially around security) that are not required on a daily basis, but make it quite cumbersome to actually follow the guide. If anything here doesn't work, or is unclear, or you want more detail, check the other page - maybe there's help on your issue.

Setting up and using sbuild

Creating the schroots

1. Install sbuild and schroot:

   $ sudo apt-get install sbuild debhelper ubuntu-dev-tools

2. Make sure you are in the 'sbuild' group:

   $ sudo adduser $USER sbuild

3. Setup for mounting $HOME and extracting the ddebs:

   • Append to /etc/schroot/sbuild/fstab:

     /home/<username>/ubuntu/scratch             /scratch             none    rw,bind     0   0

     If you use an encrypted home directory, your $HOME is mounted differently,

     so you will also have to add:

     /home/<username>                            /home/<username>     none    rw,bind     0   0

4. Create ~/.sbuildrc:

   # Mail address where logs are sent to (mandatory, no default!)
   $mailto = 'username';
   
   # Name to use as override in .changes files for the Maintainer: field
   # (mandatory, no default!).
   $maintainer_name='Your Name <user@ubuntu.com>';
   
   # Directory for writing build logs to
   $log_dir="/home/<username>/ubuntu/logs";
   
   # don't remove this, Perl needs it:
   1;

   Then make the following directory (change if specified something different in ~/.sbuildrc):

   $ mkdir -p $HOME/ubuntu/logs

5. Create ~/.mk-sbuild.rc:

   SCHROOT_CONF_SUFFIX="source-root-users=root,sbuild,admin
   source-root-groups=root,sbuild,admin
   preserve-environment=true"
   SKIP_UPDATES="1"

6. Generate a GPG keypair for sbuild to use:

   $ sbuild-update --keygen

7. Finally, create the schroots (this assumes x86_64 host, adapt as necessary):

   $ sg sbuild   # only needed if you were added to the sbuild group during this session
   # add --debootstrap-proxy=http://127.0.0.1:3142/ if you have e.g. apt-cacher-ng around
   $ mk-sbuild saucy --arch=amd64 --skip-updates \
     --debootstrap-mirror=http://archive.ubuntu.com/ubuntu --distro=ubuntu

   See man mk-sbuild for details.

8. Optional: It can often be useful to perform builds in /dev/shm, which is a tmpfs directory in RAM. While there are limitations on what you can compile and the number of concurrent builds that can be performed (based on how much memory your build machine has), building in /dev/shm can lead to much faster build times and reduce disk I/O. A way to create a 'shared' source schroot like so:

   $ sudo sh -c "sed -e 's#]#-shm]#' \
     -e 's#^\(directory=.*\)#\1\nunion-overlay-directory=/dev/shm/schroot/overlay#' \
     /etc/schroot/chroot.d/* > /etc/schroot/chroot.d/shm-overlays.conf"
   $ schroot --list

   Please note that the directory you specify for 'union-overlay-directory' must exist before using the shm chroot (can add an entry to /etc/rc.local). It might also be useful to remount /dev/shm with more memory than the default, which is 50% of RAM. Can adjust like so: sudo mount -o remount,size=75% /dev/shm.

Deleting a schroot

1. Remove the stanza for the chroot:

   $ rm /etc/schroot/chroot.d/sbuild-saucy-amd64

2. Remove the chroot from the disk:

   $ sudo rm -rf /var/lib/schroot/chroots/saucy-amd64

3. Optional Run the shm snippet above.

Using the schroot

• Using a schroot is similar to the chroot command but you specify the chroot and user you want to use rather than the directory.

  $ schroot -c saucy-amd64 -u root

  Or without root with:

  $ schroot -c saucy-amd64

  Or building via sbuild directly:

  $ apt-get source foo
  $ cd ./foo-*
  # do work...
  $ dch -i
  $ update-maintainer
  $ sbuild -d saucy-amd64
  # or from a source package
  $ bzr bd -S
  $ sbuild -d saucy-amd64 ../foo_*.dsc

Maintaining the schroots

• Can see a listing of all your chroots with:

  $ schroot -l
  chroot:saucy-amd64
  chroot:saucy-amd64-shm
  source:saucy-amd64
  source:saucy-amd64-shm
  ...

  The 'source:' chroot is the pristine chroot and you shouldn't go into it unless you want to change something for all future schroots into the chroot. Eg, if you wanted to always have 'vim' installed in your saucy-amd64 chroot, use:

  $ schroot -c source:saucy-amd64 -u root
  $ apt-get install vim
  $ exit

  Or, to upgrade the schroot (you shouldn't do that with PPAs enabled unless you have a "special" chroot for a certain PPA setup):

  $ schroot -c source:saucy-amd64 -u root
  $ apt-get update
  $ apt-get upgrade
  $ exit

Expiring active schroot sessions

Sometimes needed if schroots are left hanging around due to a crash or reboot. You can expire all active schroot sessions with:

$ schroot -e --all-sessions

Other hints, tips, and tricks

Mount your home dir

• To mount your home directory in the chroot, so that you can use files from there, add this entry to /etc/schroot/sbuild/fstab:

  /home/<username>                            /home/<username>     none    rw,bind     0   0

  Note: this is a read-write bind mount, so changes will be written to your real home directory.

Parallel builds

• To make use of all your machine's power, you can use the usual -jN option for sbuild:

  $ sbuild -d saucy-amd64 -j4

Temporarily adding PPAs

• To add a PPA without "polluting" the chroot, you can:

• Disable gpg key checking - add /etc/apt/apt.conf.d/80unauthenticated:

  $ schroot -c source:saucy-amd64 -u root
  $ echo "APT::Get { AllowUnauthenticated "1"; };" > /etc/apt/apt.conf.d/80unauthenticated
  $ exit

• Add the PPA, but don't update:

  $ schroot -c source:saucy-amd64 -u root
  $ echo "deb http://ppa.launchpad.net/ubuntu-unity/daily-build/ubuntu saucy main " > \
    /etc/apt/sources.list.d/ubuntu-unity-daily-build-saucy.list
  $ exit

  You can also access the source chroot directly in /var/lib/schroot/chroots/saucy-amd64/.

Local packages

• You sometimes need newer packages (libraries, whatnot) to build against. The Incredible BarryWarsaw did some nice things to help with that: lp:~barry/+junk/repotools.

• Branch repotools to your scratch:

  $ bzr branch lp:~barry/+junk/repotools ~/ubuntu/repo

• Install sponge:

  $ sudo apt-get install moreutils

• Modify clean.sh and scan.sh to set the repository path:

  ...
  where=/home/<username>/ubuntu/repo
  ...

• Add the following snippet to your ~/.sbuildrc:

  $external_commands = {
      "pre-build-commands" => [
          ['/home/<username>/ubuntu/repo/scan.sh'],
      ],
      "chroot-setup-commands" => [
          ['/home/<username>/ubuntu/repo/prep.sh'],
      ],
  };

• And add the following lines to /etc/schroot/sbuild/fstab:

  # Expose local apt repository to the chroot
  /home/<username>/ubuntu/repo    /repo    none   rw,bind  0 0

• Drop packages (with .dsc) into /home/<username>/ubuntu/repo and you're good to go!

ARM

• TBD: see how to cross-build or put it all on an ARM device with some persistency.

CCache

• TBD: especially useful for cross and on-device builds, ccache could speed things up even more.

CategoryPackaging