<> {{{#!irc [07:00] <@dholbach> GOOD MORNING EVERYBODY! [07:00] <@dholbach> who do we have here for the Packaging Training session? :) [07:01] o/ [07:01] It's morning? [07:01] genii: in crazy UTC people time, yeah. 11pm here :( [07:01] <@dholbach> genii: it is over here - 08:01 right now :-) [07:02] moin moin [07:02] <@dholbach> so... do we have any questions already before we kick off the session? [07:03] nope [07:03] <@dholbach> alright... :-) [07:03] * jost86 gets Coffee... [07:03] <@dholbach> https://wiki.ubuntu.com/MOTU/GettingStarted is the one page you absolutely need to bookmark :) [07:04] <@dholbach> it contains links to all important documentation [07:04] <@dholbach> like the Packaging Guide, like the Developer Videos, Ubuntu Developer process documentation, simple bugs to start working on etc [07:05] <@dholbach> ok [07:05] <@dholbach> which version of Ubuntu are you all running? [07:05] jaunty [07:05] Hardy [07:05] Jaunty :) [07:05] jaunty [07:06] 8.04.2 [07:06] <@dholbach> alright :) [07:06] hardy [07:06] breezy [07:06] kidding.. jaunty [07:06] jaunty with a karmic install available for dual-booting :-) [07:06] upgraded 8.10, to 9.04 alpha6, to beta, to rc. install that has some issues from alpha6, so will eventualy clean install [07:06] ubuntu cambridge [07:06] <@dholbach> if you want to work on Ubuntu, you absolutely need to have the latest development release [07:07] <@dholbach> https://wiki.ubuntu.com/UbuntuDevelopment/UsingDevelopmentReleases explains how to do that in a sane way :) [07:07] <@dholbach> I personally had chroots (explained on there as well) for a long time, but now I almost exclusively use kvm (used virtualbox in the past) [07:08] woowoo [07:08] <@dholbach> you absolutely need to be able to test packages in the ubuntu version you're about to upload them for :-) [07:08] teach me ol' great one! [07:09] <@dholbach> nixternal: come on... you've been around long enough already :) [07:09] * genii feeds nixternal some pie to quieten him down [07:09] mmmmm [07:09] <@dholbach> for now, let's just keep using the version we're all comfortable with, but please make a note to set up a VM at some stage :-) [07:09] <@dholbach> please run: [07:09] <@dholbach> sudo apt-get install ubuntu-dev-tools devscripts build-essential gnupg [07:10] <@dholbach> it will install a bunch of packages for you we're going to need for the session [07:10] isn't gnupg installed by default already? [07:10] <@dholbach> maybe nixternal can exaplain what they're there for [07:10] <@dholbach> nixternal: just making sure :) [07:10] k, confused me for a sec [07:10] if we're packaging for the current release, we won't need the latest dev release, right? [07:10] what! none of them is installed by default [07:10] I have to beg out, I'll be /away but checking the backscroll later [07:10] <@dholbach> raylu: for now we'll just play around with the tools, set things up and so on [07:11] raylu: correct, though you never know when you will want to start developing for the next release [07:11] <@dholbach> raylu: if you want to upload packages for karmic, you need to have them built and tested on karmic :) [07:11] <@dholbach> raylu: I hope that makes sense [07:11] * nixternal notes to fix it in the dev cycle and backport/sru as needed to the current stable release [07:11] * ara remembers that if there is someone that prefer to make questions in Spanish, they can be posted at #ubuntu-classroom-chat-es [07:11] thanks ara! [07:12] dholbach: right; just that you said we "absolutely" need the latest dev release [07:12] * nixternal goes back to watching and hushes up [07:12] <@dholbach> raylu: right, "absolutely" for testing :-) [07:12] <@dholbach> and "in some form" :) [07:12] <@dholbach> ok [07:12] <@dholbach> so this is what the packages are for: [07:12] "Please select the mail server configuration type that best meets your needs." =/ [07:13] <@dholbach> rww: you can purge mailx and the mailserver afterwards - sorry [07:13] <@dholbach> or use --no-install-recommends [07:13] <@dholbach> (in the apt-get call) [07:13] rww: just do the top selection, no configuration for now [07:13] <@dholbach> - gnupg (installed by default as nixternal said) can sign and encrypt files (we use it to sign source packages) [07:14] <@dholbach> - build-essential: will install the bare minimum to be able to compile (very simple packages): the compiler, make, etc. [07:15] <@dholbach> - devscripts: a bunch of tools that make Debian packaging much easier (especially repetitive tasks) [07:15] <@dholbach> - ubuntu-dev-tools: much like devscripts, but more closely tied to Ubuntu [07:15] <@dholbach> any questions so far? [07:15] nope [07:16] I got here late [07:16] using --no-install-recommends (or setting it in /etc/apt/apt.conf.d/) is highly advisable for a development chroot, not just because it saves download/configuration time, but because that's a more accurate model of how the buildds work [07:16] are there logs anywhere? [07:16] <@dholbach> teknico: sudo apt-get install --no-install-recommends ubuntu-dev-tools devscripts build-essential gnupg (is where we are right now) [07:17] <@dholbach> plus a bunch of links that will be available on https://wiki.ubuntu.com/Packaging/Training later on [07:17] thx [07:17] <@dholbach> ok, please edit ~/.pbuilderrc in your favourite editor and add the following [07:17] <@dholbach> COMPONENTS="main universe multiverse restricted" [07:17] <@dholbach> and then save the file [07:17] teknico: http://paste.ubuntu.com/161169/ [07:18] <@dholbach> we often use a tool called pbuilder which will build source packages in a minimal environment for us (sets up that environment, installs the build-depends packages, builds the package, etc in an automated fashion) [07:19] <@dholbach> for now just run [07:19] <@dholbach> pbuilder-dist create (hardy, jaunty, karmic, whatever) [07:19] <@dholbach> it will set up the initial pbuilder [07:19] <@dholbach> the nice thing about pbuilder-dist is, that you can set up different build environments for karmic, jaunty, intrepid, hardy, etc etc :) [07:20] <@dholbach> (also Debian versions) [07:20] <@dholbach> so that's pretty neatr [07:20] <@dholbach> neat [07:20] dholbach: if you already have a pbuilder environment setup, will this overwrite it? [07:20] <@dholbach> https://wiki.ubuntu.com/PbuilderHowto has more info about that tool [07:20] <@dholbach> Crusher: did you use pbuilder-dist? [07:20] dholbach: are "create" and "" switched around? I'm getting an error [07:20] dholbach: no [07:20] <@dholbach> rww: they are :-) [07:20] <@dholbach> pbuilder-dist create (hardy, jaunty, karmic, whatever) [07:20] <@dholbach> thanks a lot rww [07:21] * dholbach didn't sleep very well last night *yawn* [07:21] <@dholbach> Crusher: in that case it shouldn't, but you can safely skip the step as well [07:21] <@dholbach> ok... while that's running, let's set up a GPG key [07:21] <@dholbach> who of you does not have a GPG key set up yet? [07:22] i might have one... but I dont remember it [07:22] <@dholbach> awesome, looks like we can skip a step :) [07:22] well go ahead [07:22] <@dholbach> jost86: what does ls ~/.gnupg/pubring.gpg say? [07:22] <@dholbach> do you have that file? [07:23] yep, present [07:23] <@dholbach> super [07:23] <@dholbach> basically I would have run you through the steps of: [07:23] <@dholbach> gpg --gen-key [07:24] <@dholbach> gpg --send-keys [07:24] <@dholbach> and putting it up on Launchpad [07:24] i'd "gpg -K" to make sure you actually have a key [07:24] <@dholbach> raylu: good one - great [07:24] k, i think i'll figure that out :) thx [07:25] <@dholbach> if you didn't put up your GPG key on Launchpad, you can do it here: https://launchpad.net/people/+me/+editpgpkeys [07:26] <@dholbach> in the beginning you won't need the GPG key very often, as you're not directly allowed to upload packages to Ubuntu yet [07:26] <@dholbach> but I'll get back to the "upload packages part" in a bit [07:26] <@dholbach> if you use bash (default shell), please add something like this to your ~/.bashrc [07:26] <@dholbach> export DEBFULLNAME='Daniel Holbach' [07:26] <@dholbach> export DEBEMAIL='daniel.holbach@ubuntu.com' [07:27] <@dholbach> these are variables that are used by the tools in devscripts and it'll save you typing your name over and over again :) [07:27] <@dholbach> (editing changelog entries is one of the very common use-cases) [07:28] <@dholbach> another thing you need to make sure when you're working on Ubuntu packages is that you have source package repositories enabled in apt [07:28] <@dholbach> so you need something like this in your /etc/apt/sources.list [07:28] <@dholbach> deb-src http://archive.ubuntu.com/ubuntu/ jaunty restricted main multiverse universe [07:28] <@dholbach> (of course replace jaunty with karmic or hardy or whatever you're using) [07:29] <@dholbach> you can also use the System -> Administration -> Software Sources tool [07:29] <@dholbach> (or whatever it's called in English, sorry :-)) [07:29] <@dholbach> once you've configured that, you can simply run [07:30] <@dholbach> sudo apt-get update [07:30] <@dholbach> apt-get source hello [07:30] <@dholbach> and it will download you the current source code for that package from the archive [07:30] <@dholbach> any questions up until now? [07:31] <@dholbach> so let's run [07:31] <@dholbach> apt-get source hello [07:31] <@dholbach> it's a very very boring package, but we'll download it anyway now [07:31] <@dholbach> apt is going to download the following files: [07:31] <@dholbach> hello_2.2-3.diff.gz [07:32] <@dholbach> hello_2.2-3.dsc [07:32] <@dholbach> hello_2.2.orig.tar.gz [07:32] <@dholbach> (might be other versions on your system) [07:32] <@dholbach> let's go through them one by one [07:33] <@dholbach> the .orig.tar.gz file is the original pristine untouched tarball that was downloaded from the website of the software authors [07:33] <@dholbach> in this case it's from something like ftp.gnu.org [07:33] <@dholbach> it's important that we don't change it (only very rare cases where it's justified), but merely rename it to something like [07:33] <@dholbach> _.orig.tar.gz [07:34] <@dholbach> the .diff.gz file contains the compressed set of changes we need to apply to make the package build the Ubuntu or Debian way [07:34] <@dholbach> that's "the packaging" [07:35] <@dholbach> basically we add a directory called ./debian/ to the source which further describes the package, the history of the packaging, the copyright and so on [07:36] <@dholbach> when I said "make it build the Ubuntu/Debian way", I meant that we wrap a build process around the existing build system that the upstream package offers [07:36] <@dholbach> in the standard case of a C+autotools project that would be something like ./configure; make; sudo make install [07:36] <@dholbach> in a python+setuptools case that would be python setup.py install (or something like that) [07:37] <@dholbach> the reason for that is: we want just one way to build packages :-) [07:37] <@dholbach> the .dsc file contains just some bits of metadata, like the md5sum of the files, etc. [07:37] <@dholbach> are there any questions up until now? [07:38] dholbach: under what circumstances do we make native packages? [07:38] <@dholbach> Crusher: good question [07:38] dholbach: I have a directory hello-2.2 with a bunch of files in it too, that you didn't mention... [07:38] <@dholbach> rww: will get to that in a sec [07:38] <@dholbach> it seems that Crusher has read a few chapters of the packaging guide already :-) [07:38] dholbach: oh, okay. Just checking I didn't do something wrong :) [07:39] ;) [07:39] <@dholbach> rww: no, you're doing great :-) [07:39] after you answer that... what if their source relies on a directory called "debian" already? [07:39] <@dholbach> so in the case of a native package, you just have a _.tar.gz and a _.dsc file [07:40] <@dholbach> which means all Packaging changes are in the .tar.gz file too [07:40] <@dholbach> so you can't easily tell apart which change was done by the packager and which came from upstream [07:41] <@dholbach> also it means that for every small packaging change (update the package description for example) you need to upload the whole tarball (might be several MB) [07:42] <@dholbach> in the other case you just upload the .diff.gz and the .dsc (ie: 1.2-1: upload the .tar.gz, 1.2-2: just upload the .diff.gz) [07:42] <@dholbach> ok, so why do we use that option? [07:43] <@dholbach> in some cases we are upstream of the package (and don't do regular .tar.gz releases) and keep everything in svn or bzr or git or whatever [07:43] <@dholbach> in that case, especially if it's a small package, we just dump everything in that .tar.gz file and make it a native package [07:44] <@dholbach> Crusher: did that make sense? [07:44] dholbach: yes [07:44] <@dholbach> raylu: some upstream maintainers decide to ship their own Debian packaging, so there's a debian/ directory already [07:45] <@dholbach> that's one of the rare cases where we repackage the tarball and rip out the existing packaging [07:45] <@dholbach> it makes reading the .diff.gz much easier [07:45] what about asking upstream to release a prestine tarball without a debian directory? [07:45] <@dholbach> nixternal: very good point, it makes a lot of sense to do that [07:45] <@dholbach> but in the meantime that's probably what you do [07:46] <@dholbach> alright... cracking on and getting to rww's question [07:46] <@dholbach> what about the hello-2.2 directory [07:46] <@dholbach> "apt-get source" (actually it's dpkg-source -x) is a clever tool - what it did for us was [07:46] <@dholbach> - extract the tarball [07:47] <@dholbach> - use the .diff.gz and patch the code with our packaging changes [07:47] <@dholbach> (and check the md5sums along the way) [07:47] <@dholbach> so if you check out the hello-2.2 directory, you'll see that there's a debian/ directory already [07:47] <@dholbach> now let's test-build the package [07:47] <@dholbach> please run [07:47] <@dholbach> pbuilder-dist build hello_*.dsc [07:48] <@dholbach> this might take some time [07:48] <@dholbach> and I hope you all get addicted of watching build messages fly by and get involved with Ubuntu development :-) [07:49] if i wanted that, i'd be using gentoo [07:49] <@dholbach> haha :) [07:49] <@dholbach> ~/pbuilder/*_result/ should now contain the resulting .deb file [07:50] <@dholbach> raylu: right but in our case, you do one fix and 10 million other users benefit from it [07:51] <@dholbach> if you check out https://wiki.ubuntu.com/PackagingGuide/Recipes you'll find a bunch of examples of what you can do with the packaging tools and perform simpler tasks like "upgrade package to a new upstream", "make a small change", etc. [07:51] <@dholbach> once you created a small patch, you can use the Sponsorship Process to get it included in Ubuntu [07:51] <@dholbach> https://wiki.ubuntu.com/SponsorshipProcess has all the gory details [07:51] <@dholbach> so what's that for? [07:52] <@dholbach> you're only allowed to upload Ubuntu source packages directly when you're part of the development team [07:52] <@dholbach> until then you need to get your patches reviewed and then uploaded by somebody else [07:53] <@dholbach> so what you do is: attach the patch to a bug report on Launchpad (or link to the source package you uploaded somewhere) [07:53] its not as scary as it sounds [07:53] <@dholbach> then subscribe the sponsors team [07:53] <@dholbach> and they'll review the patch, give you feedback, etc. [07:53] <@dholbach> then apply the patch to the source package, sign it with their GPG key and upload it for you [07:53] <@dholbach> imbrandon: exactly [07:53] <@dholbach> the sponsors team is a really friendly bunch and you'll learn a lot that way [07:54] <@dholbach> once you've been through that process a couple of times, people will tell you that you should join the team, and then it's time to send in a short "application" [07:54] <@dholbach> https://wiki.ubuntu.com/UbuntuDevelopers has more details about that [07:54] <@dholbach> did that scare you? :-) [07:54] <@dholbach> do you have any questions about the process? [07:55] * jost86 dreams of being rich and famous ;) [07:55] o/ [07:55] the pbuilder command built a .deb package [07:55] <@dholbach> teknico: so you're scared? or do you have a question? :) [07:55] <@dholbach> teknico: yep [07:55] the dsc file is no more signed, though [07:56] how come? [07:56] dholbach: "patch" in the above means debdiffs we create with https://wiki.ubuntu.com/PackagingGuide/Recipes/Debdiff , right? [07:57] <@dholbach> teknico: I usually don't use the .dsc files from *result/, but I guess it's to make sure you don't upload them accidentally [07:57] teknico: correct you'll have to source build and resign after pbuilder if testing goes ok [07:57] <@dholbach> teknico: dput will refuse to upload them for you (unless you force it to) [07:57] rww: correct [07:57] <@dholbach> rww: a general patch should be fine, but a debdiff (with a nice changelog entry) will speed things up [07:58] <@dholbach> that's one of the most important points: always document your work properly [07:58] <@dholbach> next week we're going to have a session about PPAs, so Dustin Kirkland will talk a bit about how to upload packages to Launchpad [07:59] <@dholbach> that should be interesting and tie in quite well with this session [07:59] <@dholbach> any more questions? :) [07:59] can you point me to docs on how to package shared libs ? [07:59] <@dholbach> https://wiki.ubuntu.com/MOTU/GettingStarted should contain all the links I mentioned earlier [07:59] <@dholbach> djails: hang on [08:00] <@dholbach> http://www.debian.org/doc/debian-policy/ch-sharedlibs.html is the debian policy section for it [08:00] <@dholbach> and we had a session about that already, let me find it [08:00] thanks [08:00] <@dholbach> https://wiki.ubuntu.com/MOTU/School/LibraryPackaging [08:01] awesome ! [08:01] <@dholbach> it's been a while since that session, but most things should not have changed [08:01] <@dholbach> also I'd recommend checking out a few very simple other library packages [08:01] <@dholbach> libsexy for example [08:01] <@dholbach> any other questions? [08:02] dholbach: is there anything specific about packaging perl packages? [08:02] <@dholbach> ok cool - feel free to join #ubuntu-motu and ask all questions there - the people in there are really friendly and will help you out for sure [08:03] <@dholbach> sbeattie: I'm sure there is, I just did one in the past and used perlmodule.mk (part of CDBS) and made sure I used ${perl:Depends} somewhere [08:03] <@dholbach> sbeattie: that's all I know [08:04] dholbach: okay, thanks. [08:05] <@dholbach> ok... thanks a lot everybody - it's been a great session! [08:05] * Rail is going to request sponsorship for some packages :) [08:05] <@dholbach> I had a lot of fun and hope to see you guys around more often from now on! [08:05] <@dholbach> Rail: awesome! [08:05] dholbach: thanks, i ll try the REVU process for my pkg [08:05] dholbach: thank you very much [08:05] <@dholbach> djails: perfect [08:06] dholbach: Are the various videos on the ubuntudevelopers youtube account still up-to-date (e.g. http://www.youtube.com/watch?v=zKLabbXTqMc ), or have there been any changes in the last year or so to watch out for? [08:06] <@dholbach> have a great day everybody and hope to see you at the next sessions too :) [08:06] <@dholbach> rww: there should be only small changes like change "hardy" to "jaunty" or something [08:06] <@dholbach> rww: in principle it should be all the same still [08:07] dholbach: excellent, thanks :) [08:07] <@dholbach> rock on everybody! :-) [08:07] thanks for the session dholbach, you have a great day aswell. [08:07] thx [08:10] * thekorn hugs dholbach, great session [08:10] <@dholbach> gracias thekorn [08:11] dholbach: Very useful, thanks. }}} ---- CategoryPackaging