MultiDistroTools

Differences between revisions 1 and 2
Revision 1 as of 2006-01-04 07:51:10
Size: 5505
Editor: d213-103-255-197
Comment:
Revision 2 as of 2006-01-04 08:08:45
Size: 5368
Editor: d213-103-255-197
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= What is motutools ? = = What is MultiDistroTools ? =
Line 3: Line 3:
'''motutools''' is a package providing lots of small scripts useful to MOTUs and other people interested in improving the universe's quality. Currently, it particularly targets scripts dealing with lists of packages (all software written in ruby, all science-related software), allowing the user to get a ''global view'' of the packages it cares about. But the scope of motutools might be larger in the future. '''multidistrotools''' is a package providing lots of small scripts useful to MOTUs and other Debian derivatives developers. Currently, it particularly targets scripts dealing with lists of packages (all software written in ruby, all science-related software), allowing the user to get a ''global view'' of the packages it cares about. But the scope of multidistrotools might be larger in the future.
Line 5: Line 5:
The motutools package provides /usr/bin/mts (mts as in '''M'''otu'''T'''ool'''S'''). This executable is a wrapper for a lot of other small scripts which reside in /usr/share/motutools. Those scripts can be run directly, but it is easier to run them through ''mts''. The multidistrotools package provides /usr/bin/mdt. This executable is a wrapper for a lot of other small scripts which reside in /usr/share/multidistrotools. Those scripts can be run directly, but it is easier to run them through ''mdt''.
Line 8: Line 8:
Scripts in /usr/share/motutools should do only one thing, but do it well. Scripts longer than 50 lines should really be exceptions. Scripts in /usr/share/multidistrotools should do only one thing, but do it well. Scripts longer than 50 lines should really be exceptions.
Line 10: Line 10:
Scripts in /usr/share/motutools can be written in whatever language one want. However, limiting ourselves to a small set of languages and tools might be a good idea. The packages needed to run those scripts is listed as Recommends, because the user might not want to run all the scripts. Scripts in /usr/share/multidistrotools can be written in whatever language one want. However, limiting ourselves to a small set of languages and tools might be a good idea. The packages needed to run those scripts is listed as Recommends, because the user might not want to run all the scripts.
Line 14: Line 14:
Currently, motutools depends on ruby, because /usr/bin/mts is written in this language. However, /usr/bin/mts is less than 100 lines, so you can rewrite it in python if this dependancy bothers you too much. Currently, multidistrotools depends on ruby, because /usr/bin/mdt is written in this language. However, /usr/bin/mdt is less than 100 lines, so you can rewrite it in python if this dependancy bothers you too much.
Line 19: Line 19:
   * download Packages and Sources files for a distribution (Debian, Ubuntu) and a section (main, contrib, non-free, universe, multiverse) in a local folder (''~/.motutools/'' ?) '''done'''    * download Packages and Sources files for a distribution (Debian, Ubuntu) and a section (main, contrib, non-free, universe, multiverse) in a local folder (''~/.multidistrotools/'' ?) '''done'''
Line 37: Line 37:
(Working with motutools >= 0.0.9) (Working with multidistrotools >= 0.0.9)
Line 43: Line 43:
mts depends --reverse libruby1.8 > binpkg mdt depends --reverse libruby1.8 > binpkg
Line 46: Line 46:
mts bin2src < binpkg > srcpkg mdt bin2src < binpkg > srcpkg
Line 49: Line 49:
if [ ! -d $HOME/.motutools/sid ]; then
        mts dist-create sid http://ftp.debian.org/debian unstable main contrib non-free
if [ ! -d $HOME/.multidistrotools/sid ]; then
        mdt dist-create sid http://ftp.debian.org/debian unstable main contrib non-free
Line 52: Line 52:
if [ ! -d $HOME/.motutools/dapper ]; then
        mts dist-create dapper http://archive.ubuntu.com/ubuntu dapper main restricted universe multiverse
if [ ! -d $HOME/.multidistrotools/dapper ]; then
        mdt dist-create dapper http://archive.ubuntu.com/ubuntu dapper main restricted universe multiverse
Line 56: Line 56:
mts dist-apt-get sid update
mts dist-apt-get dapper update
mdt dist-apt-get sid update
mdt dist-apt-get dapper update
Line 60: Line 60:
mts compare-versions sid dapper > versionslist mdt compare-versions sid dapper > versionslist
Line 63: Line 63:
mts filter srcpkg < versionslist > rubyversionslist mdt filter srcpkg < versionslist > rubyversionslist
Line 66: Line 66:
mts versions2html < rubyversionslist > rubyversionslist.html mdt versions2html < rubyversionslist > rubyversionslist.html
Line 71: Line 71:
= motutools and motu-tools (with a dash): what's the difference ? =
motu-tools is a set of scripts written by ReinhardTartler, StephanHermann and others. Those scripts ({{{lpbugs.py, lpmerge.py}}}) aims at easing the merge process. At some point, they might (or not) be merged into motutools. This hasn't been discussed yet.
= multidistrotools and motu-tools (with a dash): what's the difference ? =
motu-tools is a set of scripts written by ReinhardTartler, StephanHermann and others. Those scripts ({{{lpbugs.py, lpmerge.py}}}) aims at easing the merge process. At some point, they might (or not) be merged into multidistrotools. This hasn't been discussed yet.
Line 76: Line 76:
 * If you are a MOTU, review the package on REVU to improve the packaging and get it into universe
 * If you have some personal scripts, start your own branch and add them
 * Test motutools, report problems, provide ideas for other scripts, etc (all of this will be easier when we will have a package in universe)
 * If you have some personal scripts, start your own branch and add them and ping me
 * Test multidistrotools, report problems, provide ideas for other scripts, etc (all of this will be easier when we will have a package in universe)
Line 81: Line 80:
 * LucasNussbaum's branch: {{{bzr branch http://ox.blop.info/bazaar/motutools/}}}  * LucasNussbaum's branch: {{{bzr branch http://ox.blop.info/bazaar/multidistrotools/}}}
Line 84: Line 83:
Build the package for a ["REVU"] upload: {{{dpkg-buildpackage -rfakeroot -S -sa -I.bzr}}}[[BR]]

What is MultiDistroTools ?

multidistrotools is a package providing lots of small scripts useful to MOTUs and other Debian derivatives developers. Currently, it particularly targets scripts dealing with lists of packages (all software written in ruby, all science-related software), allowing the user to get a global view of the packages it cares about. But the scope of multidistrotools might be larger in the future.

The multidistrotools package provides /usr/bin/mdt. This executable is a wrapper for a lot of other small scripts which reside in /usr/share/multidistrotools. Those scripts can be run directly, but it is easier to run them through mdt.

More info

Scripts in /usr/share/multidistrotools should do only one thing, but do it well. Scripts longer than 50 lines should really be exceptions.

Scripts in /usr/share/multidistrotools can be written in whatever language one want. However, limiting ourselves to a small set of languages and tools might be a good idea. The packages needed to run those scripts is listed as Recommends, because the user might not want to run all the scripts.

Scripts are coded in a smart way, allowing chaining. They should read their input from stdin as much as possible, and use stdout and stderr' for output.

Currently, multidistrotools depends on ruby, because /usr/bin/mdt is written in this language. However, /usr/bin/mdt is less than 100 lines, so you can rewrite it in python if this dependancy bothers you too much.

TODO / DONE list

Scripts needed:

  • Packages and Sources file management
    • download Packages and Sources files for a distribution (Debian, Ubuntu) and a section (main, contrib, non-free, universe, multiverse) in a local folder (~/.multidistrotools/ ?) done

    • Output those packages files for a distribution (Debian, Ubuntu) and a section (all of them, main, contrib, non-free, universe, multiverse)
    • Get a list of binary/source packages matching something from those Packages or Sources files:
      • Recursive dependancies of something (using Depends, Recommends or Suggests) done: depends (but still need more love to select distribution and dependancy type)

      • Recursive reverse dependancies of something (using Depends, Recommends or Suggests) done: depends

      • Section
      • any grep-dctrl predicate
  • Filter a file containing data with a list of packages done: filter

  • Build a list of binary packages from a list of source packages
  • Build a list of source packages from a list of binary packages done: src2bin

  • [http://en.wikipedia.org/wiki/Naive_set_theory Set theory] on package lists : union, intersection, difference, substraction, etc. So one can build exactly the list one want. done: list_*

  • Get a list of bugs in the Debian BTS, in the Ubuntu Bugzilla, or in Malone, for a list of packages
  • Compare versions of packages in Debian and Ubuntu
  • Get the popcon results for a list of binary packages. Also for a list of source packages, using the most installed binary package as reference.
  • Generate HTML reports for the list of bugs or the versions of packages
  • Any other cool stuff that you are going to add to this list now

Real life example

(Working with multidistrotools >= 0.0.9)

# because we will generate some temporary files
cd /tmp

# get all packages recursively depending on libruby1.8
mdt depends --reverse libruby1.8 > binpkg

# get the sources packages for those binary packages
mdt bin2src < binpkg > srcpkg

# Update our package lists (Packages and Sources files)
if [ ! -d $HOME/.multidistrotools/sid ]; then
        mdt dist-create sid http://ftp.debian.org/debian unstable main contrib non-free
fi
if [ ! -d $HOME/.multidistrotools/dapper ]; then
        mdt dist-create dapper http://archive.ubuntu.com/ubuntu dapper main restricted universe multiverse
fi

mdt dist-apt-get sid update
mdt dist-apt-get dapper update

# Lists versions in Debian and Ubuntu
mdt compare-versions sid dapper > versionslist

# Filter this versions list with the list of ruby-related sources packages
mdt filter srcpkg < versionslist > rubyversionslist

# Generate an HTML report from rubyversionslist
mdt versions2html < rubyversionslist > rubyversionslist.html

Here is [http://ox.blop.info/bazaar/rubyversionslist.html the resulting HTML for those commands].

multidistrotools and motu-tools (with a dash): what's the difference ?

motu-tools is a set of scripts written by ReinhardTartler, StephanHermann and others. Those scripts (lpbugs.py, lpmerge.py) aims at easing the merge process. At some point, they might (or not) be merged into multidistrotools. This hasn't been discussed yet.

Development info

How can you help ?

  • If you have some personal scripts, start your own branch and add them and ping me
  • Test multidistrotools, report problems, provide ideas for other scripts, etc (all of this will be easier when we will have a package in universe)

Branches

  • LucasNussbaum's branch: bzr branch http://ox.blop.info/bazaar/multidistrotools/

Building the package

Build the package: dpkg-buildpackage -rfakeroot -us -uc -I.bzrBR Read the README if you're interested in adding your scripts.


["CategoryMOTU"]

MultiDistroTools (last edited 2008-08-06 16:18:59 by localhost)