Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad Entry: package-level-tests
Packages affected:
Placeholder spec to be written. Work itself nearly implemented.
Summary
Run package-level tests (lintian and piuparts) on the Ubuntu archives and report problems.
Release Note
This spec or related does not need to be explicitly mentioned in the release notes.
Rationale
Catch package and upgrade problems early.
Use Cases
- Adalmiina maintains some packages in Ubuntu, and wants to do a good job. To achieve high quality, makes use of automatic testing tools. Even though she runs lintian and piuparts herself, she also gets benefit from having them run centrally, since the centralized service sometimes catches problems after her upload, when other packages change (sometimes the testing tools themselves).
- Bartholomew likes to work on QA on the entire distribution, and uses lintian and piuparts reports to find out about problems.
Design
Lintian and piuparts are existing tools which work pretty well already. This spec is about making use of them, not about developing them.
There needs to be a machine on which to run the programs. The results should be published on a web page. Eventually, automatic filing of bugs for serious issues would be beneficial, so that they are not lost among all the other results. The results are published as web pages similar to http://lintian.debian.org for lintian, and as raw log files as for http://piuparts.cs.helsinki.fi (which Debian uses).
The machine should be in the Canonical data center.
Implementation
- Lars sets up the programs to run on his home machine, publishing things on people.ubuntu.com.
- Piuparts should be run in a Xen or Qemu instance, and from cron.
- When a machine in the data center is available, the service is moved there.
Test/Demo Plan
Outstanding Issues
- Both piuparts and lintian make assumptions about the archive, mainly that everything interesting is in "main". This is not true for Ubuntu, requiring some adjustments to the programs. For lintian, this is done, but for piuparts, some additional work is still required to allow testing of multiple sections of the archive.
- Piuparts does not yet run from cron.
- Waiting for the machine in the data center to arrive.
Notes
http://people.ubuntu.com/~liw/lintian/ is the temporary location for lintian output.
http://people.ubuntu.com/~liw/piuparts/ is the temporary location for piuparts output.
https://lists.ubuntu.com/archives/ubuntu-devel/2007-December/024862.html is Lars's initial report of the lintian and piuparts pages to ubuntu-devel. It includes a brief tutorial on reading piuparts log files.