Launchpad Entry: qa-cert-testing-proposed
Created: 2008-12-05
Contributors: cr3
Summary
Uploads of certain packages to the -proposed archive should trigger a full test run on all certified hardware. We can currently do this manually but should add more automation.
Release Note
Rationale
When an upload lands in -proposed we want to test it for regressions across all available hardware.
Use Cases
- A new kernel is uploaded to -proposed which triggers an install + test run on all the hardware in the Canonical test labs.
- A new xorg is uploaded to -proposed which triggers an install and video+input tests on all laptops (and some desktops?)
- Steve is testing an SRU for a hardware-specific issue. He searches the hardware certification website and finds a system with the relevant hardware. From the website he can boot and install the system with -proposed enabled. He logs in via KVM and performs his testing manually.
Assumptions
Design
Triggering
By uploads to -proposed
An upload to -proposed of one of the following packages flipps a proposed-test-run flag and set a timer:
linux, xorg, lbm, udev, hal, ... XXX
XXX How do we detect -proposed uploads, parse the mailing-list?
- packages determine the machine-type and teststo be queued
linux, udev, hal: all machines
xorg: laptops (and some desktops?) with resolution tests
XXX - need complete mapping
- After a set period (say, 48hrs) the proposed run is initiated
Manually from website
XXX Need to decide what pages to trigger runs from and to mock-up a GUI
Publishing results
- Publish complete test results on the certification website
- Publish sanetised results in a publicly-viewable location
- In both cases this could be an XML file with a simple HTML transform
Implementation
UI Changes
A button should be added to the machine page to initiate an install run. Strawman:
Launch install test: [Milestone drop-down] [x] Enable -proposed
Code Changes
Test/Demo Plan
Test manually on the staging server.
Unresolved issues