UDS Intrepid QA Report

Contents

Plans for 8.10

Upgrade testing - Take over the running of mvo's upgrade scripts. Perform regular upgrade tests leading up to release in VMs with a significant number of packages installed
Stock Ubuntu KVM images - Provide stock KVM images of different flavours and releases of Ubuntu used for performing SRU, upgrade and install testing (e.g. testing migration and resizing).
KVM images for upstream software - Provide downloadable KVM images of an upstream versions of key software for testing related to bug forwarding. Initial image based on Debian testing, covering Gnome, OpenOffice and Firefox (replacing Iceweasel).
Upstream kernel builds - Work with the kernel team to provide PPA builds of upstream kernels for user testing.
Desktop test automation - Implement and deploy a basic set of desktop tests to be run regularly in the data centre on the development archive
Mobile test automation - Write test scripts for basic functionality of the UME desktop to be run in VMs and on mobile devices
ISO testing automation - Use pre-seeding to to test desktop images as well as server images (currently performed by the server team)
Improved test case management Improve the structure and level of detail in the test cases in the wiki. Fork the current cases into an Intrepid and Hardy LTS set.

Kernel bug migration - merge and close bugs from 2.6.x packages with 'linux'.
Confirmed -> Triaged bug migration - Move a significant proportion of bugs from Confirmed to Triaged.
Bug upstreaming documentation - Improve the documentation explaining how to file bugs upstream for individual upstream projects.
Bug escalation procedures - Provide mechanism for trusted community members to escalate serious bugs

Bug metrics - Continue to improve the set of available bug metrics, focusing increasingly on information that can be used to guide development as well as triage
Package-specific status pages - Implement weather-report style status pages for key packages located at to status.qa.ubuntu.com
SRU bug tracking - Improve tracking of the SRU bug lists with a set of script-generated web pages that extracts status, test case text and tag information from SRU process bugs. Provide views of main and universe as well as current and archive views
Release bug tracking - Clarification and documentation of bug milestones and nominations

QA Intrepid specs are listed QATeam/Specs.

Spec: foo

How can we measure it?
- some date transitions are recorded in Launcpad
  - available on a per task basis
    - date-left-new - particularly useful
      - this is the first time the bug's state changed
    - date-confirmed
    - date-triaged
    - date-incomplete
    - date-triaged - particularly useful
    - date-fixcommited
    - date-fix-released - particularly useful
    - date-closed
    - date-inprogress
    - date-assigned
    - date-created
      - the date-created for a bug watch would show when the upstream task was created
      - the activity log, bug-watch-updater first comment, shows when the watch was created
  - available for the bug in general
    - date reported
    - date updated
  - missing?
    - date importance set - in activity log but potentially hard to parse
  - look at the average or median response times over a 1,3,6 month period
  - a new tool in the bughelper suite could be used to show time in state statistic infromation
How can we improve it?
- making information publicly available
Display on per-package status page
- also combine some packages for example linux-source, linux-restricted-modules, linux-ubuntu-modules, linux-backports-modules
Display on a per repository basis
Display on a per team basis using something like https://bugs.launchpad.net/~ubuntu-printing/+packagebugs
- these are packages that a team is subscribed to

Spec: foo

Case 1: World breaking regressions in stable or late devel version
- Using the critical importance with an escalation bot
Case 2: Important bugs that need be made more prominent before release
- Formal and informal escalation paths within the QA team