RoadMap

Revision 13 as of 2009-06-03 16:06:18

Clear message

Ubuntu QA 9.10 Roadmap

Apport

  • Increase Apport Coverage - The QA team will drive an effort to extend coverage of apport hooks to all core Ubuntu components. [needs metric - X% of current bugs could have been filed with an apport hook]

  • Increase Apport Adoption - Adjust documentation and Launchpad functionality to steer more people towards using apport. Push back on various groups in turn to file correctly.

  • Extend Apport Functionality - Some bugs require more information than just automatically attaching system logs to a bug. Some bugs (eg suspend/resume bugs) also need a series of questions to also be answered. Checkbox already provides this type of functionality. It would be good to investigate how to extend apport to leverage checkbox.

  • Apport In ubuntu-server - discussion with server team about including apport by default as well as increasing Apport coverage for server packages.

Bugs

Testing

  • UNR Testing Automation - Perform install testing of UNR on all laptop and netbook class devices as part of a regular rotation of flavours. Incorporate automated desktop UNR tests from the OEM team.

  • Daily Desktop Testing - Add desktop tests to the daily hardware testing schedule on laptops and netbooks.

  • Functional Server Testing - Extend server testing to cover basic functionality and performance of key server packages.

  • Metrics Based Testing - All our current testing is a binary pass/fail but we would also like to track the evolution of certain parameters such as boot speed and power usage that may not have a clear pass/fail threshold but for which historical data is desireable. We will extend out infrastructure to collect and analyse such data.

  • Extended Audio Testing - Extend both manual and automated test cases to test a range of use cases and hardware configurations on a regular basis.

  • Virtual Testing Environments - Explore using regularly updated KVM instances that can be downloaded with various versions of Ubuntu for testing and debugging use. Alternatively consider EC2 instances that people can log into remotely.

  • Package Testing - Setting up regular piuparts and lintian runs.

  • Distribution Upgrade Testing - automated or semi-automated test distribution upgrades (i.e. jaunty -> Karmic). Coordination with Michael Vogt who already does some automated tests.

Tracking & monitoring

  • Regression Immunisation - Put in place procedures to discover major upstream infrastructure changes early and chart the regression potential inherent in these for our users. Use this information at around Feature Freeze to decide whether to push out the new changes or roll back.

  • Test Tracker - We will work with ISD to develop a system used for organising test cases and tracking a wide range of test results. The development is driven by ISD with Platform QA as a key setter of requirements.

  • Testing Weather Report - A web application similar to the Ubuntu release weather report (https://blueprints.launchpad.net/ubuntu/+spec/developer-weather-report) oriented towards Ubuntu pre-release testers that will display various bits of relevant information on the state of the development branch (archive integrity, ongoing library transitions, high-impact bugs, critical or high importance bugs in core components, daily image health and build status, new packages, updates in components related to release goals, calls for testing by developers, etc.)

QA tools

  • Firefox Launchpad Plugin - The Firefox launchpad plugin contains some useful searches for Ubuntu bug reporters particularly the search that will get you a package's bug list. This is something that is not navigatable in Launchpad (you have to hack the url) so would be helpful for reporters to find "their" bug report. How can we get that search if not all the LP ones into the a default install of Ubuntu?

  • Package Greasemonkey Scripts - The Launchpad greasemonkey scripts project seems like a useful one and could help bug triagers a lot however there is no package to install these scripts. The easiest thing to do would be to find a way to package the greasemonkey scripts and have them installed in the right place. Alternatively, perhaps the greasemonkey scripts should be modified into an extension which then can be packaged.

Checkbox