UNRTestingAutomation

Revision 2 as of 2009-05-26 10:00:26

Clear message

Summary

Perform install testing of UNR on all laptop and netbook class devices as part of a regular rotation of flavours. Incorporate automated desktop UNR tests from the OEM team.

Release Note

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.

Rationale

This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified.

User stories

Assumptions

Design

You can have subsections that better describe specific parts of the issue.

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

Include:

  • data migration, if any
  • redirects from old URLs to new ones, if any
  • how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

UNR images need to be tested on physical hardware rather than VMs

Booting images

Currently UNR is built as a USB image, but the current test infrastructure requires an ISO

  • We should ask the UNR folks to create ISOs for testing ("alternate" image)
  • Also determine if UNR USB images can be booted in VMWare
    • (It currently does not work in KVM)

Potential method:
  • Leave a blank USB stick in the machine
  • Boot from PXE, which installs image to USB drive
  • Reboot from USB stick and perform install
  • After testing, wipe USB stick and hard drive

Another method
  • Keep a USB stick in the machine
  • Always boot from PXE
  • PXE server tells the system where to boot from (USB image or local drive)
  • We will need an environment whose only purpose in life is to reset the boot media (i.e. write the latest image to a USB stick or CDRW)
    • This environment would need to be certified on each new piece of hardware; this might be unwieldy

Current Testing Issues

  • Clutter is not testable (no toolkit support for a11y)
    • adding ATK support is a target for Karmic
  • Some hardware on netbooks have no tests in Checkbox
    • 3G
      • don't have enough SIM cards to leave these in machines in Lexington
      • don't have SIM cards at all in Montreal
      • can partially test by looking to see if the hardware is detected
    • TV tuner
    • video out
      • do we need specialized hardware? (e.g. a KVM)
      • can XRandR provide info?
  • Need to ensure that both wired and wireless networking is tested
  • audio
    • internal and external mics
  • detect windows that exceed the screen resolution
    • might be added to OEM application launching tests

Questions

  • What is the best way to write images to USB drives or CDs?
    • can use dd, try other methods if this does not work
  • Potentially different access points can have different support; should we test this?
    • e.g. grab the cheapest possible access point for testing, as these tend to be common and the most problematic

Action items

  • Determine if there is a way to tell grub to boot from a USB device (Marc)
  • Assign someone in OEM to become familiar with Checkbox Satellite (Chris Gregan)
  • Pseudo-sprint to determine design needs of this system (Ronald and Chris)
    • followup with Marc to get it implemented
  • Find out about how to get SIM cards for hardware (Ronald)
  • Integrate all OEM tests into mainline Checkbox (Ronald and Chris)
  • Look into sound generation and reception testing (Ronald)
    • get results to Chris and Marc
    • Colin King also uses something similar (or possibly the same thing?)

CategorySpec