NattyGraphicsTestPlan

Differences between revisions 32 and 50 (spanning 18 versions)
⇤ ← Revision 32 as of 2010-12-17 17:31:39 →
Size: 47944
Editor: ACaen-151-1-57-139
Comment:
← Revision 50 as of 2011-03-17 17:01:13 → ⇥
Size: 3439
Editor: ACaen-151-1-93-135
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from QATeam/NattyProprietaryXDriversTestPlan
## page was copied from QATeam/NattyTestPlan
Line 4: Line 6:
== Purpose ==
The purpose of this document is to explain the different testing efforts for Unity in the Natty cycle. It is designed to prescribe the scope, approach, resources, and schedule for all testing activities during this cycle.
Line 7: Line 7:
This document will be modified and enhanced as required throughout the verification testing engagement. The purpose of this document:

 * Defines the overall test approach.
 * Identifies software and system hardware to be tested.
 * Identifies hardware, software, and tools to be used to support the testing efforts.
 * Defines the types of tests to be performed.

''Note: Specific effort for regression testing ?''		 == Objectives ==
The objective of this document is to define a test plan for ATI/NVidia proprietary driver testing, graphics and performance in Unity.
Line 17: Line 11:

The testing tasks for the components of Ubuntu includes:
 * Functional testing of the applications (automated and manual - Desktop Testing initiative)
 * Integration with the desktop environment
   * Unity
   * 2D Desktop (gnome fallback when 3D support is not available)
 * Server Testing
 * ISO Testing
 * Hardware Testing
 * Additional Testing: i18n, a11y, printing		 The testing tasks includes:
 * Testing of ATI and NVidia proprietary drivers
 * Test cases that are known for:
   * Crashes
   * Visual corruption
   * Performance impact
Line 29: Line 19:
 * Collect, provide and review Test Cases that uncover new bugs or cover existing known issues.
Line 34: Line 23:
Testing will be performed all along during the development cycle of Natty. The main milestones of the test plan are aligned on the [[https://wiki.ubuntu.com/NattyReleaseSchedule|Release Schedule]] and [[https://wiki.ubuntu.com/NattyReleaseInterlock|Natty Release Interlock]] Testing is a very dependant activity, As a result, test planning is a continuing activity performed throughout the system development life cycle. == Deliverables ==
 * This test plan
 * At the end of each testing milestone QA will provide a test report for that testing cycle.
 * All defects found will be reported in Launchpad.
Line 36: Line 28:
''Note: HW Testing -> Cert Team: to be included in this doc ?'' == Testing procedure ==
Line 38: Line 30:
== Project Milestones / Schedule == The testing procedure and call for testing is available from the [[https://wiki.ubuntu.com/X/Testing/ProprietaryDrivers/Natty/WeeklyProgram|X Testing Wiki]]
Line 40: Line 32:
 * [[https://wiki.ubuntu.com/NattyReleaseSchedule|Release Schedule]]
 * [[https://wiki.ubuntu.com/NattyReleaseInterlock|Natty Release Interlock]]		 = X Drivers Testing =
Line 43: Line 34:
= Test Items and Features =
== New Features in Natty Narwhal (Ubuntu 11.04) ==
''List the new features that need extensive testing''		 There are 11 test cases covering all aspects of the ATI and NVidia proprietary drivers, from installation, basic and standard functionalities (resolutions, rotation, monitor hot plugging,...) and removal of the drivers.
Line 47: Line 36:
 * GTK 3.0 with GObject Introspection is still a fairly young technology which is still in flux, so we need to regularly test our applications which got ported to it:
  * Apport: Try `ubuntu-bug audio` and select the "record" case, then follow the steps, and finally check if you get a reasonable output and behaviour of the details window (like unreasonable window size, or unreadable text). Please make sure to open the Details expander.		 They are available from [[http://testcases.qa.ubuntu.com/Hardware/X/ProprietaryDrivers|testcases.qa.ubuntu.com]]
Line 50: Line 38:
== Application ==
=== Category of applications to be tested ===		 = Unity Graphics Performance and reliability =
Line 53: Line 40:
The categories of applications to be tested are:
 * Core components (Help, menus, ...)
 * Audio/Video
 * Games
 * Graphics
 * Internet
 * Office
 * Utilities
 * System Preferences
 * System Administration		 These test cases are available from http://testcases.qa.ubuntu.com/Hardware/X/GraphicsVideoPerformance
Line 64: Line 42:
[[https://wiki.ubuntu.com/QATeam/NattyTestPlan/AppList|Each application]] must be covered by at least 1 automated test case to ensure that it can be launched on the platform. Additionally, because using manual testing will cover very few test cases and requires important man power, we will focus on test cases that uncover new bugs and cover existing known issues. = Target Hardware =
== NVIDIA Cards ==
GPUs such as GeForce series 6 or newer are supported and have the hardware capabilities needed to run Unity.
Line 66: Line 46:
'''ACTION'''
 * Work with bdmurray and pedro to do some datamining in launchpad and identify known issues for the aforementioned applications''		 For a complete list, install the package nvidia-current and see /usr/share/doc/nvidia-current/README.txt.gz
Line 69: Line 48:
=== Automated Testing === == ATI Cards ==
 * R6xx (HD 2xxx, HD 3xxx)
 * R7xx (HD 4xxx)
 * R8xx (HD 5xxx) "Evergreen" series
 * (HD 6xxx) "Northern Islands" series
Line 71: Line 54:
We will use Mago (lp:mago) and its helper tool magomatic (lp:magomatic) to automate test cases.  * + Mobile cards
 * And more generally all the graphics chips that are sold today by AMD have the hardware capabilities we need to run Unity. We can safely go as far as 5 years and the GPUs that were made by either of these companies will still offer the hardware features we need.
Line 73: Line 57:
Unity itself is not accessible, so the Unity interface cannot be accessed directly with the tools we have right now. Luckily, [[http://mago.ubuntu.com|Mago]] is tool agnostic and, although is mostly based on [[http://ldtp.freedesktop.org/|LDTP]], we will have to add other tools to help us testing the interface itself (see Actions below for example of tools) = Acceptance Criteria =
== Acceptance Criteria: Phase 1 - Alpha ==
The Phase 1 Alpha deliverable will be considered met when all the basic functionality of the software and hardware is complete. The client will have the ability to test the deliverable software and hardware functionality. There may be defects in this milestone that will be addressed in the next Phase. Performance and reliability requirements are out of scope for the Alpha release.
Line 75: Line 61:
'''ACTIONS'''
 * Investigate XPresser (Limited to image matching)
 * xautomation: Only useful bit is visgrep but no success so far
 * xnee/cnee: Crashes desktop in Natty.
 * Xtest: Integration of mouse and keyboard support: DONE
 * Unity.Debug to return the coordinates of the Desktop components (DX Team)		 The image may have Critical, High and Medium bugs.
Line 82: Line 63:
=== Manual Testing === == Acceptance Criteria: Phase 2 - Beta ==
The Phase 2 Beta release will be considered feature complete based on the Statement of Work. This image will also include bug fixes. The image may have a limited number of Critical bugs. Performance and reliability requirements are acceptable for the Beta release.
Line 84: Line 66:
The Desktop Testing Program started last cycle. It provided a good way for the community to test the desktop applications every milestone. As part of the modifications for this program in the Natty cycle, we will be including Unity as part of the program. == Acceptance Criteria: Phase 3 – Final Release ==
This image will be the Final release.
Line 86: Line 69:
For each of the applications, we will be duplicating the testcases, to be run in Unity and also the 2D experience.

The resources available to the community are:
 * [[http://testcases.qa.ubuntu.com/| Test Cases wiki]] to be extended with the following [[http://people.canonical.com/~j-lallement/natty/testcases.html|additional test cases]]
 * The [[http://desktop.qa.ubuntu.com/| Desktop Testing Tracker]]

== Desktop ==
=== Unity ===

Although the DX team is the main responsible of testing the Unity interface itself, we need to collaborate with them so our tests work better. The QA Team is responsible of the testing of the integration of the application with Unity (ex. Shotwell Menu not integration in the global menu)

The DX team will provide us "only dbus access, not dbusmenu. The D-Bus Debug Service will provide a service to collect the state of the Unity interface i.g Panel, Menu, Indicator, Files place, ...

Based on that, we can send dbus commands to compiz/unity to know the state of the interface without atk support.

If we want to test global menu bits, e.g. the global /appmenu/, we can do so with the gnome-panel version which call 'hidden' menus in Unity. The tests should be easily transposable to the Unity version once that menu gets atk support as well.

All the applications shipped by default with Natty must be tested in Unity. The applications on Unity can be tested using the same a11y tools that we already use for the 2D experience. One of the main caveats is that Unity does not run under Xvfb and, therefore, we cannot run it on a server without having a full desktop up and running.

The subsystems of Unity that needs to be covered are
 * [[http://testcases.qa.ubuntu.com/Applications/Unity|Basic manual Test Cases]]
 * [[https://wiki.ubuntu.com/JeanBaptisteLallement/SandBox/NattyTestCases|Launcher]]
 * Panel
    * Presence: the panel appears, is always visible except when a fullscreen app is on display
    * Content: the panel contains a set of required indicators: Ubuntu button, appmenu, session, clock, sound, battery, network (NM)
 * Panel/Appmenu in particular
    * needs to show up with supported toolkits (GTK and QT)
    * needs to fallback gracefully with Mozilla, Chormium apps, OOo, Java, SWT
 * Applications
    * Note: should be "remote controlled" via the global appmenu, not the hidden menu that is maintained for compatibility reasons; that will test the actual appmenu support for the critical apps on the list
 * Notifications
 * Messaging Menu
 * MeMenu
 * Sound Menu
 * Dash Search
 * Files
 * Dash (Application Places)

An example of integration Test Case is:
 * create a document in gedit,
 * verify that it appears in the Files Place.
 * Then rename that document
 * verify that the document modified appears in places.

'''ACTIONS'''
 * Research on how to run the tests using Hudson (or something else) in an automated fashion
 * Run the mago tests on Unity at least twice per week in a machine with Unity running as mean time solution
 * The tests results will be publicly available


'''Additional tool to extend Mago'''
[[https://wiki.ubuntu.com/Xpresser|Xpresser]], which is based in image comparison, is not suitable to check the state of Unity after running a test, as the design will change a lot during the Natty cycle.
Xpresser, although based in image comparison, can be good to launch applications from the launcher, as it is not accessible yet, and the icons don't tend to change that much. We need to be careful, though, as Xpresser is not being actively maintained and there could be some bugs.


=== 2D Experience ===

All the applications must be tested in the 2D (legacy) environment.

The 2D experience needs to be tested as well and, actually, as the 2D experience runs well in a Xvfb, it will be easier to run using Hudson.

The subsystems of the 2D experience that needs to be covered are
 * Notifications
 * Messaging Menu
 * MeMenu
 * ApplicationIndicators
 * Application Menu
 * Sound Menu

'''ACTIONS'''
 * Run a Hudson server on the datacenter
 * Run a cronjob to update the system on a daily basis
 * Run the mago tests on a daily basis in an automated fashion


== Support Features ==

The following additional component are part of the Testing effort:
 * Accessibility (see with charlie_tca which items and features need to be reviewed)
 * Translation (see with dpm)
 * Printing (tkampeter was expected to provide test cases)


== Server ==
Although we now have expanded hardware support on Natty (AMD64, AMD64-mac, i386, PowerPC, PowerPC-ps3, ARMEL-dove, possibly others?) we will test only i386 and AMD64. This expanded support is mostly a consolidation of the different ports -- see [[https://blueprints.launchpad.net/ubuntu/+spec/ubuntutheproject-foundations-n-cdimage-ports-consolidation|Natty cdimage ports consolidation]].

=== Server itself ===

Expand tool-chain testing with more packages (currently we daily-build libvirt, postgresql-8.4, mysql-dfsg-5.1, openldap, php5, python2.6, and coreutils).

=== Applications ===


=== Cloud ===

We will test both Eucalyptus and Openstack; amount of coverage of Openstack will depend on availability.

 * add euca2ools tests
 * verify euca2ools compatibility with EC2, and other Cloud technologies
 * automate as much as possible the UEC tests currently run (depends on updates on euca-* utilities)
 * test image compatibility (Lucid, Maverick, Natty) with EUC-Natty


== Hardware Testing ==
=== HW Team ===
 * Any input from the HW Team ?

=== Laptop Testing Initiative ===
 * [[https://blueprints.launchpad.net/ubuntu/+spec/ubuntutheproject-qa-n-laptop-testing-tracker|Laptop Testing BluePrint]] (From the Ubuntu-IT Team)

We are going to set up a new tracker to track the results (http://laptop.qa.ubuntu.com). The Italian Testing team will be the main responsible to maintain the milestones and to provide reports.

== ISO Testing ==

As part as our per milestone testing, we need to be testing Unity on desktops and the 2D experience fallback.
Many of us use VMs to test the images and we need to find a VM technology capable of running Unity.

Also, we need to modify the testcases (or add new ones) to match the new possibilities.

=== Architectures and Flavours ===

||<style="border-top: 2px solid #000000; border-left: 2px solid #000000"> <<BR>> ||<-7 style="border-top: 2px solid #000000; border-bottom: 1px solid #000000; border-left: 1px solid #000000; border-right: 1px solid #000000"> '''Architecture''' ||
||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000; border-left: 2px solid #000000; border-right: 1px solid #000000"> '''Flavour''' ||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000; border-left: 1px solid #000000"> '''amd64''' ||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000"> '''armel''' ||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000"> '''dove''' ||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000"> '''i386''' ||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000"> '''omap3''' ||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000"> '''omap4''' ||<style="border-top: 1px solid #000000; border-bottom: 1px solid #000000; border-left: 1px solid #000000; border-right: 2px solid #000000"> '''Total''' ||
||<style="border-top: 1px solid #000000; border-left: 2px solid #000000; border-right: 1px solid #000000"> Edubuntu DVD ||<style="border-top: 1px solid #000000; border-left: 1px solid #000000"> 1 ||<style="border-top: 1px solid #000000"> <<BR>> ||<style="border-top: 1px solid #000000"> <<BR>> ||<style="border-top: 1px solid #000000"> 1 ||<style="border-top: 1px solid #000000"> <<BR>> ||<style="border-top: 1px solid #000000"> <<BR>> ||<style="border-top: 1px solid #000000; border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Kubuntu Alternate ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Kubuntu Desktop ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || 1 || 1 ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''4''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Kubuntu DVD ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Kubuntu Mobile ||<style="border-left: 1px solid #000000"> <<BR>> || 1 || <<BR>> || 1 || 1 || 1 ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''4''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Mythbuntu Desktop ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Alternate ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu ARM Preinstalled ||<style="border-left: 1px solid #000000"> <<BR>> || <<BR>> || <<BR>> || <<BR>> || 1 || 1 ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Desktop ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu DVD ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Netbook ||<style="border-left: 1px solid #000000"> <<BR>> || 1 || 1 || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''3''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 EBS (Asia-Pacific) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 EBS (Europe) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 EBS (US-East) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 EBS (US-West) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 instance (Asia-Pacific) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 instance (Europe) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 instance (US-East) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server EC2 instance (US-West) ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Server UEC ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Ubuntu Studio Alternate ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Upgrade Edubuntu ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Upgrade Kubuntu ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Upgrade Ubuntu ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Upgrade Ubuntu Server ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Upgrade Wubi ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Upgrade Xubuntu ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Xubuntu Alternate ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-left: 2px solid #000000; border-right: 1px solid #000000"> Xubuntu Desktop ||<style="border-left: 1px solid #000000"> 1 || <<BR>> || <<BR>> || 1 || <<BR>> || <<BR>> ||<style="border-left: 1px solid #000000; border-right: 2px solid #000000"> '''2''' ||
||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000; border-left: 2px solid #000000; border-right: 1px solid #000000"> '''Total''' ||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000; border-left: 1px solid #000000"> '''27''' ||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000"> '''2''' ||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000"> '''1''' ||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000"> '''29''' ||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000"> '''3''' ||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000"> '''3''' ||<style="border-top: 1px solid #000000; border-bottom: 2px solid #000000; border-left: 1px solid #000000; border-right: 2px solid #000000"> '''65''' ||


''' Other Reference '''
 * [[https://blueprints.launchpad.net/ubuntu/+spec/other-qa-n-testing-different-architectures|Testing on different architectures]]

'''ACTIONS'''
 * Communication with the installer and design teams to modify the testcases
 * Include the new testcases into the ISO tracker

== Feature not to be tested ==


 * [[https://wiki.ubuntu.com/Testing/LoCoTeam|LoCo Testing Wiki]]

= Testing Process =
== Test Deliverables ==
 * This test plan
 * [[http://testcases.qa.ubuntu.com|The updated list of test cases]] and their results on associated *.qa.ubuntu.com tracking site.
 * Any test scripts used for automation or verification
 * A list of defects filed in Launchpad

== Testing Tasks ==
Testing will execute test cases to verify each of the items in section "Test Items and Features". Tests will be run on different hardware and executable environment as defined in section "Architectures and Flavours"

== Responsibilities ==
The QA team is responsible for executing this test plan. The responsibility can be delegated to other team for specific testing especially:
 * Unit testing of new features
 * Hardware testing
 * Laptop testing
 * ...

Participation from other teams and community is strongly encouraged.

== Resources ==
=== Community Involvement ===
The community can actively participate in all the testing effort during that cycle:
 * Writing manual test cases
 * Writing automated test cases for Mago
 * Application Testing
 * Desktop Testing
 * Hardware Testing (especially with the laptop initiative from the IT LoCo Team

=== Engage the LoCo Teams ===

Additionally to the Desktop Testing program and the Laptop Testing initiative, we will rely on the following efforts:
 * [[https://blueprints.launchpad.net/ubuntu/+spec/ubuntutheproject-qa-n-loco-testing|LoCo Testing Blueprint]]

== Testing Schedule ==

||<rowbgcolor="#cccccc"> '''Week''' || '''Date''' ~-(Thursday)-~ || '''Work Item Iteration''' || '''Status''' || '''Testing Schedule''' || '''Notes''' ||
||<-6 #eeeeee> '''October 2010''' ||
|| 1 || October 21st || A-2 || Toolchain Uploaded || <<BR>> || <<BR>> ||
|| 2 || October 28th || A-2 || Developer Summit || <<BR>> || <<BR>> ||
||<-6 #eeeeee> '''November 2010''' ||
|| 3 || November 4th || A-2 || <<BR>> || Test Plan || <<BR>> ||
|| 4 || November 11th || A-2 || <<BR>> || Test Plan || <<BR>> ||
|| 5 || November 18th || A-2 || <<BR>> || Mago Tests Writing || <<BR>> ||
|| 6 || November 25th || A-2 || [[https://wiki.ubuntu.com/FeatureDefinitionFreeze|FeatureDefinitionFreeze]] || Mago Tests Writing || <<BR>> ||
||<-6 #eeeeee> '''December 2010''' ||
|| 7 || December 2nd || Iteration Planning || Alpha 1 || A1 ISO Testing / Mago Tests Writing || <<BR>> ||
|| 8 || December 9th || A-2 || <<BR>> || Mago Tests Writing || <<BR>> ||
|| 9 || December 16th || A-2 || <<BR>> || Mago Tests Writing || <<BR>> ||
|| 10 || December 23rd || A-2 || <<BR>> || Mago Tests Writing || <<BR>> ||
|| 11 || December 30th || A-2 || [[https://wiki.ubuntu.com/DebianImportFreeze|DebianImportFreeze]] || Mago Tests Writing || <<BR>> ||
||<-6 #eeeeee> '''January 2011''' ||
|| 12 || January 6th || Iteration Planning || <<BR>> || <<BR>> || <<BR>> ||
|| 13 || January 13th || A-2 || <<BR>> || <<BR>> || <<BR>> ||
|| 14 || January 20th || A-2 || <<BR>> || 10.04.2 SRU Cleaning || <<BR>> ||
|| 15 || January 27th || A-2 || Ubuntu 10.04.2 || 10.04.2 ISO Testing || <<BR>> ||
||<-6 #eeeeee> '''February 2011''' ||
|| 16 || February 3rd || Iteration Planning || Alpha 2 || A2 ISO Testing || <<BR>> ||
|| 17 || February 10th || A-3 || <<BR>> || <<BR>> || <<BR>> ||
|| 18 || February 17th || A-3 || <<BR>> || <<BR>> || <<BR>> ||
|| 19 || February 24th || A-3 || [[https://wiki.ubuntu.com/FeatureFreeze|FeatureFreeze]] || <<BR>> || <<BR>> ||
||<-6 #eeeeee> '''March 2011''' ||
|| 20 || March 3rd || Iteration Planning || Alpha 3 || A3 ISO Testing || <<BR>> ||
|| 21 || March 10th || Beta UI || <<BR>> || Desktop Testing || <<BR>> ||
|| 22 || March 17th || Beta UI || <<BR>> || Desktop Testing || <<BR>> ||
|| 23 || March 24th || Beta UI || [[https://wiki.ubuntu.com/UserInterfaceFreeze|UserInterfaceFreeze]],  [[https://wiki.ubuntu.com/BetaFreeze|BetaFreeze]] || <<BR>> || <<BR>> ||
|| 24 || March 31st || Quality || Beta || Beta ISO Testing || <<BR>> ||
||<-6> '''April 2011''' ||
|| 25 || April 7th || Quality || [[https://wiki.ubuntu.com/DocumentationStringFreeze|DocumentationStringFreeze]] || Desktop Testing || <<BR>> ||
|| 26 || April 14th || Quality || [[https://wiki.ubuntu.com/FinalFreeze|FinalFreeze]],  [[https://wiki.ubuntu.com/KernelFreeze|KernelFreeze]],  [[https://wiki.ubuntu.com/NonLanguagePackTranslationDeadline|NonLanguagePackTranslationDeadline]] || Desktop Testing || <<BR>> ||
|| 27 || April 21st || Quality || [[https://wiki.ubuntu.com/ReleaseCandidate|ReleaseCandidate]],  [[https://wiki.ubuntu.com/LanguagePackTranslationDeadline|LanguagePackTranslationDeadline]] || RC ISO Testing || <<BR>> ||
|| 28 || April 28th || Quality || [[https://wiki.ubuntu.com/FinalRelease|FinalRelease]] Ubuntu 11.04 || Final ISO Testing || <<BR>> ||


= Other Requirements =
== Hardware ==
== Software ==
== Tools ==
== Risks and Assumptions ==
 * For ISO testing, a new test run will be initiated if a new media is provided during a run. Test results will be reset.
 * UNITY '''doesn't run in a virtual environment'''. Because of this ISO testing of UNITY can only be done on real hardware with a GPU. This means that ISO Testing will be much slower than usual.
 * Natty comes with 2 versions of the desktop: UNITY and 2D/Gnome. This means that there are twice the number of cases to test.

= Reporting Bug =
When a defect is discovered, please file a report by following the [[https://help.ubuntu.com/community/ReportingBugs|Reporting Bugs Guidelines]]


= Appendix A: Detailed Test Plan for the default applications =
Category: General
Case ID: genmenu-001
Affected Package: Metacity (Not Unity)
Automated: Yes

 Purpose:
     1. Application Menu
     2. Confirm that applications can be launched with the menu

 Steps:
     1. Verify that each application in the 'Sound & Video' menu can be launched
     2. Verify that each application in the 'Internet' menu can be launched
     3. Verify that each application in the 'Office' menu can be launched
     4. Verify that each application in the 'Applications' menu can be launched

 Verification:
     1. Have the conditions been met with no adverse effects?

---
Category: General
Case ID: gensan-001
Affected Package: All applications in the Applications and System menus
Automated: Yes (For the list of packages installed by default)

 Purpose:
     1. Check that an application can be launched
     2. Confirm that the application is rendered and usable

 Steps:
     1. Verify that the application can be launched
     2. Verify that the 'About Dialog' can be displayed and closed
     3. Verify that the application closes cleanly

 Verification:
     1. Have the conditions been met with no adverse effects?

---
Category: AudioVideo
Case ID: appsoubra-001
Affected Package: brasero
Automated: YES (burns an ISO)

 Purpose:
     1. Brasero - Basic Features
     2. Confirm Basic Functionality

 Steps:
     1. Verify Brasero detects a disk and gives the option to burn or not
     2. Verify Brasero can create a music CD/DVD
     3. Verify Brasero can create a data CD/DVD
     4. Verify Brasero can create a video DVD/VCD
     5. Verify Brasero can create a disk image from the inserted media
     6. Verify Brasero erases media automatically and other functionality

 Verification:
     1. Have the conditions been met with no adverse effects?


---
Category: AudioVideo
Case ID: appsouban-001
Affected Package: banshee
Automated: YES (import m2u, pls simple and extended, play file)

 Purpose:
     1. Banshee - Media

 Steps:
     1. Click Media > Import Playlist choose a playlist from a folder and import into Banshee. Import both video and audio
     2. Click Media > Open Location, pick a media file and see if it plays ; audio and video
     3. Click Media > Subscribe to Podcast, enter a URI(www.npr.og) . Choose one of the options and see if it downloads

 Verification:
     1. Have all the conditions been met with no adverse effects?

---
Category: AudioVideo
Case ID: appsouban-002
Affected Package: banshee
Depends on: applications/sound-and-video/banshee/launch
Automated: YES (Add podcast, Enable/Disable plugin)

 Purpose:
     1. Banshee - Preferences

 Steps:
     1. Click Edit > Add to playlist, pick an audio or video file and add to a new playlist
     2. Click Edit > Write to, (CD for Audio and DVD for Video) and choose a playlist to be burnt
     3. Click Edit > Edit Information, pick a track and edit the information and title of the track listing
     4. Confirm you can Select All tracks and also delete tracks from the library
     5. Click Edit > Properties , pick a track and confirm the properties of the track are listed
     6. Click Edit > Preferences > General, confirm you can select your Music Library
     7. Click Edit > Preferences > General , select "Copy files to media folders when imported" import a file and confirm the file was imported to your selected media file
     8. Click Edit > Preferences > General , confirm you can choose your File system Organization to the different options

 Verification:
     1. Have all the conditions been met with no adverse effects?

---
applications/acc/disk-usage-analyser/launch
Case ID: appaccdis-001
Affected Package: gnome-utils
Automated: YES

 Purpose:
     1. Disk Usage Analyzer - Launching

 Steps:
     1. Click Applications -> Accessories -> Disk Usage Analyzer

 Verification:
     1. Did disk usage analyzer launch as expected?

---
applications/acc/disk-usage-analyser/scan
Case ID: appaccdis-002
Affected Package: gnome-utils
Depends on: applications/acc/disk-usage-analyser/filesys
Automated: YES

 Purpose:
     1. Disk Usage Analyzer - Scan Disk

 Steps:
     1. Analyzer -> Scan Home Folder
     2. Home folder contents are displayed

 Verification:
     1. Is the Home folder contents displayed?

----
applications/acc/disk-usage-analyser/filesys
Case ID: appaccdis-003
Affected Package: gnome-utils
Depends on: applications/acc/disk-usage-analyser/folder
Automated: YES

 Purpose:
     1. Disk Usage Analyzer - Scan Filesystem

 Steps:
     1. Analyzer -> Filesystem
     2. Filesystem contents are displayed

 Verification:
     1. Is the Filesystem contents displayed?

---
Category: Internet
Case ID: appoffevo-004
Affected Package: evolution
Depends on:
Automated: YES

 Purpose:
     1. Email - Evolution - Messenging

 Steps:
     1. Verify you can create and send new messages
     2. Verify you receive message receipts
     3. Confirm deletion of new compositions
     4. Confirm deletion of received messages
     5. Confirm "draft" ability

 Verification:
     1. Have the conditions been met with no adverse effects?

---
Category: Internet
Case ID: appoffevo-
Affected Package: evolution
Depends on:
Automated: YES

 Purpose:
     1. Email - Evolution - Messenging

 Steps:
     1. Verify you receive message receipts
     3. Confirm that the notification icod reacts and shows new message count

 Verification:
     1. Have the conditions been met with no adverse effects?

---
applications/int/firefox/about
Case ID: appintfir-009
Affected Package: firefox

 Purpose:
     1. Firefox - About

 Steps:
     1. Open About dialogue and verify version, copyright, and credits

 Verification:
     1. Are the version, copyright and credits correct?

---
applications/int/firefox/search
Case ID: appintfir-008
Affected Package: firefox
AUTOMATED: YES (Google)

 Purpose:
     1. Firefox - MenusWeb Search

 Steps:
     1. Verify web search using Google
     2. Verify web search using Yahoo
     3. Verify web search using Ask
     4. Verify web search using Dictionary
     5. Verify web search using Ebay

 Verification:
     1. Have all the conditions been met with no adverse effects?

---
applications/int/firefox/tabs
Case ID: appintfir-026
Affected Package: firefox
Automated: YES

 Purpose:
     1. Firefox - Tab Management
     2. Confirm tab feature functionality

 Steps:
     1. Open new tabs (Ctr+T)
     2. Move and close tabs
     3. Verify user is asked to save tabs when closing browser
     4. Verify that tabs saved on closing are restored when re-opening browser

 Verification:
     1. Have all the conditions been satisfied with no adverse effects?

---
applications/int/firefox/tools
Case ID: appintfir-007
Affected Package: firefox
Automated: YES (Tabs)

 Purpose:
     1. Firefox - MenusTools, History, Bookmarks

 Steps:
     1. Add and remove bookmarks
     2. Verify History shortcut keys function properly
     3. Open history and verify that usability
     4. Open the Tools and verify that usability

 Verification:
     1. Have the conditions been met with no adverse effects?

---
Case ID: appacccal-001
Affected Package: gcalctool
Depends on: applications/acc/calculator/launch
Automated: YES

 Purpose:
     1. Calculator - Mathematical functions verification

 Steps:
     1. Confirm simple math functions (+,-,/,*)
     2. Confirm nested math functions ((,))
     3. Confirm fractional math
     4. Confirm decimal math

 Verification:
     1. Does the calculator work as expected?

---
applications/acc/calculator/launch
Case ID: appacccal-004
Affected Package: gcalctool
Automated: YES

 Purpose:
     1. Calculator - Verify application launches

 Steps:
     1. Applications -> Accessories -> Calculator
     2. Press the test button the launch the app from command line

 Verification:
     1. Did the application launch?

---
Case ID: appacccal-00
Affected Package: gcalctool
Depends on: applications/acc/calculator/launch
Automated: YES

 Purpose:
     1. Calculator - Thousand sperator

 Steps:
     1. Enable the thousand separator
     2. Enter a number with thousands
     3. Confirm that the separator is shown and correct

 Verification:
     1. Does the calculator work as expected?

---
Case ID: appacccal-00
Affected Package: gcalctool
Depends on: applications/acc/calculator/launch
Automated: YES

 Purpose:
     1. Calculator - Views

 Steps:
     1. Select differents views

 Verification:
     1. Does the calculator work as expected?

---
applications/acc/take-screenshots/window
Case ID: appacctak-001
Affected Package: gnome-utils
Automated: YES

 Purpose:
     1. Take Screenshot window
     2. Screenshot

 Steps:
     1. Press test button
     2. Take Screenshot of the window

 Verification:
     1. Did the screenshot of the window get taken properly?

---
applications/acc/take-screenshots/area
Case ID: appacctak-002
Affected Package: gnome-utils
Depends on: applications/acc/take-screenshots/delay
Automated: YES

 Purpose:
     1. Take Screenshot of an area
     2. Screenshot

 Steps:
     1. Grab an area of the screen
     2. Press test button
     3. Take Screenshot of an area

 Verification:
     1. Did the screenshot of the area get taken properly?

---
applications/acc/take-screenshots/delay
Case ID: appacctak-003
Affected Package: gnome-utils
Depends on: applications/acc/take-screenshots/window
Automated: YES

 Purpose:
     1. Take Screenshot with a delay
     2. Screenshot

 Steps:
     1. Pick a the delay time
     2. Press test button
     3. Take Screenshot of the window

 Verification:
     1. Did the screenshot get taken at the selected delay time?

---
applications/acc/text-editor/save
Case ID: appacctex-004
Affected Package: gedit
Depends on: applications/acc/text-editor/open
Automated: YES

 Purpose:
     1. Text editor - Features
     2. Save

 Steps:
     1. Confirm any text and formating is still present after save and open
     2. Confirm that notepad can save as several different

 Verification:
     1. Does the above text editor functionality work properly?

---
applications/acc/text-editor/save
Case ID: appacctex-004
Affected Package: gedit
Depends on: applications/acc/text-editor/open
Automated: YES

 Purpose:
     1. Text editor - Unicode

 Steps:
     1. Confirm any text and formating is still present after save and open
     2. Confirm that notepad can save as several different

 Verification:
     1. Does the above text editor functionality work properly?

---
applications/acc/tomboy/launch
Case ID: appacctom-002
Affected Package: tomboy
Automated: YES

 Purpose:
     1. Tomboy - Verify application can launch

 Steps:
     1. Click on the yellow note icon in the panel
     2. Click the test button

 Verification:
     1. Did the application launch in both instances?

---
applications/acc/tomboy/search
Case ID: appacctom-003
Affected Package: tomboy
Depends on: applications/acc/tomboy/launch
Automated: YES

 Purpose:
     1. Tomboy - Search Notes

 Steps:
     1. Create another note
     2. Click Search All Notes
     3. Enter a search term found in both notes

 Verification:
     1. Were both notes found in the search?

---
preferences-passwords-encryption-keyring
Case ID: syspas-002
Affected Package: seahorse
Depends on: preferences-passwords-encryption-launch
Automated: YES

 Purpose:
     1. Seahorse - Passwords and Encryption - New/Import
     2. New key/Password
     3. Create a new keyring

 Steps:
     1. File -> New -> Keyring
     2. Choose a name TESTkeyring

 Verification:
     1. Has a new keyring named TESTkeyring been created?

---
preferences-passwords-encryption-pgp
Case ID: syspas-003
Affected Package: seahorse
Depends on: preferences-passwords-encryption-launch
Automated: YES

 Purpose:
     1. Seahorse - Create a new PGP key

 Steps:
     1. File -> New -> PGP
     2. Choose a name TESTpgp

 Verification:
     1. Has a new PGP key named TESTpgp been created?

---
preferences-passwords-encryption-ssh
Case ID: syspas-004
Affected Package: seahorse
Depends on: preferences-passwords-encryption-launch
Automated: YES

 Purpose:
     1. Seahorse - Create a new key/password

 Steps:
     1. File -> New -> SSH
     2. Choose a name TESTssh

 Verification:
     1. Has a new secure shell key named TESTssh been created?

---
preferences-appearance-customize
Case ID: sysapp-001
Automated: YES

 Purpose:
     1. Customize existing themes

 Steps:
     1. Click Test to launch the Appearance properties
     2. Confirm the ability to change color
     3. Confirm the ability to change font
     4. Confirm the ability to change icons
     5. Confirm the ability to change backgrounds

 Verification:
     1. Have the condition been met with no adverse effects?

---
preferences-appearance-new-themes
Case ID: sysapp-002
Automated: YES

 Purpose:
     1. Apply new themes

 Steps:
     1. Click Test to launch the Appearance properties
     2. Confirm the ability to select from additional default themes
     3. Confirm that selected theme is applied

 Verification:
     1. Have the condition been met with no adverse effects?


---
Case ID: seatoo-001
Automated: YES

 Purpose:
    1. Search for an UTF8 file

 Steps:
    1. Search for an UTF8 file

 Verification:
     1. Have the condition been met with no adverse effects?

---
Case ID: seatoo-002
Automated: YES

 Purpose:
    1. Search for a file with RTL name in Hebrew

 Steps:
    1. Search for a file with RTL name in Hebrew

 Verification:
     1. Have the condition been met with no adverse effects?

---
Case ID: seatoo-003
Automated: YES

 Purpose:
    1. Search for a file with RTL name in Arabic

 Steps:
    1. Search for a file with RTL name in Arabic

 Verification:
     1. Have the condition been met with no adverse effects?

---
Case ID: seatoo-004
Automated: YES

 Purpose:
    1. Search for files with a wildward

 Steps:
    1. Search for files with .ogg extension

 Verification:
     1. Have the condition been met with no adverse effects?

---
Case ID: seatoo-005
Automated: YES

 Purpose:
    1. Search for a newly created file trough the filesystem

 Steps:
    1. Search for a newly created file trough the filesystem

 Verification:
     1. Have the condition been met with no adverse effects?

---
Case ID: seatoo-002
Automated: YES

 Purpose:
    1. Enable all the search options

 Steps:
    1. Search for an UTF8 file

 Verification:
     1. Have the condition been met with no adverse effects?

---
Category: AudioVideo
Case ID: appsousho-001
Affected Package: shotwell
Automated: YES

 Purpose:
     1. Shotwell - Media

 Steps:
     1. Import Folder and Copy Files
     2. Import Folder and Link Files

 Verification:
     1. Have all the conditions been met with no adverse effects?

---
applications/graphics/simple-scan/basic
Case ID: appgraxsa-001
Affected Package: simple-scan
Automated: YES (With TEST backend from XSane)

 Purpose:
     1. Basic functionality
     2. Connect to scanner

 Steps:
     1. Test Scanning
         a. Run simple-scan (Applications -> Graphics -> Simple Scan)
         b. Verify that the scanner is detected (i.e. the message "No devices detected" is not displayed)
         c. Place a document in the scanner in the proper orientation
         d. Click "Scan"
         e. Verify Document is scanned
         f. Verify Document image resembles original document
         g. Verify for both Text and Photo quality

 Verification:
     1. Did all the above conditions get satisfied without any adverse effects?

---
applications/software-center/05-searching
Case ID: appusc-005
Affected Package: software-center
Automated: YES

 Purpose:
     1. Searching in "Get Software"

 Steps:
     1. Click on Applications->Software Center
     2. Type Ctrl+F; the search field should be focused
     3. Type "fire" (without quotes then press Enter)
         a. The "Search Results" path button element should appear
         b. The main pane should be replaced by search results, with Firefox first
     4. Press the down arrow key once; Firefox should be selected in the results
     5. Press Enter; the Center should navigate to the software item screen for Firefox
     6. Close the Software Center

 Verification:
     1. Did all the above conditions get satisfied without any adverse effects?


---
applications/software-center/08-install
Case ID: appusc-008
Affected Package: software-center
Automated: YES

 Purpose:
     1. Installing an application

 Steps:
     1. Click on Applications->Software Center
         a. When opened directly, the Center should by default display the lobby screen of the "Get Software" section, scrolled to the top
         b. The search field should be empty
         c. Default focus should be on the search field
     2. Search for "liferea"
     3. Click on the entry for liferea
     4. Click on the install button
     5. Type in your password
     6. While it is installing, navigate to the "In Progress" section, and open the "View" menu
         a. Its items should be insensitive
     7. Close the Software Center
     8. When its finished installing look to see if it's installed in Applications->Internet->liferea

 Verification:
     1. Did all the above conditions get satisfied without any adverse effects?
 NOTES:
     1. In the standard gnome-panel "Menu Bar" applet, "Ubuntu Software Center" should be the last item in the "Applications" menu, with a separator immediately before it
     2. If the package is known and currently installed, the "Installed Software" item screen for the package
     3. If the package is known and not currently installed (even if it is currently being installed), the "Get Software" item screen for the package

---
applications/software-center/09-remove
Case ID: appusc-009
Affected Package: software-center
Automated: YES

 Purpose:
     1. Removing an application

 Steps:
     1. Click on Applications->Software Center
     2. Search for "liferea"
     3. Click on the entry for liferea
     4. Click on the remove button
     5. Type in your password
     6. Close the Software Center
     7. When its finished installing look to see if it's been removed from Applications->Internet->liferea

 Verification:
     1. Did all the above conditions get satisfied without any adverse effects?

---
applications/acc/tomboy/search
Case ID: appacctom-003
Affected Package: tomboy
Depends on: applications/acc/tomboy/launch
Automated:

 Purpose:
     1. Tomboy - Search Notes

 Steps:
     1. Create another note
     2. Click Search All Notes
     3. Enter a search term found in both notes

 Verification:
     1. Were both notes found in the search?

---
applications/acc/tomboy/delete
Case ID: appacctom-003
Affected Package: tomboy
Depends on: applications/acc/tomboy/launch
Automated:

 Purpose:
     1. Tomboy - Delete

 Steps:
     1. Create another note
     2. Delete the Note

 Verification:
     1. Did all the above conditions get satisfied without any adverse effects?

---
administration/update-manager-installation
Case ID: admupd-003
Automated: YES

 Purpose:
     1. Verify update manager's ability to install updates

 Steps:
     1. Click on Install Updates button
     2. Confirm that applying the updates begins the download process
     3. Confirm that only selected updates are downloaded
     4. Confirm that once download is complete, updates install
     5. Confirm that when installation is complete, update manager notifies user
     6. Confirm that if an error occurs, update manager notifies user
     7. Confirm that failed updates can be re-tried
     8. Confirm that updated software is now available and running

 Verification:
     1. Does the Update Manager installation process provide the functionality expected?		 All features will be complete.
There are no Critical bugs. A minimum of High level bugs may be present.

Introduction

Objectives

The objective of this document is to define a test plan for ATI/NVidia proprietary driver testing, graphics and performance in Unity.

Scope

The testing tasks includes:

  • Testing of ATI and NVidia proprietary drivers
  • Test cases that are known for:
    • Crashes
    • Visual corruption
    • Performance impact

For each testing task we will also:

  • Collect test data and review testing results.
  • Reporting of all test results.
  • Risk analysis and communication of the blocking issues.

Deliverables

  • This test plan
  • At the end of each testing milestone QA will provide a test report for that testing cycle.
  • All defects found will be reported in Launchpad.

Testing procedure

The testing procedure and call for testing is available from the X Testing Wiki

X Drivers Testing

There are 11 test cases covering all aspects of the ATI and NVidia proprietary drivers, from installation, basic and standard functionalities (resolutions, rotation, monitor hot plugging,...) and removal of the drivers.

They are available from testcases.qa.ubuntu.com

Unity Graphics Performance and reliability

These test cases are available from http://testcases.qa.ubuntu.com/Hardware/X/GraphicsVideoPerformance

Target Hardware

NVIDIA Cards

GPUs such as GeForce series 6 or newer are supported and have the hardware capabilities needed to run Unity.

For a complete list, install the package nvidia-current and see /usr/share/doc/nvidia-current/README.txt.gz

ATI Cards

  • R6xx (HD 2xxx, HD 3xxx)
  • R7xx (HD 4xxx)
  • R8xx (HD 5xxx) "Evergreen" series
  • (HD 6xxx) "Northern Islands" series
  • + Mobile cards
  • And more generally all the graphics chips that are sold today by AMD have the hardware capabilities we need to run Unity. We can safely go as far as 5 years and the GPUs that were made by either of these companies will still offer the hardware features we need.

Acceptance Criteria

Acceptance Criteria: Phase 1 - Alpha

The Phase 1 Alpha deliverable will be considered met when all the basic functionality of the software and hardware is complete. The client will have the ability to test the deliverable software and hardware functionality. There may be defects in this milestone that will be addressed in the next Phase. Performance and reliability requirements are out of scope for the Alpha release.

The image may have Critical, High and Medium bugs.

Acceptance Criteria: Phase 2 - Beta

The Phase 2 Beta release will be considered feature complete based on the Statement of Work. This image will also include bug fixes. The image may have a limited number of Critical bugs. Performance and reliability requirements are acceptable for the Beta release.

Acceptance Criteria: Phase 3 – Final Release

This image will be the Final release.

All features will be complete. There are no Critical bugs. A minimum of High level bugs may be present.

QATeam/NattyGraphicsTestPlan (last edited 2011-03-17 17:01:13 by ACaen-151-1-93-135)