Ubuntu Wiki

DRAFT

Summary

The purpose of this document is to explain the different testing efforts for Unity in the Natty cycle.

Desktop

Unity

Test Cases

Although the DX team is the main responsible of testing the Unity interface itself, we need to collaborate with them so our tests work better.

Unity itself is not accessible, so the Unity interface cannot be accessed directly with the tools we have right now. Luckily, Mago is tool agnostic and, although is mostly based on LDTP, we can add other tools to help us testing the interface itself.

The DX team will provide us "only dbus access, not dbusmenu. The plan is to use the existing compiz dbus plugin and connect it to the base automation system inside unity."

Based on that, we can send dbus commands to compiz/unity and run it without atk support.

Current status is:

• njpatel ported the automation system over to the new compiz code base
• alexlauni is going to connect that with the compiz dbus plugin

DX team is expecting something to be working middle of next week.

If we want to test dbusmenu bits, e.g. the global /appmenu/, we can do so with the gnome-panel version. The tests should be easily transposable to the Unity version once that menu gets atk support as well.

Xpresser, which is based in image comparison, is not suitable to check the state of Unity after running a test, as the design will change a lot during the Natty cycle.

System State and Control

• Notifications
• Messaging Menu

• MeMenu

• ApplicationIndicators

• Application Menu (dbusmenu)
• Sound Menu

Software Management

• Software Discovery
• Software Installation/Removal
• Software Update

Window Management

• Workspaces

File Browser

• Context Menu
• Single Click / Double Click

2D Experience

The 2D experience needs to be tested as well and, actually, as the 2D experience runs well in a Xvfb, it will be easier to run using Hudson, without

System State and Control

• Notifications
• Messaging Menu

• MeMenu

• ApplicationIndicators

• Application Menu
• Sound Menu

Software Management

• Software Discovery
• Software Installation/Removal
• Software Update

Window Management

• Workspaces

File Browser

• Context Menu
• Single Click / Double Click

ACTIONS

• Run a Hudson server on the datacenter
• Run a cronjob to update the system on a daily basis
• Run the mago tests on a daily basis in an automated fashion

Applications

• List of applications to test

The applications on Unity can be tested using the same a11y tools that we already use for the 2D experience. One of the main caveats is that Unity does not run under Xvfb and, therefore, we cannot run it on a server without having a full desktop up and running.

ACTIONS

• Research on how to run the tests using Hudson (or something else) in an automated fashion
• Run the mago tests on Unity at least twice per week in a machine with Unity running as mean time solution
• The tests results will be publicly available

Xpresser, although based in image comparison, can be good to launch applications from the launcher, as it is not accessible yet, and the icons don't tend to change that much. We need to be careful, though, as Xpresser is not being actively maintained and there could be some bugs.

Support

• Accessibility
• Translation
• Printing

Server

Although we now have expanded hardware support on Natty (AMD64, AMD64-mac, i386, PowerPC, PowerPC-ps3, ARMEL-dove, possibly others?) we will test only i386 and AMD64. This expanded support is mostly a consolidation of the different ports -- see Natty cdimage ports consolidation.

Server itself

Expand tool-chain testing with more packages (currently we daily-build libvirt, postgresql-8.4, mysql-dfsg-5.1, openldap, php5, python2.6, and coreutils).

Applications

Cloud

We will test both Eucalyptus and Openstack; amount of coverage of Openstack will depend on availability.

• add euca2ools tests
• verify euca2ools compatibility with EC2, and other Cloud technologies
• automate as much as possible the UEC tests currently run (depends on updates on euca-* utilities)
• test image compatibility (Lucid, Maverick, Natty) with EUC-Natty

Hardware Testing

• Any input from the Cert Lab ?

ISO Testing

As part as our per milestone testing, we need to be testing Unity on desktops and the 2D experience fallback. Many of us use VMs to test the images and we need to find a VM technology capable of running Unity.

Also, we need to modify the testcases (or add new ones) to match the new possibilities.

Arch / Flavour ?

Schedule

ACTIONS

• Communication with the installer and design teams to modify the testcases
• Include the new testcases into the ISO tracker

Community Involvement

Engage the LoCo Teams

• LoCo Testing Blueprint

• LoCo Testing Wiki

Laptop Testing Initiative

• Laptop Testing BluePrint (From the Ubuntu-IT Team)

We are going to set up a new tracker to track the results (http://laptop.qa.ubuntu.com). The Italian Testing team will be the main responsible to maintain the milestones and to provide reports.

Desktop Testing

The Desktop Testing Program started last cycle and it provided a good way for the community to test the desktop applications every milestone. As part of the modifications for this program in the Natty cycle, we will be including Unity as part of the program.

For each of the applications, we will be duplicating the testcases, to be run in Unity and also the 2D experience.