TouchTesting
|
Size: 2578
Comment:
|
Size: 3705
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| <<Include(QATeam/Header)>> <<BR>> |
|
| Line 3: | Line 5: |
| /!\ '''W.I.P.''' /!\ |
|
| Line 6: | Line 6: |
| Welcome to the testing page of the Touch Testing SWAT. These pages provide up to date information of current testing activities for Touch until the release of Saucy, check back frequently. | Welcome to the testing page of the Touch Testing SWAT. These pages provide up to date information of current testing activities for Touch. |
| Line 9: | Line 9: |
| * '''Mailing List''' - The quality mailing list [https://lists.ubuntu.com/mailman/listinfo/ubuntu-quality|ubuntu-quality@lists.ubuntu.com] is used to coordinate testing activities with all participants. | * '''Mailing List''' - The quality mailing list [[https://lists.ubuntu.com/mailman/listinfo/ubuntu-quality|ubuntu-quality@lists.ubuntu.com]] is used to coordinate testing activities with all participants. |
| Line 13: | Line 13: |
| This describes the daily process of testing. We will flash our devices, look for bugs, then look to write autopilot tests for any fixed bugs to prevent that bug from re-occurring. | |
| Line 14: | Line 15: |
| TODO = Reporting a bug = == There is a crash file in /var/crash == If the bug is a crash and there is a crash file in /var/crash, you can report it with apport. 1. Connect to the device with adb |
== Daily Image Testing == === Install the current stable image === Flash your phone to the latest stable image (note: we want the "current" image of touch_ro). If you have not yet bootstrapped or unlocked your phone's bootloader, [[http://developer.ubuntu.com/start/ubuntu-for-devices/installing-ubuntu-for-devices/|see the install page for more information]]. |
| Line 25: | Line 19: |
| $ adb shell | $ ubuntu-device-flash }}} In order to wipe clean and start fresh with the latest current image: {{{ $ ubuntu-device-flash --no-backup |
| Line 28: | Line 26: |
| 2. Check for any crash file in /var/crash 3. Report a bug with apport: |
To test with the proposed image: |
| Line 33: | Line 28: |
| $ apport-cli /var/crash/name_of_crash_file.crash | $ ubuntu-device-flash --channel=devel-proposed |
| Line 36: | Line 31: |
| 4. Follow the instructions | |
| Line 38: | Line 32: |
| 5. To the question "What would you like to do? Your options are:", answer "S: Send report" | ==== Perform a system update ==== Alternatively, once you are running the ubuntu image you may perform a system update from the device itself. To do this, select 'system-settings' application, then click update. Press the download button, and once the download is completed, the install button. |
| Line 40: | Line 35: |
| 6. Wait for the upload to proceed | [[attachment:update.gif|Visual Walkthrough of updating]] |
| Line 42: | Line 37: |
| 7. Upon upload you'll be presented with a prompt like: | [[attachment:noupdate.png|Example of no updates found]] |
| Line 44: | Line 39: |
| {{{ *** To continue, you must visit the following URL: |
=== Check to see what's changed === A list of changes between stable images can be [[http://people.canonical.com/~ogra/touch-image-stats/current/|found here]]. The list contains the list of changing packages. |
| Line 47: | Line 42: |
| https://bugs.launchpad.net/ubuntu/+source/upstart-app-launch/+filebug/<UUID> | Use these changes to drive exploratory testing below. |
| Line 49: | Line 44: |
| You can launch a browser now, or copy this URL into a browser on another computer. | === Test === * Run through the list of new features and changed areas of the phone, paying specific attention to spot regressions and verify the features are working. |
| Line 51: | Line 47: |
| * Perform smoke testing on a specific packageset of the phone. | |
| Line 52: | Line 49: |
| Choices: 1: Launch a browser now C: Cancel }}} |
* File bugs against any issues encountered |
| Line 57: | Line 51: |
| 8. Copy the URL to a browser running on your desktop/laptop, it is more convenient than running a browser on the phone to report bug and proceed with the bug reporting process in launchpad | * Remember this is exploratory testing, so try and break things! |
| Line 59: | Line 53: |
| 9. Then follow the usual tagging plan and triaging process | === Report Bugs === See the [[QATeam/TouchTesting/BugReporting|Reporting a Bug page]] for details. |
| Line 61: | Line 56: |
| == There is no crash file == | === Add a testcase for reported bugs === Once testing is completed, review the list of reported bugs with the [[https://bugs.launchpad.net/bugs/+bugs?field.tag=touch-needs-autopilot| touch-needs-autopilot tag]]. Pick a testcase to [[https://wiki.ubuntu.com/Touch/CoreApps/Testing/ContributeAutopilotTestcase|automate using autopilot]] that tests the bug to ensure the same bug doesn't reappear. So we can track work happening on these bugs, follow this process while writing the test; |
| Line 63: | Line 59: |
| 1. Connect to the device with adb | 1. Remove the '''touch-needs-autopilot''' tag from the original bug 1. Open a new bug against the same project, tagged with '''qa-touch-automated''' 1. Assign yourself to the bug and update progress as usual. When you release the test, close the bug with fix released. |
| Line 65: | Line 63: |
| {{{ $ adb shell }}} |
=== Confirm an existing bug === Not finding any bugs yourself? Look at [[QATeam/TouchTesting/BugReporting#Triaging|Triaging]] and confirm bugs others have found. |
| Line 69: | Line 66: |
| 2. Find the right package (TODO: Document difference between preinstalled dpkg based packages and click package) | = Tips and Tricks = |
| Line 71: | Line 68: |
| {{{ $ ubuntu-bug maliit-framework What would you like to do? Your options are: S: Send report (4.3 KB) https://bugs.launchpad.net/ubuntu/+source/maliit-framework/+filebug/<UUID>? }}} 3. On your local machine, click on the link or copy/paste it into your browser. = Test Activities = Describe here any specific test activities. = TODO = * Add link to reference documentation (how to flash, specific testing like MIR, how to troubleshoot, ...) |
Check out these [[QATeam/TouchTesting/Tips|handy tips]] to help you while testing. |
Touch Test SWAT
Welcome to the testing page of the Touch Testing SWAT. These pages provide up to date information of current testing activities for Touch.
Test coordination
Mailing List - The quality mailing list ubuntu-quality@lists.ubuntu.com is used to coordinate testing activities with all participants.
IRC - join the Ubuntu QA in #ubuntu-quality and and Ubuntu Touch #ubuntu-touch IRC channels on freenode.
Test process
This describes the daily process of testing. We will flash our devices, look for bugs, then look to write autopilot tests for any fixed bugs to prevent that bug from re-occurring.
Daily Image Testing
Install the current stable image
Flash your phone to the latest stable image (note: we want the "current" image of touch_ro). If you have not yet bootstrapped or unlocked your phone's bootloader, see the install page for more information.
$ ubuntu-device-flash
In order to wipe clean and start fresh with the latest current image:
$ ubuntu-device-flash --no-backup
To test with the proposed image:
$ ubuntu-device-flash --channel=devel-proposed
Perform a system update
Alternatively, once you are running the ubuntu image you may perform a system update from the device itself. To do this, select 'system-settings' application, then click update. Press the download button, and once the download is completed, the install button.
Visual Walkthrough of updating
Check to see what's changed
A list of changes between stable images can be found here. The list contains the list of changing packages.
Use these changes to drive exploratory testing below.
Test
* Run through the list of new features and changed areas of the phone, paying specific attention to spot regressions and verify the features are working.
* Perform smoke testing on a specific packageset of the phone.
* File bugs against any issues encountered
* Remember this is exploratory testing, so try and break things!
Report Bugs
See the Reporting a Bug page for details.
Add a testcase for reported bugs
Once testing is completed, review the list of reported bugs with the touch-needs-autopilot tag. Pick a testcase to automate using autopilot that tests the bug to ensure the same bug doesn't reappear. So we can track work happening on these bugs, follow this process while writing the test;
Remove the touch-needs-autopilot tag from the original bug
Open a new bug against the same project, tagged with qa-touch-automated
- Assign yourself to the bug and update progress as usual. When you release the test, close the bug with fix released.
Confirm an existing bug
Not finding any bugs yourself? Look at Triaging and confirm bugs others have found.
Tips and Tricks
Check out these handy tips to help you while testing.
QATeam/TouchTesting (last edited 2015-01-22 17:18:23 by xdsl-83-150-81-40)