Procedures

Differences between revisions 19 and 32 (spanning 13 versions)
⇤ ← Revision 19 as of 2011-06-06 10:11:56 →
Size: 7752
Editor: host185-242-dynamic
Comment:
← Revision 32 as of 2013-10-23 16:57:28 → ⇥
Size: 5648
Editor: adsl-98-70-51-119
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
<<Include(Testing/Header)>> <<Include(QATeam/Header)>>
Line 19: Line 19:
 * Now you have to have an account in the tracking system, which lets you report your test results. We are going to use it to generate data (it will be publicly available).
Line 21: Line 20:
  1. Click on "Log In" and "Create New Account"   1. Click on "Log In" button, enter your Launchpad account and authenticate using Single-Sign-On service.
Line 23: Line 22:
  '''''Note:''''' ''If you have already registered for one of the other QA sites (e.g. ISO Tracker) you don't need to do it for the laptop tracker also, unless you are going to test more than a laptop. In this case you need a new registration for each additional laptop you own. A nice suggestion could be to name each new tracker username as Username-Laptopmake, where Laptopmake = Asus, Acer, Toshiba etc. ''
 * Set up your profile providing all these required following info (mandatory):


  1. '''Laptop make and model'''

  {{attachment:make-model.png}}<<BR>>

  Fill in with the manufacturer name and the model of your laptop, as outputted by the commands reported. On Lenovo laptops you need also to run 'sudo dmidecode -s system-version' to get the first part of the model name as in this example:
  ||<tablestyle="font-size: 0,9em; text-align: center; border: 1px; width: 850px; background-color: #f1f1ed; font-weight: bold; table-layout: fixed;"> Model name ||

  ||<tablestyle="text-align: center; border: 1px; vertical-align: center; width: 850px; table-layout: fixed;"> '''First Part'''|| '''Second part''' ||
  || !ThinkPad X61 Tablet || 776294G ||


  1. '''Laptop hardware info'''

   
  || {{attachment:lspci.png}} || Fill in with the output of ''lspci -v'' command. ||

  || {{attachment:lsusb.png}} || Fill in with the output of ''lsusb'' command without any external device attached. ||

  || {{attachment:lshw.png}} || Fill in with the output of ''lshw -short'' command. <<BR>><<BR>>''(You'll get a warning that you should be a super user or the list could be incomplete. Don't worry about that)'' ||		  * Now you need to set up an hardware profile for your laptop.
  * [[https://wiki.ubuntu.com/QATeam/Hardware|Checkout this page for information]] on how to do it and place it online to link to during your results submission.
Line 48: Line 26:
  1. '''Launchpad username '''


  || {{attachment:laptop-lpusername.png}} || Fill in with your Launchpad username ||
Line 55: Line 27:
 * You can sign up for individual test cases on any ISO. This helps everyone to see which tests are being covered and sends you a notification email when a new milestone is available to be tested, provided you have opted in to email notification (user settings).

 {{attachment:iso-email.png}}		  * You can sign up for individual test cases on any ISO selecting them and clicking on Subscribe. This helps everyone to see which tests are being covered and sends you a notification email when a new milestone is available to be tested.
Line 64: Line 34:
{{attachment:iso-images1.png}}
{{attachment:iso-download.png}}		 {{attachment:iso-download1.png}}
''Download page example''
Line 67: Line 37:
The images currently being tested are listed on [[http://laptop.qa.ubuntu.com/|laptop.qa.ubuntu.com]] with a direct download link and the appropriate rsync command for your convenience (click {{attachment:info.png}} for the download page).
If you already have the previous milestone you can use rsync to update it , as this will reduce the turn around time on testing. Rsync only downloads the parts of the image that have changed. See [[https://help.ubuntu.com/community/RsyncCdImage|instructions on using rsync]].		 The images currently being tested are listed on [[http://laptop.qa.ubuntu.com/|laptop.qa.ubuntu.com]] with a direct download link and the appropriate zsync or rsync command for your convenience (click {{attachment:info.png}} for the download page).
If you already have the previous milestone you can use zsync to update it , as this will reduce the turn around time on testing. Zsync only downloads the parts of the image that have changed and typically takes about 10% of the full download time with reasonably fresh images. See [[https://help.ubuntu.com/community/ZsyncCdImage|instructions on using zsync]].
Line 72: Line 42:
On the tracker you will be presented with a list of available milestones to test, the stable ones like Lucid Lynx 10.04.2 and those of the current development release of Ubuntu. On the tracker you will be presented with a list of available milestones to test, the stable ones like Oneiric Ocelot and those of the current development release of Ubuntu.
Line 81: Line 51:
To subscribe to a specific ISO and tests, select the architecture from the list. Then select the tests you wish to subscribe to and then hit ''subscribe''. You will receive an email announcing there are tests available, provided you also opt-in to email notification on your profile. To subscribe to a specific ISO and tests, select the architecture from the list. Then mark the checkboxes next to the tests you wish to subscribe to and then hit ''Subscribe''. You will receive an email announcing there are tests available.
Line 85: Line 55:
The test results are tracked on the laptop testing tracker website at [[http://laptop.qa.ubuntu.com/|laptop.qa.ubuntu.com]]. Once you are registered, you can log in to report your test results. The website is divided into several sections, and results are reported based on the following criteria: The test results are tracked on the laptop testing tracker website at [[http://laptop.qa.ubuntu.com/|laptop.qa.ubuntu.com]]. You need to log in to report your test results. The website is divided into several sections, and results are reported based on the following criteria:
Line 93: Line 63:
Laptop test cases normally last for at least half an hour. To avoid duplicating efforts, it is good to let others know that you have started a test case. Laptop test cases normally last for at least half an hour. To avoid duplicating efforts, it is good to let others know that a test is in progress.
Line 95: Line 65:
To mark a test as "Started" select the result "Started" and submit the result. It will show up in the list with a "clock" icon. To mark a test as "In progress" select the result "In progress" and submit the result. It will show up in the list with a "clock" icon.
Line 97: Line 67:
{{attachment:started.png}} {{attachment:inprogress1.png}}
Line 101: Line 71:
When you have conducted a test you can report the results in the Laptop tracker with a general comment and bug numbers if applicable. When you have conducted a test you can report the results in the Laptop tracker providing the link to you laptop hardware profile (Ubuntu Friendly or wiki one), a general comment and bug numbers if applicable.
Line 103: Line 73:
{{attachment:testreport.png}} {{attachment:testreport1.png}}
Line 113: Line 83:


== Results summary ==

{{attachment:laptop-menu.png}}

You can see a summary of results clicking on the link '''Laptops summary''' present in ''Action'' left-hand menu and are reported based on the following criteria:
 * Laptop manufacturer (Acer, Dell, Asus etc.)
 * Laptop model
 * Release (Lucid, Natty etc.)
 * Architecture (i386, amd64)
 * Category of hardware (e.g., Audio, Video, Input/Output etc.)
 * Piece of hardware (e.g., Speakers, Keyboard, Hotkeys, etc.)

Preparations

What you need

  • A laptop to be tested in a live environment or in an installed system. (A virtual machine cannot be used because we need to test real hardware.)

  • A moderate level of Linux experience so you are able to find the relevant log files and produce useful bug reports: ReportingBugs

  • Some available time just after a milestone release to perform tests and file reports.

Setting up

laptoptesting-logo.png

  • If you do not already have one, set up a Launchpad account so you can report bugs. If you have signed up to edit the Ubuntu wiki you will already have a Launchpad account.

  • Join the Ubuntu Laptop Testing Team on Launchpad.

    1. Go to laptop.qa.ubuntu.com.

    2. Click on "Log In" button, enter your Launchpad account and authenticate using Single-Sign-On service.
  • Now you need to set up an hardware profile for your laptop.
  • Select the ISO images you expect to test and look over the relevant test cases, making sure you have the test setup you need.
  • You can sign up for individual test cases on any ISO selecting them and clicking on Subscribe. This helps everyone to see which tests are being covered and sends you a notification email when a new milestone is available to be tested.

Downloading images

Milestone ISO images are found at releases.ubuntu.com.

iso-download1.png Download page example

The images currently being tested are listed on laptop.qa.ubuntu.com with a direct download link and the appropriate zsync or rsync command for your convenience (click info.png for the download page). If you already have the previous milestone you can use zsync to update it , as this will reduce the turn around time on testing. Zsync only downloads the parts of the image that have changed and typically takes about 10% of the full download time with reasonably fresh images. See instructions on using zsync.

Testing

On the tracker you will be presented with a list of available milestones to test, the stable ones like Oneiric Ocelot and those of the current development release of Ubuntu.

Every milestone we will be starting a new testing cycle. Everybody is free to submit their results in the weeks that follow the official release of a milestone (although we encourage you to run them as soon as possible, so developers can have enough time to fix the bugs).

You have to choose the milestone and the architecture (32 or 64 bit) you want to test and each one has a set of test cases associated with it. Click test.png on the tracker to see details of the test case.

laptop-report.png

To subscribe to a specific ISO and tests, select the architecture from the list. Then mark the checkboxes next to the tests you wish to subscribe to and then hit Subscribe. You will receive an email announcing there are tests available.

Reporting results

The test results are tracked on the laptop testing tracker website at laptop.qa.ubuntu.com. You need to log in to report your test results. The website is divided into several sections, and results are reported based on the following criteria:

  • Ubuntu architecture (e.g., Ubuntu i386 or amd64)
  • Category of hardware (e.g., Audio, Video, Input/Output etc.)
  • Piece of hardware to test (e.g., Speakers, Keyboard, Hotkeys, etc.)

Starting a test

Laptop test cases normally last for at least half an hour. To avoid duplicating efforts, it is good to let others know that a test is in progress.

To mark a test as "In progress" select the result "In progress" and submit the result. It will show up in the list with a "clock" icon.

inprogress1.png

Adding test reports

When you have conducted a test you can report the results in the Laptop tracker providing the link to you laptop hardware profile (Ubuntu Friendly or wiki one), a general comment and bug numbers if applicable.

testreport1.png

You can mark a test as Passed or Failed. In the Laptop test tracker we mark as failures those experiences that prevented us to do what we wanted to achieve in that test case. I.e. If we want that Audio works, and you can't hear from speakers, that's a failure. If you hear slight noise from speaker, the test didn't fail as such. We would mark that as success, but will link the non-critical bugs that we find.

Filing bugs

As with all testing it is important to file bugs. If you find a bug, please, search if it has already been reported, and if it hasn't, report it yourself. You should also refer to the bug filing/debugging guide for that specific piece of hardware to make sure you are aware of known issues and have attached the relevant log files.

To mark a test as Failed it is mandatory to link the test report with at least one bug in Launchpad.

Testing/Laptop/Procedures (last edited 2013-10-23 16:57:28 by adsl-98-70-51-119)