Server

Differences between revisions 4 and 5
⇤ ← Revision 4 as of 2009-11-06 20:16:47 →
Size: 1155
Editor: CPE0006258ec6c1-CM000a73655d0e
Comment:
← Revision 5 as of 2016-12-08 09:01:35 → ⇥
Size: 3016
Editor: powersj
Comment: Started re-write, not done
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
As we gear up for the Ubuntu 10.04 LTS release, the Ubuntu Server Team is refocusing its efforts on testing. == Overview ==
Areas under test and opportunities for testing of Ubuntu Server.
Line 3: Line 4:
This page is intended to gather Ubuntu Server Testing ideas, prioritize the deliverables, and track their implementation. == ISOs and Cloud Images ==
As the delivery vehicle for obtaining Ubuntu Server the ISOs and cloud images must be tested. This is the first touch users have with Ubuntu Server and therefore a positive user experience is essential.
Line 5: Line 7:
== What to Test ==
 * Traditional Ubuntu services
  * apache
  * bind
  * mysql
  * postfix
  * postgres
  * dovecot
  * openldap
  * ssh
  * samba
  * php
  * open-iscsi
  * drbd
  * bacula
  * opie
  * ...
 * UEC
  * various topologies
  * power management
 * Virtualization
  * linux KVM as a hypervisor
  * qemu-kvm as the userspace virtualization driver model and vm launcher
  * libvirt as the virtualization management layer
  * live and cold migration
 * Live upgrades
 * EC2 Images and their behavior
 * VMBuilder
 * Server Load/Performance		 === ISOs ===
ISOs are produced for two release cadences:
Line 35: Line 10:
== How to Test ==
 * Automation
  * autotest
  * checkbox
  * ec2. We will use this to test our ec2 images, but there is no reason that we can't run tests inside those images.
 * Manually
  * community testing
  * servers within KVM
 * ISO Testing
 * Enabling testsuites		   * [[http://cdimage.ubuntu.com/ubuntu-server/daily/|Daily ISOs]] are produced to test the latest packages seen during a development release
  * [[http://releases.ubuntu.com/|Point Release ISOs]] are produced to make updates to install components to fix bugs or add hardware support
Line 46: Line 13:
== When to Test ==
 * Daily ISOs
 * Milestone ISOs		 ==== Automated Testing ====
Testing of the daily ISOs is done in an automated fashion whenever a new "pending" daily ISO is produced. The testing is done across both the amd64 and i386 architectures, although work is underway to do the same for ppc64el. This automated testing consists of the following:

 1. Download and verify hashes
 1. Smoke test of a single, default install to verify no obvious issues
 1. Promote "pending" to "current"
 1. Installs with a variety of preseed and tasksel options set to validate specific software packages and configuration settings.

This testing occurs on Jenkins instance using jobs built by [[https://code.launchpad.net/~canonical-platform-qa-jenkins/qa-jenkins-jobs/trunk|Jenkins Job Builder files]] and here is the code used to validate the [[https://code.launchpad.net/~ubuntu-server-qa/ubuntu-test-cases/server-tests-raring|preseed and tasksel]].

==== Manual Testing ====
Because of the importance of the ISO experience manual testing is still required. This testing occurs via the [[http://iso.qa.ubuntu.com/|ISO Testing Tracker]].

Testing is outlined by point-release milestone. Inside each milestone has a list of the architectures to test and inside there a list of test cases. Above each test case there is a link to the specific ISO that needs to be tested. The layout looks like the following:

{{{
Xenial 16.04.1
 |
 ---> amd64
      |
      ---> Install (Default)
      ---> Install (Default + crypted LVM)
      ---> Install (Default + LVM)
      ---> Install (Bind9)
      ---> Install (Default + RAID1)
      ---> etc.
 ---> i386
 ---> ppc64el
 ---> powerpc
 ---> arm64
 ---> s390x
}}}

The [[https://bazaar.launchpad.net/~ubuntu-testcase/ubuntu-manual-tests/trunk/files/head:/testcases/image/|test cases]] are available for updating.


=== Cloud Images ===


== Packages ==


== Usage ==

=== Personal Scenarios ===

=== Deployments ===

=== Live Patch ===


== Bug Work ==
As a part of owning quality it is important to stay involved in reported defects. During the defect resolution process, bugs should have tests verifying that the bug is actually solved. These tests should then be put back into the package's tests (e.g. dep8) or pushed back to the upstream package's project.

=== Daily Triage ===

=== Backlog Triage ===

Overview

Areas under test and opportunities for testing of Ubuntu Server.

ISOs and Cloud Images

As the delivery vehicle for obtaining Ubuntu Server the ISOs and cloud images must be tested. This is the first touch users have with Ubuntu Server and therefore a positive user experience is essential.

ISOs

ISOs are produced for two release cadences:

  • Daily ISOs are produced to test the latest packages seen during a development release

  • Point Release ISOs are produced to make updates to install components to fix bugs or add hardware support

Automated Testing

Testing of the daily ISOs is done in an automated fashion whenever a new "pending" daily ISO is produced. The testing is done across both the amd64 and i386 architectures, although work is underway to do the same for ppc64el. This automated testing consists of the following:

  1. Download and verify hashes
  2. Smoke test of a single, default install to verify no obvious issues
  3. Promote "pending" to "current"
  4. Installs with a variety of preseed and tasksel options set to validate specific software packages and configuration settings.

This testing occurs on Jenkins instance using jobs built by Jenkins Job Builder files and here is the code used to validate the preseed and tasksel.

Manual Testing

Because of the importance of the ISO experience manual testing is still required. This testing occurs via the ISO Testing Tracker.

Testing is outlined by point-release milestone. Inside each milestone has a list of the architectures to test and inside there a list of test cases. Above each test case there is a link to the specific ISO that needs to be tested. The layout looks like the following: 

Xenial 16.04.1
 |
 ---> amd64
      |
      ---> Install (Default)
      ---> Install (Default + crypted LVM)
      ---> Install (Default + LVM)
      ---> Install (Bind9)
      ---> Install (Default + RAID1)
      ---> etc.
 ---> i386
 ---> ppc64el
 ---> powerpc
 ---> arm64
 ---> s390x

The test cases are available for updating.

Cloud Images

Packages

Usage

Personal Scenarios

Deployments

Live Patch

Bug Work

As a part of owning quality it is important to stay involved in reported defects. During the defect resolution process, bugs should have tests verifying that the bug is actually solved. These tests should then be put back into the package's tests (e.g. dep8) or pushed back to the upstream package's project.

Daily Triage

Backlog Triage

Testing/Server (last edited 2017-01-26 18:23:44 by powersj)