BugWorkflow

Revision 11 as of 2006-11-10 02:19:16

Clear message

Summary

Use cases

The projects that use Launchpad for bug tracking can be classified into four broad types.

  • Small upstream: Bugs typically go directly from reported to fixed, and there is no formal verification of fixes. Releases early and often (usually), but there is often still some lag between fixes being committed and appearing in a release. Example: Gobby.
  • Large upstream: Large enough that vetting and clarifying new bug reports is a useful volunteer task in itself. Releases may have alphas and betas, so whether a bugfix has been "released" is a difficult question. There may be security fixes for past releases. Example: Firefox.
  • Web application: A production server is updated semi-regularly from mainline or from a branch, with a combination of rollouts and cherrypicked urgent fixes. Therefore the lag between a fix landing in mainline and appearing on the server is highly variable. Example: Launchpad.
  • Distribution: Includes software from many diverse sources, so the developers often do not get involved in fixing lower-severity bugs. Has releases every few months or few years, with security and other critical fixes provided for past releases. Example: Ubuntu.

User stories

Amanda develops a simple program and she wants to use [:Malone] as her BugTrackingSystem. This is a small upstream project. Her project's workflow looks like Figure 1, below.

[http://people.ubuntu.com/~sfllaw/bug-workflow-1.png] Figure 1.

Ben is a developer for a large software project, a GNU/Linux distribution, or a website that uses [:Malone] as its bug tracking system. His distribution's workflow looks like Figure 2, below.

[http://people.ubuntu.com/~sfllaw/bug-workflow-2.png] Figure 2.

Ben's could also have been working for a large upstream or a web application, which would have a similar workflow.

Catie is triaging bugs and she encounters many duplicate roblem reports related to the X server breaking. She wants to know whether the bug is in a stable release, or in the development version.

Dimitri wants to become a developer and is interested in KDE bugs. He needs to find unclaimed bugs

Bug statuses

Rationale

Now that Ubuntu and a variety of upstreams are using Launchpad for bug tracking, we need to revisit the bug statuses available to see whether they allow and encourage useful bugtracking processes.

Currently a bug report has seven possible statuses.

  • Unconfirmed: Initial status. Has not been looked at by a developer or QA volunteer.

  • Needs Info: Needs clarification from the reporter, or someone else who can reproduce the problem.

  • Rejected: Either it is not a bug, or it is a bug that will not be fixed in this particular place.

  • Confirmed: Contains enough information, presented well enough, for a developer to start fixing the bug.

  • In Progress: Currently being fixed.

  • Fix Committed: A fix has been included in the code, but this code is not necessarily available in a released version.

  • Fix Released: The bug is fixed in a released version.

(Whether a bug is a duplicate is orthogonal to its status.)

Current plans are to introduce a "Not For Us" status (see MaloneSimplifications), which would mean that the bug is valid but is not going to be fixed in that particular place. (For example, a bug in Windows that cannot be worked around by an application it affects; or a bug in Ubuntu that is never going to be fixed for Dapper.)

Problems identified with these statuses:

  • The meaning of "Confirmed" is unclear. Ideally it should mean "contains all necessary information for a developer to fix the bug", but many people interpret it as "has been experienced by more than one person".
  • "Needs Info" is ambiguous. Some people treat it as "needs clarification from the reporter", while others treat it as "needs information from someone other than the assignee".
  • When a bug should be moved from "Fix Committed" to "Fix Released" is unclear. Should it be done when a new package is uploaded? When an alpha is released containing the fix? When an OS version is released containing the fix?
  • There is no process for verifying bugs, for those projects that want to do that.
  • The statuses are too wordy generally.

Design

  • Launchpad should have these bug statuses:

    • New: This bug report needs an experienced user or developer to evaluate the report and ensure it has enough information for a developer to reproduce the problem. This is the first state a report is in when filed and nobody has claimed responsibility.

    • Needs Info: Needs clarification from the reporter (and/or possibly other people) to provide sufficient information about the problem.

    • Not a Bug: Not actually a bug. It could be a feature request, designed behaviour, user error, or any number of explanations.

    • Not for Us: A valid bug, but it should not be fixed by this particular project. It may be fixed in another release or forwarded to another project.

    • Triaged: An experienced user or developer has verified that the problem described by this bug report is a valid bug. It also means that there is enough preliminary information that a developer can reproduce or isolate the bug.

    • In Progress: A developer is fixing this bug.

    • Committed: The fix for this bug is available somewhere, but isn't yet available in an official release.

    • Fix Verified: In software that has a verification process, the fix for this bug has been tested to not break existing functionality and has fixed the appropriate faults. This will be an optionally state that can be enabled on a per-project basis.

    • Released: launchpad-bazaar, buildd, or a human has noticed that the bugfix has been included in a release.

The Registrant for a distribution or a product should be able to choose whether bugfixes in that project need verifying; by default they should not (to make life easier when registering small upstream products). The "Fix Verified" state should be available only for projects that have chosen to require verification. If a project that required verification changes to no longer require it, all bugs currently in Fix Verified state should change to Committed (without sending any notification mail).

Implementation

Schema changes:

  • Introduce "bug fixes need verifying" flag for distributions and products.
  • Change bug statuses:

    • rename "Unconfirmed" -> "New"

    • rename "Rejected" -> "Not a Bug"

    • introduce "Not for Us"

    • rename "Confirmed" -> "Triaged"

    • rename "Fix Committed" -> "Committed"

    • introduce "Fix Verified"

    • rename "Fix Released" -> "Fixed".

Code changes:

  • A checkbox for whether a project's bugfixes should be verified.
  • Code that changes "Fix Verified" bugs to "Fix Committed" if verification is turned off.
  • Code that maps old names to new ones, so that existing links to searches are not broken.

Searching for bug contacts

Rationale

Currently, development teams use various methods for tracking unassigned bugs. For instance, the GNOME desktop team assigns any GNOME desktop bug to their own team. But the kernel team does not want this, because it generates too many e-mail notifications.

We need to have some way of generating a list of those bugs that nobody is responsible for fixing, but that are related to a particular team or interest group.

Design

Each package in a distribution has a bug contact, which is usually the maintainer or a team of maintainers. Launchpad should allow searching for all bugs in packages with a particular bug contact, in the same way as it currently allows searching for bugs by reporter and/or assignee. Any such bug that is assigned to Nobody needs a volunteer to fix it.

This should be presented as a new "Bug contact" field in the "People" group, for the Advanced search in Malone.

Implementation

XXX (BjornT to fill in)

Version hint

Rationale

When ResolutionForBug57153 was occurring, we noted a lot of duplicates in bugs being filed against the X server. However, triagers assumed that this was in the development (Edgy) version of X and not the stable version.

Design

Either the Description field should encourage people to enter the version/release where they experience the bug, or there should be a separate Version field. Which of these is appropriate will be determined as follows.

Implementation

We will start by sampling bug reports created with the current interface, measuring the proportion that should contain version numbers but don't.

Then we should tweak the bug-reporting form by encouraging people, in the caption for the "Description" field, to include the version(s) of the package and/or the distribution where the bug occurs. A month later, we will sample bug reports and measure again.

If that does not improve the situation enough, we should add a separate "Version or Release:" field to the bug-reporting form. The text entered in this field should be prepended to the description when the bug report is created. This will not involve any schema changes; it will be done entirely in browser code.

"Enough" will be defined at the discretion of Simon Law.

Potential drawback: The proportion might be increasing or decreasing over time for other reasons, in which case a comparison between only two or three datapoints will be misleading.

Future work

  • Using bug contacts to group bugs that people might be interested in is a bit non-obvious, and doesn't allow overlapping groups. Perhaps in the future we should consider having package groups, so that (for example) Muine can be in both a "Mono" group and a "Multimedia" group, with bugs in it fixable by those watching either. (Cf. Google Code Hosting's use of project tags.)

Unresolved issues

  • "Committed", "Fixed Verified", and "Fixed"? That's a lot. The benefit of separating "Committed" and "Fixed" is that "Committed" bugs still show up in bug listings, so people are less likely to report duplicates of recently-fixed bugs. Is it worth the complexity?

    • Arguably, we don't have an equivalent "Committed" state for "Not a bug" which is why rejected bugs are often duplicated.

Notes

  1. Isolate the following categories of bugs, which should be addressed by Ubuntu developers:
    • Bugs which are specific to Ubuntu

      • Rationale: Either we created these bugs, and so they are our responsibility, or they are only manifest in our environment, in which case we are best equipped to diagnose them.

    • Other bugs which are very severe or relate to core goals of Ubuntu

      • Rationale: These bugs justify the direct attention of Ubuntu developers because they jeopardize our mission of producing high-quality releases

  2. Pass on all other categories of bugs to the appropriate party:
    • Complete bug reports which affect upstream should be made the responsibility of upstream (forwarded)

      • Rationale: The upstream community is better equipped to fix these bugs, and the fix should eventually go upstream anyway, so it's best for it to be fixed there in the first place

    • Incomplete bug reports should be made the responsibility of the reporter by requesting further information ("Needs info")

      • Rationale: Bug reports must be reasonably complete in order to be diagnosed by a developer. Because the development team is relatively small compared to the community of bug reporters, the development team must focus on complete reports.

    • Improper bug reports should be rejected

      • Rationale: Some bug reports are hopeless, for example if the reporter is unresponsive to questions, and must be abandoned in order for the bug system to remain manageable.

attachment:bug-workflow.png

["CategoryBugSquad"]