Ubuntu Wiki

• Launchpad Entry: kernel-lucid-bug-handling

• Created:

• Contributors:

• Packages affected:

Summary

It is still apparent that the incoming volume of kernel bugs remains problematic to manage. The ratio of incoming bugs to resources still doesn't scale. The goal of this spec is to re-evaluate our current bug management work flow and practices and determine a more effective way to manage kernel bugs.

Release Note

Due to the increasing volume of incoming kernel bugs an improved and sustainable approach to bug management is being introduced. See KernelTeam/KernelLucidBugHandling and KernelTeamBugPolicies for more information.

Rationale

During the Karmic cycle we started implementing Kernel Bug Days as well as writing kernel arsenal scripts to combat the growing volume of kernel bugs. If we stop to look at the numbers, up until Karmic Beta we were seeing much better improvements compared to Jaunty. In the 5 month time span between Karmic opening and Karmic Beta, there was an increase of ~1000 open bugs of which the bugs in a New state increased by ~300. Compare that to the 5 month time span between Jaunty opening and Jaunty Beta, there was a far less total increase of ~300 open bugs but the bugs in a new state increase by ~750.

However, then Karmic Beta landed and we were completely smothered with the increased bug volume. Again, just looking at the statistics, between Karmic Beta and Karmic Final (a time frame of less than 1 month), we saw an increase of ~1500 open bugs of which the bugs in a New state increased by ~1250. Compare that to the time frame between Jaunty Beta and Jaunty Final which saw an increase of ~900 open bugs of which the bugs in a New state increased by ~700.

Then we compound this with the fact that things like kerneloops wasn't disabled until after Karmic's Final release and in the 1 week following Karmic's release, we saw an additional increase of ~1200 open/new bugs.

It's obvious that we need to continue to examine our current bug management work flow and determine how we can effectively deal with this ever growing volume of kernel bugs.

User stories

* Bob reported a bug originally against Hardy and hasn't updated his bug since. However, this bug remains open and contributes to the large volume of bugs that must be tracked. This bug should be confirmed against the latest kernel or closed. * John reports a kernel oops but is unaware that this has already been reported and has a possible fix for the issue. John's bug should be detected via a bugpattern or launchpad script to mark it as a duplicate. * Sue reports a high impact bug against the Lucid Alpha release but it lacks the necessary debug information to help narrow down the root cause. This bug gets set to incomplete and lost amongst the massive volume of existing kernel bugs. * Joe reports a regression he's seen from Karmic to Lucid but it never gets tagged as a regression and goes unnoticed. * Sally has reported 3 bugs all of which have gone untouched. She feels it's pointless to report bugs and thus stops reporting them.

Design

Some ideas to improve handling new bugs coming in:

1. Interactive apport hooks for the kernel. Note: patch already accepted to apport for Lucid. See bug 444672
   • Is the issue confirmed with the usptream kernel?
   • Is this a regression?
   • Is this reproducible?
   • If not reproducible, how often does the issue appear?
   • Should we add something like "Is there a known patch to fix this issue?" so we know the ones we should be able to close quickly?
2. LP lib scripts to search for known issues with fixes (for ex kerneloops)

   • Ex https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bugs?field.searchtext=ieee80211_teardown_sdata

     • https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/457523 master bug

   • Let's make sure to add these to the kernel arsenal for all to share
3. Improve arsenal scripts
   • Need to make these smarter so they can be run without intervention
   • See lp:arsenal-devel for current set of scripts and lp:~leannogasawara/arsenal/kernel for additional merge request

Ideas for handling old bugs:

1. Bug Bankruptcy

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

CategorySpec