CrashReporting

Differences between revisions 1 and 23 (spanning 22 versions)
⇤ ← Revision 1 as of 2006-07-31 12:05:45 →
Size: 3727
Editor: 203-118-156-188
Comment: Use cases, Design problems, Comparisons
← Revision 23 as of 2007-05-15 18:05:57 → ⇥
Size: 3955
Editor: 87
Comment: move reasons for authenticated bug filing to design
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## Register at https://launchpad.net/distros/ubuntu/+specs
 * '''Launchpad entry''': none yet
 * '''Created''': [[Date(2006-07-31T12:05:45Z)]] by MatthewPaulThomas
 * '''Contributors''': MatthewPaulThomas
 * '''Packages affected''': 		 * '''Launchpad entry''': https://launchpad.net/distros/ubuntu/+spec/crash-reporting
 * '''Packages affected''' `apport`, Malone
Line 9: Line 6:
Currently apport sends crash reports as public Malone bugs. We will improve this process to avoid exposing potentially sensitive data to the public and avoid sending unwanted bug email to developers.

Because a few bugs cause most crashes, this system should eventually involve a database of crash reports, automatically aggregated by type so that developers can allocate their time to the top crashers.
Line 11: Line 12:
We want to improve the reliability of Ubuntu software, by making it easy for people to report details about crashes.

See also:
 * [:UbuntuDownUnder/BOFs/AutomatedCrashReporting:AutomatedCrashReporting]
 * BugReportingTool		 The original idea was to create an entirely separate crash database and web frontend for it. However, since we need a lot of the features of a bug tracker (assignments, release tracking, searching, duplication, email/web interaction, getting feedback from reporters), this database will just use Malone in an appropriate way.
Line 21: Line 18:
 * Aunt Tillie was adding to her genealogical records in Gramps when it crashed. She doesn't know anything about reporting bugs, and has no desire ever to report any.

 * Millie was logging in to her bank account when Firefox crashed. She's used to clicking the "Send" button for Windows Error Reporting, but it would be a bad idea for her to report this problem since anyone could find her banking password in the crash informaation.

 * Thunderbird has just crashed on Billie's machine for the third time in three minutes. She angrily blats away the error reporting alert within 0.8 seconds of it opening.		  * Millie was logging in to her bank account when Firefox crashed. She's used to clicking the "Send" button for Windows Error Reporting, so she does the same for apport. Her banking password can be found in the crash information, so this needs to be treated confidentially.
Line 29: Line 22:
The crash reporting interface is an interruption; it is not at all related to the user's goal (designing a logo, recording genealogy, online banking, etc). To make things worse, the crash itself has probably just eaten some of the victim's work. They will likely be angry with computers in general and Ubuntu in particular. Therefore the crash reporting interface must be ''extremely'' simple, apologetic, and well-humored. === Authenticated bug filing ===
Line 31: Line 24:
Another design problem is that most people who have come from Windows 2000 or later, or Mac OS X 10.0 or later, will be used to crash reports that are confidential to Microsoft, Apple, or trusted ISVs. For Edgy, Ubuntu crash reports will not be like this: if someone reports a bug and attaches their crash report, anyone will be able to see it. Bug reports can be marked private after the fact, but most of the prevention of information leaking must occur in the crash reporting interface itself. We currently do not want bugs to be filed anonymously, because:
Line 33: Line 26:
=== Comparisons ===  * we want to keep the possibility of contacting the reporters for further information,
 * the current testing community is large enough to supply us with much more reports that the triagers can handle anyway (and enough bugs to keep the reprocessing infrastructure busy), and
 * mostly this would just result in even more duplicates.

=== New approach of crash bug reporting workflow ===

 0. Create Launchpad users which control access to the bug, split by main/restricted and universe/multiverse.

 0. Apport files bugs as private/nonsecurity by default, with only the crash reprocessing bot subscribed (Launchpad user ''apport'', the "Apport retracing service").

 0. The retracing bot generates the symbolic stack traces, handles duplicates (see UbuntuSpec:apport-crash-duplicates), and removes the core dump attachment. Then it subscribes the relevant team to the bug.

 0. The triaging teams regularly inspect crash reports (preferably prioritized by number of duplicates). After verifying that a stack trace does not contain sensitive information, they can set the bug to "public".

=== Comparisons to other crash feedback agents ===
Line 36: Line 43:
 * [http://developer.apple.com/technotes/tn2004/tn2123.html Apple TN2123: CrashReporter]
Line 38: Line 46:
 * [http://unsanity.org/archives/000424.php Smart Crash Reports]

=== Edgy ===

For Edgy, it will not possible to report a bug directly. So we should (a) apologize for the error, (b) make it easy for people to report a bug if they want to, and (c) make it let easy for them to reopen the program in question.

attachment:alert-edgy.jpg attachment:alert-edgy-no-reopen.jpg

(This follows standard HIG layout for an error alert with alternate buttons.)

The alert must not contain the "Reopen" button if the program is one of those that is automatically restarted anyway (`gnome-panel` or `nautilus`).

=== Edgy+1 ===

For Edgy+1, we assume there will be some sort of database (like Launchpad's Oops system) that can record crash reports, for aggregation and bug reporting by QA staff; crash victims do not fill out bug reports themselves.

attachment:funky.jpg
Line 58: Line 49:
=== Code ===  0. Create two new Launchpad teams ''ubuntu-crashes-main'' (for components `main` and `restricted`) and ''ubuntu-crashes-universe'' (for components `universe` and `multiverse`), set a "black hole" contact email address and put the `ubuntu-core-dev`, respectively `ubuntu-dev` teams into it. This avoids sending crash email to these teams (explicit requirement from our desktop team) and allows for later adjustments for accessing crash bugs.
Line 60: Line 51:
== Unresolved issues ==  0. Malone currently has the ability to set tags in the blob that gets uploaded to the cloakroom. This needs to get extended to mark a bug as private/nonsecurity.

 0. (design is self-explanatory)

 0. (design is self-explanatory)

Summary

Currently apport sends crash reports as public Malone bugs. We will improve this process to avoid exposing potentially sensitive data to the public and avoid sending unwanted bug email to developers.

Because a few bugs cause most crashes, this system should eventually involve a database of crash reports, automatically aggregated by type so that developers can allocate their time to the top crashers.

Rationale

The original idea was to create an entirely separate crash database and web frontend for it. However, since we need a lot of the features of a bug tracker (assignments, release tracking, searching, duplication, email/web interaction, getting feedback from reporters), this database will just use Malone in an appropriate way.

Use cases

  • Willy was creating a logo for his soccer club in Inkscape when it crashed. Being the family's Ubuntu expert, he feels a responsibility to help improve the system. He's reported one or two bugs in Malone before, though it wasn't a particularly enjoyable experience.
  • Millie was logging in to her bank account when Firefox crashed. She's used to clicking the "Send" button for Windows Error Reporting, so she does the same for apport. Her banking password can be found in the crash information, so this needs to be treated confidentially.

Design

Authenticated bug filing

We currently do not want bugs to be filed anonymously, because:

  • we want to keep the possibility of contacting the reporters for further information,
  • the current testing community is large enough to supply us with much more reports that the triagers can handle anyway (and enough bugs to keep the reprocessing infrastructure busy), and
  • mostly this would just result in even more duplicates.

New approach of crash bug reporting workflow

  1. Create Launchpad users which control access to the bug, split by main/restricted and universe/multiverse.

  2. Apport files bugs as private/nonsecurity by default, with only the crash reprocessing bot subscribed (Launchpad user apport, the "Apport retracing service").

  3. The retracing bot generates the symbolic stack traces, handles duplicates (see apport-crash-duplicates), and removes the core dump attachment. Then it subscribes the relevant team to the bug.

  4. The triaging teams regularly inspect crash reports (preferably prioritized by number of duplicates). After verifying that a stack trace does not contain sensitive information, they can set the bug to "public".

Comparisons to other crash feedback agents

Implementation

  1. Create two new Launchpad teams ubuntu-crashes-main (for components main and restricted) and ubuntu-crashes-universe (for components universe and multiverse), set a "black hole" contact email address and put the ubuntu-core-dev, respectively ubuntu-dev teams into it. This avoids sending crash email to these teams (explicit requirement from our desktop team) and allows for later adjustments for accessing crash bugs.

  2. Malone currently has the ability to set tags in the blob that gets uploaded to the cloakroom. This needs to get extended to mark a bug as private/nonsecurity.
  3. (design is self-explanatory)
  4. (design is self-explanatory)

CategorySpec