CrashReporting

Differences between revisions 3 and 22 (spanning 19 versions)
⇤ ← Revision 3 as of 2006-08-01 10:22:23 →
Size: 5646
Editor: 203-118-156-188
Comment: tweaks
← Revision 22 as of 2007-05-15 17:53:56 → ⇥
Size: 3919
Editor: 87
Comment: redesign spec for using Malone
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## Register at https://launchpad.net/distros/ubuntu/+specs
 * '''Launchpad entry''': none yet
 * '''Created''': [[Date(2006-07-31T12:05:45Z)]] by MatthewPaulThomas
 * '''Contributors''': MatthewPaulThomas
 * '''Packages affected''': 		 * '''Launchpad entry''': https://launchpad.net/distros/ubuntu/+spec/crash-reporting
 * '''Packages affected''' `apport`, Malone
Line 9: Line 6:
Currently apport sends crash reports as public Malone bugs. We will improve this process to avoid exposing potentially sensitive data to the public and avoid sending unwanted bug email to developers.

Because a few bugs cause most crashes, this system should eventually involve a database of crash reports, automatically aggregated by type so that developers can allocate their time to the top crashers.
Line 11: Line 12:
We want to improve the reliability of Ubuntu software, by making it easy for people to report details about crashes. ''See also'' [:UbuntuDownUnder/BOFs/AutomatedCrashReporting:AutomatedCrashReporting], BugReportingTool. The original idea was to create an entirely separate crash database and web frontend for it. However, since we need a lot of the features of a bug tracker (assignments, release tracking, searching, duplication, email/web interaction, getting feedback from reporters), this database will just use Malone in an appropriate way.

We currently do not want bugs to be filed anonymously, because:

 * we want to keep the possibility of contacting the reporters for further information,
 * the current testing community is large enough to supply us with much more reports that the triagers can handle anyway (and enough bugs to keep the reprocessing infrastructure busy), and
 * mostly this would just result in even more duplicates.
Line 17: Line 24:
 * Aunt Tillie was adding to her genealogical records in Gramps when it crashed. She doesn't know anything about reporting bugs, and has no desire ever to report any.

 * Millie was logging in to her bank account when Firefox crashed. She's used to clicking the "Send" button for Windows Error Reporting, but it would be a bad idea for her to report this problem since anyone could find her banking password in the crash informaation.

 * Thunderbird has just crashed on Billie's machine for the third time in three minutes. She angrily blats away the error reporting alert within 0.8 seconds of it opening.		  * Millie was logging in to her bank account when Firefox crashed. She's used to clicking the "Send" button for Windows Error Reporting, so she does the same for apport. Her banking password can be found in the crash information, so this needs to be treated confidentially.
Line 25: Line 28:
The crash reporting interface is an interruption; it is not at all related to the user's goal (designing a logo, recording genealogy, online banking, etc). To make things worse, the crash itself has probably just eaten some of the victim's work. They will likely be angry with computers in general, and Ubuntu in particular. Therefore the crash reporting interface must be very simple and apologetic. === New approach of crash bug reporting workflow ===
Line 27: Line 30:
Another design problem is that most people who have come from Windows 2000 or later, or Mac OS X 10.0 or later, will be used to crash reports that are confidential to Microsoft, Apple, or trusted ISVs. For Edgy, Ubuntu crash reports will not be like this: if someone reports a bug and attaches their crash report, anyone will be able to see it. Bug reports can be marked private after the fact, but the crash reporting interface itself must take some responsibility for discouraging leaking of sensitive information.  0. Create Launchpad users which control access to the bug, split by main/restricted and universe/multiverse.
Line 29: Line 32:
''Mozilla.org can [http://www.mozilla.org/quality/qfa.html say that] "Sensitive data, such as passwords, Web sites visited, and e-mail addresses will not be collected". Why can't we guarantee the same?''  0. Apport files bugs as private/nonsecurity by default, with only the crash reprocessing bot subscribed (Launchpad user ''apport'', the "Apport retracing service").
Line 31: Line 34:
=== Comparisons ===  0. The retracing bot generates the symbolic stack traces, handles duplicates (see UbuntuSpec:apport-crash-duplicates), and removes the core dump attachment. Then it subscribes the relevant team to the bug.

 0. The triaging teams regularly inspect crash reports (preferably prioritized by number of duplicates). After verifying that a stack trace does not contain sensitive information, they can set the bug to "public".

=== Comparisons to other crash feedback agents ===
Line 38: Line 45:
=== Edgy ===

For Edgy, it will not possible to report a bug without using Launchpad's Web interface. So we should (a) apologize for the error, (b) make it easy to report a bug, and (c) make it easy to reopen the program in question.

attachment:edgy.jpg attachment:edgy-no-reopen.jpg

If a human-readable name can be found for the program (from a `.desktop` file), the first sentence of the alert should be "Sorry, Name of Program closed unexpectedly." Otherwise, it should be "Sorry, the program “binary-name” closed unexpectedly."

The "If you weren’t doing anything confidential..." satisfies Millie's use case.

Which buttons are present should depend on the program that has crashed.
 * If the crashed program is not now running, the "Leave Closed" and "Reopen" buttons should be present. If the program is running (for example, the automatically-restarted `gnome-panel` or `nautilus`, or a manually-restarted program), these should be replaced by an "OK" button.
 * The "Report a Bug…" button should be present only if the crashed executable belongs to a package in Main or Universe. ''Is this possible?''
The keyboard equivalent for the "Reopen" or "OK" button should be Enter, not a letter.

Clicking "Report a Bug…" should open both a Web browser to Ubuntu's Bugs page in Launchpad; and also a floating window near the top left corner of the screen, containing the bug information.

attachment:edgy-report.jpg

In the floating window, the icon should be draggable into the browser's filepicker to select that file, and the pathname should also be copyable text. The "What does the file contain?" expander should disclose a read-only text field containing the crash log as wrapped text.

=== Edgy+1 ===

For Edgy+1, we assume there will be some sort of database (like [http://talkback-public.mozilla.org/search/start.jsp Mozilla Talkback] or [https://sodium.ubuntu.com/~jamesh/oops.cgi Launchpad Oops]) that can record crash reports, for aggregation and bug reporting by Canonical staff. Crash victims should no longer have to fill out bug reports manually, because it's complicated, and because [http://www.microsoft.com/whdc/maintain/WERHelp.mspx 80 percent of crashes come from 20 percent of the bugs].

This simplifies the initial crash alert a little ...

attachment:funky.jpg

... And it simplifies the resulting reporting interface a lot.

attachment:funky-report.jpg

The window should now be a normal window, not a floating window or a dialog.
Line 75: Line 47:
=== Code ===  0. Create two new Launchpad teams ''ubuntu-crashes-main'' (for components `main` and `restricted`) and ''ubuntu-crashes-universe'' (for components `universe` and `multiverse`), set a "black hole" contact email address and put the `ubuntu-core-dev`, respectively `ubuntu-dev` teams into it. This avoids sending crash email to these teams (explicit requirement from our desktop team) and allows for later adjustments for accessing crash bugs.
Line 77: Line 49:
== Unresolved issues ==  0. Malone currently has the ability to set tags in the blob that gets uploaded to the cloakroom. This needs to get extended to mark a bug as private/nonsecurity.

 0. (design is self-explanatory)

 0. (design is self-explanatory)

Summary

Currently apport sends crash reports as public Malone bugs. We will improve this process to avoid exposing potentially sensitive data to the public and avoid sending unwanted bug email to developers.

Because a few bugs cause most crashes, this system should eventually involve a database of crash reports, automatically aggregated by type so that developers can allocate their time to the top crashers.

Rationale

The original idea was to create an entirely separate crash database and web frontend for it. However, since we need a lot of the features of a bug tracker (assignments, release tracking, searching, duplication, email/web interaction, getting feedback from reporters), this database will just use Malone in an appropriate way.

We currently do not want bugs to be filed anonymously, because:

  • we want to keep the possibility of contacting the reporters for further information,
  • the current testing community is large enough to supply us with much more reports that the triagers can handle anyway (and enough bugs to keep the reprocessing infrastructure busy), and
  • mostly this would just result in even more duplicates.

Use cases

  • Willy was creating a logo for his soccer club in Inkscape when it crashed. Being the family's Ubuntu expert, he feels a responsibility to help improve the system. He's reported one or two bugs in Malone before, though it wasn't a particularly enjoyable experience.
  • Millie was logging in to her bank account when Firefox crashed. She's used to clicking the "Send" button for Windows Error Reporting, so she does the same for apport. Her banking password can be found in the crash information, so this needs to be treated confidentially.

Design

New approach of crash bug reporting workflow

  1. Create Launchpad users which control access to the bug, split by main/restricted and universe/multiverse.

  2. Apport files bugs as private/nonsecurity by default, with only the crash reprocessing bot subscribed (Launchpad user apport, the "Apport retracing service").

  3. The retracing bot generates the symbolic stack traces, handles duplicates (see apport-crash-duplicates), and removes the core dump attachment. Then it subscribes the relevant team to the bug.

  4. The triaging teams regularly inspect crash reports (preferably prioritized by number of duplicates). After verifying that a stack trace does not contain sensitive information, they can set the bug to "public".

Comparisons to other crash feedback agents

Implementation

  1. Create two new Launchpad teams ubuntu-crashes-main (for components main and restricted) and ubuntu-crashes-universe (for components universe and multiverse), set a "black hole" contact email address and put the ubuntu-core-dev, respectively ubuntu-dev teams into it. This avoids sending crash email to these teams (explicit requirement from our desktop team) and allows for later adjustments for accessing crash bugs.

  2. Malone currently has the ability to set tags in the blob that gets uploaded to the cloakroom. This needs to get extended to mark a bug as private/nonsecurity.
  3. (design is self-explanatory)
  4. (design is self-explanatory)

CategorySpec