= bdmurray's notes =

Are we responding effectively and appropriately to bug reports?  Are we doing the right thing?

Are we replying within in a good time frame?
 * mdz has mentioned the time to first response as a metric before

"QA Team should be a mirror for the platform team" - mdz

How many known regressions did we ship in 9.04?  How are doing with 9.10 in comparison?
 * look at quantity of tags that went from regression-potential to regression-release
 * How can I find out the numbers of bugs that were tagged regression- at any point in time?
  * need this for to find out how many regression-potential bugs there were at alpha 3 for example
  * http://people.canonical.com/~brian/complete-graphs/regression-potential/regression-potential.data

Might also be interesting to look at the number of SRU's done a per release basis
 * however this should only compare LTS to LTS and non-LTS to non-LTS

How long does it take to fix the hottest bugs?
 * perhaps use a database query to find the 'hottest' / highest gravity bugs and look at their time to fix
 * or look at karmic fixes report and add-in gravity there

mdz is also concerned about backwater packages that don't get watched but are important to the distribution 
 * consider querying for packages without subscribers for example net-tools - https://edge.launchpad.net/ubuntu/+source/net-tools
  * dhcp is one also but it is only in dapper
 * perhaps also looking to see if the packages are seeded as a way of determining their importance to the distro

Bug gravity should be modified to add points for bug importance

Add importance to -fixes reports to and counts of importance of bugs fixed

Bug gravity should be added to team assigned lists for engineering managers

Take a look at a representative sample of bug reports to see how things went with those bug reports
 * Did the right thing happen with them?
 * Should they really be in the state they are in? 
 * How many are apport-bug / apport-crash reports?
  * Does this have an effect on their triaging?  Are they less likely to be incomplete?
 * How quickly did the first person respond?
 * If the bug is new speculate as to why it hasn't been looked at / triaged yet.

Try to estimate the time it takes to fix a bug report
 * break this down into bugs w/ patches, bugs w/ debdiffs, bugs w/ branches
  * are different kinds fixed faster(?)
 * how many get fixed on a per release basis?
  * this will be useful for estimating the amount of work we can reasonably do
  * How are the bug-fixing reports inadequate for this? - http://qa.ubuntu.com/reports/bug-fixing/

= mdz's notes =

These are the key areas for measuring and reporting that I mentioned on the                                                    
call today:                                                                                                                    
                                                                                                                               
 * regressions                                                                                                                 
 * performance over time                                                                                                       
 * neglected packages                                                                                                          
 * importance                                                                                                                  
 * fixing and triage (separately)                                                                                              
                                                                                                                               
Suggestion: produce a report at certain points in the release cycle,                                                           
comparing our performance at that point to a similar one in the previous                                                       
release cycle