Ubuntu Wiki

Introduction

This topic is a bit overwhelming, so I'm open to suggestions for how to narrow and customize based on the audience desire. Basically though I am going to talk about the debuggers I've written or worked on for GNU bash, GNU make, Python, and Ruby. Topics can range from historical to practical.

In the practical arena, I could show how to use to debug an GNU automake project. There could be a theoretical area such as how to structure writing a debugger. (The ones for Ruby and Python are interesting here.)

In the past people have asked me in the past, "what's the fascination with the debugger thing?" Or another favorite remark from a contractor is "I never use a debugger, I just write code". As a systems administrator one is often confronted with lots of code that you didn't write. You suspect the code might not even be written all that well, but you are more sure that there isn't much in the way of documentation or comments in the code. And you've got a problem to solve. Here is where a debugger comes into play.

I've also heard the observation that you don't need a debugger because you go into a Bash, Python or Ruby shell and just start typing statements in the language. Again if you have a high-level idea of what's going on, that's great. However the hardest part here is getting the context correct. I need a foo object from class Foo and then need to hook the bar object from class Bar which means importing or requiring module baz into foo before I can call this method fubar that is of interest to me. With a debugger you work on code that purports to work and largely does. So for the most part, most of the necessary setup has been done for you. In a sense a good debugger is really not all that different than a shell, it's just that the context has already been set up for you. We'll see that in the Python debugger later.

Why model debuggers off of gdb?

Because it is

• fairly complete --- suggests how to expand lesser debuggers (e.g. pdb and Ruby's debug). For example, consider signal handling.

• well known --- learning one helps learn another. It helps programs learn too!

• it's there --- why invent a new interface? Because is GPL can cull from gdb manual, with citation

Issues in Writing or Using a Debugger

• Speed --- debuggers can slow down your program
• Matching virtual reality to reality; which naturally leads to...

• Heisenbugs (and speed contributes to this as well)

GNU bash

Historical stuff

The sordid history of trap DEBUG. Originally like other signals, it came after a statement. But for debugging you need to stop before. Think "rm -fr /" and when you'd like to know about it. It took about 2 years after I posted a suggestion for the change to get into bash (and I never got an acknowledgment about this). Debug support was going way too slowly. About 4 years between those releases.

So I forked the bash code to add debug support and timestamped history. This code was incorporated into bash 3.0. Many thanks to Masatake YAMATO.

Practical example

• PS4 trick

• Using to debug SYSV start/stop scripts.

• debugger set_trace trick? Note: set_trace is deprecated in favor of debugger

Note: if you are going to debug configure scripts, you want the readarray builtin. This has since been encorporated into bash 4.0 and is not needed in that and later versions

At this point we should have seen three paradigms of debugging that will come up over again in the other debuggers:

• line tracing
• invoking from the debugger at the outset
• calling the debugger (somehow) from inside the program.

There's one more technique that's a little hard to use in bash but is more prevalent elsewhere

• post-mortem debugging.

Work since the talk

In 2008, I did something in between a port and a rewrite of this to support ksh and zsh and colorized output via pygments. As before, changes were needed in the shells themselves to support more accurate location reporting and to be able to do more program introspection. Much to my delight, these enhancements were done quickly and with little involvement on my part. But this means you do need a recent release of these shells. The debugger code is maintained on github. See http://github.com/rocky/zshdb and http://github.com/rocky/kshdb.

GNU make

I have a short (13 minute) video which is really intended to be the first part of a more extended example.

Historical stuff

It's deja vu all over again.

What needed to be done (same as in bash):

• better position location reporting
• keep target stack (like a call stack)
• get tracing working first and then add a interactive command loop

Practical example

Method I use to solve a Makefile problem:

• use normal make (until failure)

• run again with remake -x

• if above doesn't work narrow target and run with remake -X target

Show:

• make basic debugging: make -d basic

• make tracing: remake -x

• post-mortem debugging
• 'step' and 'next'

• showing dependencies and write command

• Debugging a automake-generated Makefile.

Work since the talk

The original version was based on GNU make in-between version 3.80 and 3.81. In 2008, a fresh conversion was done for 3.81, 3.82, and 4.1. I've added a simple target commenting mechanism and a Ruby rake-like --tasks flag to show interesting targets. https://vimeo.com/97397484 has the second part of the video alluded to above.

Python

Historical stuff

There are two gdb-like debuggers. The stock python debugger, pdb, and one that grew from that, called pydb. The name pydb is because at about the time pdb was developed Richard Wolff was working in parallel on a debugger to be embedded on ddd. He has since retired and with his blessing I've taken over the name and have been sort of maintaining the ddd embedding as well.

Stack frame display dilemma: do Python programmers draw their trees with the roots at the bottom?

Practical example

I have a short 15-minute video showing off some non-pdb things.

Example: how ipython handles help --- signal handling (initially added by Matt Fleming as part of a Google 2006 Summer-of-code project. Possibly some of the thread debugging features.

Recent and Future work

Integration into ipython was recently done. Release 1.21 added a number of usability conveniences like better command completion, and pydoc help.

Remote debugging. This is needed to hook into many IDEs like Eric, IDLE, winpdb (and therefore SPE), and Eclipse. The bad news though is that each IDE has defined it's own protocol for working remotely.

In late 2008, I started working on a complete rewrite. Some features planned:

• remote debugging

• much more modular (and more along the lines of ruby-debug)

• distribution via egg and pip packaging

Code is available by running:

• pip install trepan

It lives in git in http://github.com/rocky/python2-trepan and http://github/rocky/python3-trepan.

Ruby

There are in fact two ruby debuggers. debug which comes with the base Ruby install and ruby-debug by Kent Sibilev. Both are roughly gdb like. Kent's debugger is I think is coded the cleanest of any debugger I've seen (although it does have some warts). Each command is its own class. By way of comparison, in the stock Python debugger a command module is used. Commands are methods in this (sub)class whose names start with the "do_". This is a pattern akin to one used in unit testing where tests start with "test_". But having a class per command is cooler. It means that commands can have properties. One property in ruby-debug is whether you can run this command if the program has crashed. (The help command you can run, but the step command you can't).

Practical Example

require 'debug' trick? Stopping in a Rails unit test.

Note: since this talk, ruby-debug has become the de-facto debugger that is used in Ruby, JRuby, Ruby/Rails, merb and other frameworks. The successor to the require 'debug' trick is now require 'ruby-debug/debugger' or some variation of that. See Calling the debugger from inside your Ruby Program and The Debugger Module for even more detail.

Future Work and Work since the talk

Nowadays there are number of choices for debuggers.

I rewrote the ruby-debug for Ruby 1.9 and 2.1 to make it less Rube-Goldberg. But this code needs a patched Ruby. You can get at: https://sourceforge.net/projects/ruby-debugger-runtime/. After that is installed:

• gem install trepanning

The code is on github in https://github.com/rocky/rb-trepanning .

Debuggers since the talk

Devel::Trepan

After years of hesitation, I wrote a debugger for Perl. The reason I waited so long is that Perl has always had a powerful debugger. But it was yet another one-off any other debugger. And the code was like too many other debuggers, one file with 10K lines. (It's not the 10K I mind, my debugger probably has that many lines too; its the fact that it is all in one file.)

trepanjs

trepanjs is a debugger for nodejs. It is more gdb-like. However, command syntax follows Javascript notation, is does its predecessor "node debug" instead of a shell-like command set that gdb follows. For example you write:

• list(5, 10)

rather than:

• list 5 10

Note: as a special hack to command parsing, I do allow special cases where no parameters to a command are given or a single parameter is given. So you can write:

• list

or:

• list 5

without using parenthesis. However, if the the first parameter is a string, you'd still need to provide quotes. So you need:

• help '*'

rather than

• help *

And none of these extensions to debugger command parsing are available if instead of using trepanjs you use the stock node debugger node debug.

CategoryDebugging