Runtime-Support for Breakpoints, Step-in, and Step out

Introduction

One of the things my enhanced runtime for Ruby has is support for breakpoints, step over and step out.  I'd like to explain why.

Motivation — Speed

Most debuggers in dynamic languages hook into a callback mechanism provided by the language. The callback is triggered by an event, such as just before running a new statement. Thus, the debugger is written in the same language it is debugging. There are callback hooks to support debugging in Ruby, Python, Perl, and POSIX shell.

Without breakpoint step-out or step-over code, debuggers suffer from the "are we there yet?" syndrome. At each event the debugger code has to check to see if

1.  we have reached a breakpoint, 
2.  we are single stepping, or 
3.  we have  just stepped out of a function. 

In those cases where we want to "step out", "step over" or "step to the next breakpoint", too often the result is "no, don't stop; keep going".

But dynamic languages are 10-60 times slower than C, and we often want to debug large programs. So this means that running step out or step over under a debugger can be an order of magnitude slower than normal execution.

Existing Art

This is one reason why Kent Siblev could bill his ruby-debug as the "fast" debugger: he wrote key parts, including as breakpoint handling, in a C extension, whereas rdebug.rb is written entirely in Ruby.

I have seen the "are we there yet" slowdown also in Python. Nir Aides promoted his winpdb as a fast Python debugger. As an artifact of the way it is written, the debugger allows you to step at fewer places; consequently it does fewer checks.

Interestingly, the venerable Perl interpreter supports debuggers better in this respect. It has support for breakpoints inside the interpreter.

ruby-debug does its checks for breakpoints in a C extension. Neither it nor Perl has support for "step over" or "step out" in the runtime. I wondered if it was possible to do better, and I think the answer is yes, with a caveat described later. So to test whether this was so, I embarked on changing the runtime. So far I am pleased with the results.

The caveat is that these checks incur a small amount of overhead when debugging is not enabled. But Mark Moseley did some initial benchmarking and found the overhead negligible. I'll describe how I drive down the overhead later, but since this a completely separate interpreter, if what you need is that last bit of speed, don't use this interpreter. Switch to it only when it is needed. Down the line it might be interesting to figure out how to seamlessly switch interpreters mid-execution.

Let me now describe how breakpoints, step over, and step out are implemented.

Breakpoint Support

Breakpoints are implemented by allocating essentially a bit vector in those instruction sequences where we need it.

In the instruction sequences where we want a breakpoint, the breakpoints are implemented using what can be thought of as a bit vector although, for speed and simplicity at the expense of space, it is really a byte vector.

Even a bit vector for each instruction is a little more space than is strictly needed because not all indices in an instruction sequence are on VM instruction boundaries and I currently only allow setting breakpoints on statement boundaries, specifically VM "TRACE" instructions. If we allow instruction sequences to be modified, we could set this breakpoint bit inside the trace instruction, but we have to be aware of possible run-time implications. There are security, performance, and architecture advantages in having read-only instruction sequences, although the current Ruby interpreter does not make use of these.

So, the additional test for breakpoint stopping occurs at the same place we test for calling trace hooks. And to further speed things up, we first perform a zero-comparison test to see if there are any breakpoints set in the instruction sequence before indexing into the byte vector to see if the specific instruction has a breakpoint. Since at any given time, there is a small number of breakpoints, often none, we usually don't have to index into the byte vector. We just test for zero.The Ruby code uses a program UNLIKELY() to indicate to a compiler like gcc that this test is likely to be false.

When no trace hooks are in effect the tests are, in fact ,the same tests that Ruby normally uses.

Let's compare what's done here versus what the "fast" ruby-debug has to do. Our code is part of the runtime, so there is no callback even a callback to C code. Just that saves a lot of time. But ruby-debug then has to get the source-code file name and line number; then it uses that to compare against a list of breakpoints. When there are no breakpoints this isn't so bad. But when there are some, it can't do a quick test to see if we are in an instruction sequence containing breakpoints. And we do not need to access the source-code filename or line number. ruby-debug 1.9 and greater could use some of these ideas.

Step Over/Step Out Support

Test to support step over/step out inside Ruby, while straight-forward, are a bit time consuming. To perform step out, we need to check:

• Is the current nesting level less than or equal to the starting nesting level? (exception handling and change the nesting level by more than one)
• Are we in the same thread?

And of course this means accessing the current thread id, and computing the nesting level (which means counting the number of entries in the call stack). As before, when stepping over large chunks of code, there can be a lot of "no, keep going" results which take time before getting to a "yes, stop".

So with runtime support, how can we handle this? The implementation is pretty simple. We record in the stack frame whether we should step-stop in this frame or not. An additional bit indicates whether we should step-stop in frames created from the frame. So there is a small bit-test done on frame creation to set a trace bit or not. But the overhead is measured in terms of a couple of instructions rather than hundreds of instructions, as is the case in Ruby.

And as before we can use the UNLIKELY() pragma to indicate that most of the time we won't be in "step over" or "step out".