EuRuKo 2010 talk
Clone this wiki locally
Below here is a somewhat heavily edited transcript. Since this talk, the debugger name has changed. (When Polish people tell you that you have an unpronounceable name for your debugger, you know it must be true.) As a result of the debugger change, links have changed as well. I’ve taken the opportunity here to use the current names.
Previously Piotr Szotkowski had given an excellent talk about improving the performance of a prime sieve program written in Ruby. I make mention of that various place in the talk. Piotr also showed pictures of Dennis Ritchie, John McCarthy, and Matz mentioning that all had Unix beards. So before the video started I made some off-hand comments mentioning that I was the first person to talk with a beard (and was of a similar age and hairline).
I’ll also note that Matz’ laptop is a ThinkPad, same as mine and perhaps in his keynote talk you can find gkrellm running on the side just as I run. The other neat thing about Matz’ keynote was that he put up a slide of things to do and debug followed by VM were top of the list.
Hi – My name is Rocky Bernstein. It is a really fantastic conference!
I am just doing this talk as an advertisement for code I have been working on for 9 about months and it will probably about be another 9 months before it is really complete.
The code I am going to talk about is located at
It requires support from these two things:
The other reason I am giving this talk is I need to build support to petition to get some of this code into Ruby 1.9 itself.
And I would like to thank EuRuKo for giving me an opportunity to present this talk.
Why would one use a Debugger?
I find that people who learn Ruby as their first programming language – I am an old guy – do not know what a debugger is. And if they do, they are not aware that there are debuggers for Ruby.
[problem with microphone. My delight as a former sysadmin at the possibility of using redundant hardware, here: two microphones]
So the first half of the talk is what a debugger is and so I am going to use the new debugger to show off debugging. And most of it is going to be a live demo. I hope things don’t crash.
Many people don’t know what a debugger is and the popular books on Ruby generally don’t mention debuggers. When they do, they don’t say much about them.
Sometimes it may happen that you need to fix bugs in code that others have written, and that that code is neither modular, doesn’t comes with tests, nor is well documented. This is typically where you use a debugger.
In some respects, a debugger isn’t all that different from going into irb or script/console in [Ruby on] Rails. So I want to elaborate on that comment. I am going to do that with a Ruby on Rails program. And if you don’t know Ruby on Rails, that’s even better because it will give you a feeling of bewilderment that a system administrator often experiences.
My boss says to me, “Here is a rails application that was written by a guy, we fired him, he doesn’t work here. Now you to go change this code”.
[add browser screen shot for http://localhost:3000/posts]
I run the code, there is my Rails 2 application. It was created right from one from the post scaffold in Rails. And what you see here is all that I know.
If you look at the log for that — this is the console for that — the log says there is a “posts/index”. If you know a little bit about Ruby, the file for that would be “app/views/posts/index.rhtml.erb”. And I happen to have that here
<h1>Listing posts</h1> <table> <tr> <th>Title</th> <th>Body</th> </tr> <% @posts.each do |post| %> <tr> <td><%=h post.title %></td> <td><%=h post.body %></td> <td><%= link_to 'Show', post %></td> <td><%= link_to 'Edit', edit_post_path(post) %></td> <td><%= link_to 'Destroy', post, :confirm => 'Are you sure?', :method => :delete %></td> </tr> <% end %> </table> <br /> <%= link_to 'New post', new_post_path %>
This code is created right from the [rails] generator. I’ve worked with two Ruby on Rails programmers and in fact neither of them know what a debugger was before I worked with them. And this is in fact what both of them would do.
They would run script/console the see @posts in the file, you might look at the controller and you find a model called Posts. So here we go. [Notice already I had to understand that there is a controller file I would have to look at which later on isn’t required.]
[Microphone stand delivered. Technology is improves all the time!]
I want to find out what’s going on in this code so I write
$ ./script/console Loading development mode >> @posts = Post.find(:all)
And I want to see what “post.title” is so now have to type another assignment:
>> post = @posts
Now I have the first element of that array:
>> post.title # works
Now I get down to this code here this link_to code. Can I run that? So far I’ve been pretty successful. No. [ link_to isn’t defined in this the script/console context]
So this is what my comment is. You’ve seen I had to set up some context. I had to set up @posts, I had to set up post. And it looks like I need more context for link_to.
So now instead let’s try it with the debugger I am working on.
[Adds line of code:] <% @posts.each do |post| debugger %> ...
Above is the boilerplate code for now. In the ruby-debug, the existing debugger, it’s a little bit simpler. I need to require the rbdbgr gem inside development.rb for Rails 2. For Rails 3 configuration is a little bit different. But the gem [trepanning] has been required.
And if you are familiar with Rails if you change the file and refresh, boom, automatically stuff happens. Nothing happens. [I didn’t save the file.] This time for sure (If you know Rocky and Bullwinkle, that is a line ).
What has happened is that I am stopped inside the middle of the program and now when I type @posts, I see the same thing as I saw before. I see that variable post and let’s look at link_to. Boom. that works. If I want to know edit_posts is, that works. …
As I am typing commands here it is being read by a debugger but some people like irb much better, so you can also type irb and it is like script/console but those variables, @posts, and post are set up.
So the only point here is this isn’t really no different than being in script/console but all the context [is set up]. Recall, I don’t know that much about the program.
And that’s why you might use a debugger.
Why a rewrite?
My first introduction to a Ruby debugger was the one that comes with Ruby, called debug.rb. Probably nobody has used it other than me. I started extending that and then I learned of a debugger by Kent Sibilev’s, who I’ve never met, called ruby-debug . [See also ruby-debug in 30 seconds, the ruby-debug manual or this screencast ]. It was much more complete and faster. So I abandoned my changes and I started contributing to that.
There are many cool things that Kent Sibilev invented for ruby-debug. I don’t think he has written a debugger before that. And so it was very interesting for someone coming at this new and look at things a little bit differently.
One novel and unusual workaround is this notion that you have to turn on and off debugger tracking. And the reason for that is because the run-time does not support what you would need in a debugger [by default].
So when tracking is turned on there is additional code gets run. And then that means it slows things down and makes Piotr unhappy. [Piotr’s talk was about speeding up a particular Ruby program.]
Over time, due to suggestions of folks like you, I added more features and more workarounds. I got to a stage where I realized that if I rewrote some of the fundamental aspects of the debugger and of Ruby itself, some of the problems I encountering would be fixed more simply and better. So that is what I want to give examples of next. I am just going to give two examples.
This [changing the runtime] was very reminiscent of a situation I had IBM Research. I worked at IBM research a while ago, and something like that went on there in architecture. For the RISC architecture, a fundamental question we were interested in is what should be done in a high-level language and what should be done in hardware, as opposed to adding microcode to make mainframes or Intel architectures look all the same. So this was also in the back of my mind.
I am trying to figure out what is too slow to be outside of the runtime system and what has to be in the runtime system.
Stopping Before a Return Statement
One deficiency of ruby-debug and the debugger that comes with Ruby is their inability to stop before a return statement. Let me show that.
I am going to first show the existing debugger called ruby-debug.
def five ; 5 end def stop_before_return five # Can't stop after call but before return in ruby-debug end x = stop_before_return y = File.basename(__FILE__) z = 0
[The debugger command] “frame” is a hacky way to refresh the display in the Emacs interface I am using.
I want to stop right after this method five is called but before the end happens. I am showing this in the old debugger. One way to [try] to do that is to set a temporary breakpoint using a command called “continue”. And I go to line 3 with it. So now I am stopped right before [the call of method] five:
... def stop_before_return --> five # Can't stop after call but before return in ruby-debug end x = stop_before_return ...
Anybody who has used gdb or ruby-debug may know that command to “step over” a function is called “next”. So we will do that.
And now you will see I have stepped over the return.
... def stop_before_return five # Can't stop after call but before return in ruby-debug end x = stop_before_return --> y = File.basename(__FILE__) ... ...
I was in method “stop_before_return”, and then I did a “step over” and there was no subsequent statement and so down I am. [I meant to run “step into” or “step” so show that this makes matters worse because then I automatically skip over two levels of method calls.]
What I see most people do, and it is really horrible, is they change the program like this:
... def stop_before_return x = five # Can't stop after call but before return in ruby-debug return x end ...
When you think about the style of programming that Piotr likes to do, with his functional style, this is really horrible. So one of the goals is that I want to support that kind of functional style.
We will just to the same thing with the debugger I am working on.
Compare this line here:
with this line there:
The control-Z (^Z) happens to be because I am in Emacs.
But the important thing is the inoccuous double-dash line here,
-- (/src/git/... ^^
that means that I am stopped at a line event. In the new debugger you can type “next”, but you can also type “step over”.
(rbdbgr): step over <- (/src/git/rbdbgr-papers/EuRuKo/code/stop-before-return.rb:4)
And this is what I wanted to see. That icon value has changed to and that is a “return event” and the next line
<- (/src/git/... R=> 5 ^^^^^
is saying that the return value is 5. And I do “set return”, I can change the return value:
(rbdbgr): set return "flaczki" Return value was: 5 New value is: "flaczki"
It doesn’t have be the same type, but that’s the way Ruby is. I do a step,
New value is: "flaczki" (rbdbgr): step
and print x:
... New value is: "flaczki" (rbdbgr): step -- (/src/git/rbdbgr-papers/EuRuKo/code/stop-before-return.rb:6) y = File.basename(__FILE__) (rbdbgr): print x flaczki
and there we go. But wait, there’s more! In the next line we have a function that is implemented as a C call. And that is something else that ruby-debug and existing debuggers can’t handle. I do another step
.. y = File.basename(__FILE__) (rbdbgr): print x flaczki (rbdbgr): step CFUNC File#basename("/src/git"...) C> (/src/git/rbdbgr-papers/EuRuKo/code/stop-before-return.rb:6) y = File.basename(__FILE__) (rbdbgr):
And that [“C>”] is the icon for a C call. This here “/src/git”… is the initial part. It is elided if you want to see entire parameter you do info args:
info args 1: "/src/git/rbdbgr/papers/EuRuKo/code/stop-before-return.rb" (rbdbgr):
If I do another step, I am right before a [“C”] return event, and again I can change the value:
info args 1: "/src/git/rbdbgr/papers/EuRuKo/code/stop-before-return.rb" (rbdbgr): step <C (/src/git/rbdbgr-papers/EuRuKo/code/stop-before-return.rb:6) R=> "stop-before-return.rb" y = File.basename(__FILE__) (rbdbgr):
and again I can the value, set return “grzyb”:
.. <C (/src/git/rbdbgr-papers/EuRuKo/code/stop-before-return.rb:6) R=> "stop-before-return.rb" y = File.basename(__FILE__) (rbdbgr): set return "grzyb" Return value was "stop-before-return.rb" New value is: "grzyb" (rbdbgr):
I like that word (“grzyb”) an awful lot. I’ve had 4 kinds [in Poland] — so far. And now we look at y:
(rbdbgr): set return "grzyb" Return value was "stop-before-return.rb" New value is: "grzyb" (rbdbgr): s # same as step -- (/src/git/rbdbgr-papers/EuRuKo/code/stop-before-return.rb:7) z = 0 (rbdbgr): p y "grzyb" D=> "grzyb"
So the deal here is that now the debugger can see the calls and returns of C functions, [even though] the underlying mechanism in Ruby does allow stopping in C calls and returns. ruby-debug and the debugger that comes with Ruby are written in such a way that they can only handle “line” events.
Stepping into an evaluated string
One more example which is motivated by code I’ve used myself but also code that is in Rails. The idea is debugging inside an eval string. First, this is with Ruby 1.8 debugger on the file eval.rb:
x = <<EOF a = 2 b = 3 y = 4 # line 4 EOF eval(x)
I have a string there [x = <<EOF …] and I’m going to evaluate it. The problem is that when I go to evaluate it and I want to see what’s in the string in ruby-debug that doesn’t work. So let me show it.
$ rdebug eval.rb (eval.rb:1) x = <<EOF (rdb:1)
So I “step” and then I do another “step”:
(rdb:1) step (eval.rb):6 (rdb:1) step (eval):1 (rdb:1)
and it says “eval” and it says “1”, i.e. “(eval):1”. And if I do another step I it says “(eval):2”. A common debugger command is list to show the source code:
(eval):1 (rdb:1) step (eval):2 (rdb:1) list [-3,6] in (eval) *** No sourcefile available for (eval)
and it is saying “I don’t really have a clue where you really are.” So now let’s try with the new debugger:
$ rbdbgr eval.rb -- (eval.rb:1) x = <<EOF (rbdbgr): step -- (eval.rb:6) eval(x) (rbdbgr): step CFUNC Kernel#eval("a = 2\nb"...) C> (eval.rb:6) eval(x) (rbdbgr):
Here we see that I am in Kerne#eval.
And what’s cool here is that the string that is going to be eval’d next is a parameter. So this is where tracing into C calls now comes into play. The debugger can now look at that string and use that. So I do a “step”:
CFUNC Kernel#eval("a = 2\nb"...) C> (eval.rb:6) eval(x) (rbdbgr): step -- (/tmp/eval-20100712-7273-1qk0iqp.rb:1) a = 2
and there we are. The astute and (and those that know emacs) may notice that the debugger created a temporary file /tmp/eval-20100429-12737-1qk0iqp.rb) and both debuggers have a mechanism for remapping filenames or line ranges.
Stepping into an evaluated function
That actually is the easier case to handle. The harder case is what happens in rails where the methods are strings that are defined and those then are run. Here is an example:
$ cat eval-fn.rb $x = <<EOF def five 5 end EOF eval($x) five rbdbgr eval-fn.rb -- (eval-fn.rb:1) $x = <<EOF
As before [I “step over” a statement]:
(rbdbgr): n -- (eval-fn.rb:6) eval($x)
$x has a string in it and I eval it,
(rbdbgr): n -- (eval-fn.rb:7) five (rbdbgr): s METHOD Object#five() -> ((eval):1 via eval-fn.rb:7) five
and if I “list”:
five (rbdbgr): list 2 def five 3 5 4 end 5 EOF 6 eval($x) 7 five 8 (rbdbgr):
What you see in here [“(eval): 1 via eval-fn.rb:7)”] is that I am a little bit more honest [should be “helpful”] than what ruby-debug shows you. In ruby-debug just shows you. ruby-debug shows you eval line one. Here I am saying it is eval’d but it is eval’d via that line [7 which is shown in the list]. And the way the debugger gets that is that it looks at the call stack and picks up the last place that it knew about.
There is a command called “set substitute string” which does the instruction-sequence to string contents to filename remapping.
(rbdbgr): set substitute string . $x
“.” means remap the instruction sequence that I am currently at. And I’ve rigged this example to make $x a global variable so I can get at it.
I now if I do a step.
-- (/tmp/(eval)-$x-20100730-23462-1vhez0k.rb:2) 5 (rbdbgr): list 1 def five 2 5 3 end
Same thing as before. A temporary file was created and remapping done.
There are alot of things I haven’t mentioned. https://github.com/rocky/rb-trepanning/wiki/Cool-things has what is different between the new version (rbdbgr) and the old one (ruby-debug).
And there instructions on how to install: https://github.com/rocky/rb-trepanning/wiki/How-to-Install-rb-trepanning
What’s more important to me is getting feedback from you all. dziękuję