Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pry.config.* should not use method missing. #1277

Closed
ogrishman opened this issue Jul 31, 2014 · 7 comments
Closed

Pry.config.* should not use method missing. #1277

ogrishman opened this issue Jul 31, 2014 · 7 comments

Comments

@ogrishman
Copy link

In an IRB session I can do:

MacBook-Pro:sample_app$ irb
2.1.2 :001 > require 'rspec/expectations'
 => true 
2.1.2 :002 > include RSpec::Matchers
 => Object 
2.1.2 :003 >

When I try to do the same thing in pry, I got errors.

MacBook-Pro:sample_app hehe$ pry
[1] pry(main)> require 'rspec/expectations'
=> true
[2] pry(main)> include RSpec::Matchers
/Users/hehe/.rvm/gems/ruby-2.1.2/gems/rspec-expectations-3.0.2/lib/rspec/matchers.rb:902:in `method_missing': private method `print' called for #<RSpec::Matchers::BuiltIn::Output:0x007fd8d96b0ee0> (NoMethodError)
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/output.rb:25:in `block in print'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/output.rb:24:in `each'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/output.rb:24:in `print'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/output.rb:16:in `block in puts'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/output.rb:12:in `each'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/output.rb:12:in `puts'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:379:in `rescue in rescue in show_result'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:373:in `rescue in show_result'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:384:in `show_result'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:334:in `block in handle_line'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_class.rb:369:in `critical_section'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:333:in `handle_line'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:241:in `block (2 levels) in eval'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:240:in `catch'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:240:in `block in eval'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:239:in `catch'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_instance.rb:239:in `eval'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/repl.rb:77:in `block in repl'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/repl.rb:67:in `loop'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/repl.rb:67:in `repl'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/repl.rb:38:in `block in start'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/input_lock.rb:61:in `call'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/input_lock.rb:61:in `__with_ownership'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/input_lock.rb:79:in `with_ownership'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/repl.rb:38:in `start'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/repl.rb:15:in `start'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/pry_class.rb:169:in `start'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/cli.rb:219:in `block in <top (required)>'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/cli.rb:83:in `call'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/cli.rb:83:in `block in parse_options'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/cli.rb:83:in `each'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/lib/pry/cli.rb:83:in `parse_options'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/gems/pry-0.10.0/bin/pry:16:in `<top (required)>'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/bin/pry:23:in `load'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/bin/pry:23:in `<main>'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `eval'
    from /Users/hehe/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `<main>'
MacBook-Pro:sample_app hehe$ 

Below are some of my versions.

MacBook-Pro:~ hehe$ ruby --version
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
MacBook-Pro:~ hehe$ pry --version
Pry version 0.10.0 on Ruby 2.1.2
MacBook-Pro:~ hehe$ 

Thanks.

@JoshCheek
Copy link
Contributor

TL; DR Pry::Config is a linked list implemented through method_missing, including RSpec::Matchers adds Object#output, which interferes with the method missing chain, returning RSpec::Matchers::BuiltIn::Output.new(nil) instead of $stdout. So ultimately, metaprogramming + guerrilla patching leading to a type error. User solution: use extend instead of include. Pry solution: move away from metaprogramming.

Getting to the problem point in Pry

This isn't necessary for understanding the problem, but given that I did the work to figure out how to get to where the problem is, I figured I'd go ahead and post it anyway. Maybe it will be useful context, or maybe just a thing I can refer back to next time I go traipsing around this codebase, and it's easy enough to skip if you're just interested in the problem.

  • For each line of input, we read it in
  • Then eval it
    inside of the pry instance
    (which came from here
    which came from here)
  • Pry#eval delegates through: handle_line,
    process_command_safely,
    process_command
  • Pry#process_command delegates to Pry.commands.process_line, code, ultimately returning false
    • We'll see how Pry#commands was defined and resolved below, when we look at how output gets defined and resolved (they are the same)
      it ultimately returns Pry::CommandSet::Result.new(false)
    • It's not a command (here,
      from here,
      from here,
      from here),
      so process_command returns false,
      so process_command_safely returns false,
  • So handle_line sets @eval_string to the line here
  • Uninteresting stuff happens
  • Finally we pass the input (stored in @eval_string) to evaluate_ruby
  • Which evaluates it in our binding here
  • And saves it in last_result (here
    to here
    defined here
    which returns back up the callstack into handle_line's result variable here)
  • Now that we have the result, we hand it to show_result
  • Which invokes the print method, defined and resolved in the same way as output below here
  • How these are resolved will be explored below, for now, it ultimately winds up at Pry::DEFAULT_PRINT
  • Which calls pry.pager
  • Which returns Pry::Pager.new(self) (where self is the pry instance)
  • It then calls pager.open which yields the best_available pager
  • The best available pager is SimplePager.new(_pry_.output) here
    • The output variable will get saved in @out here from here
  • And then DEFAULT_PRINT will tell the simple pager to print the output prefix here
  • Which delegates to write
  • Which delegates to @out.print
  • Remember, @out came from _pry_.output here
  • This is where the issue is.

Understanding the problem point in Pry

So, based on the above, we ultimately boil down to _pry_.output.print "some string". Now we need to understand how Pry finds this output. This is also what happens for _pry_.commands, and _pry_.print, which I skipped above.

  • Pry#output was defined by config_shortcuts

  • config_shortcuts is defined here, which is made available here

  • We give it :output, which comes from Pry::Config.shortcuts,
    from Convenience::SHORTCUTS,
    which defines it here

  • So ultimately, _pry_.output becomes _pry_.config.output here

  • _pry_.config is an instance of Pry::Config.new here
    from here

  • The Pry::Config class is pretty hostile to exploration, but after reading it a bit, I figured out that it is ultimately a linked list of config objects

    • Each node either has the method that is asked for, or gets its method missing hook hit (comes from Pry::Config::Behavior)
    • If it hits the method_missing hook, it will check to see if it is in an internal @lookup hash here
    • If not, it will delegate to the next node (named @default) in the linked list here
    • At this point, I've got a pretty good idea where the bug is, but lets still see how this resolves.
  • To aid exploration, I make a find_config helper, whose job is just to traverse this chain until it finds the value, reporting on what it explores.

    def find_config(name, stream: STDOUT, config: Pry.toplevel_binding.local_variable_get('_pry_').config)
      puts = lambda { |message| stream.puts message }
      name = name.intern
      puts.call "Searching for #{name.inspect} in #{config.inspect}"
      if config.methods.include?(name)
        puts.call "  Found a #{name.inspect} method!"
        return config.__send__ name
      end
      puts.call "  Does not have an #{name.inspect} method defined"
    
      if config.key? name
        puts.call "  Found an #{name.inspect} key! returning its value!"
        return config[name]
      end
      puts.call "  Does not have a #{name.inspect} key"
    
      if config.default
        puts.call "  Going on to next node in linked list"
        __send__(__method__, name, stream: stream, config: config.default)
      else
        puts.call "  That's the end of the linked list, should get a no method error"
      end
    end
  • Now, lets find config.output:

    > find_config :output
    Searching for :output in #<Pry::Config:0x3fc260f43e88 local_keys=['input','hooks'] default=#<Pry::Config:0x3fc260419210 local_keys=['hooks','editor','pry_doc','has_pry_doc','pry_theme','theme_options'] default=#<Pry::Config::Default:0x3fc260419878 local_keys=['gist','history'] default=nil>>>
      Does not have an :output method defined
      Does not have a :output key
      Going on to next node in linked list
    Searching for :output in #<Pry::Config:0x3fc260419210 local_keys=['hooks','editor','pry_doc','has_pry_doc','pry_theme','theme_options'] default=#<Pry::Config::Default:0x3fc260419878 local_keys=['gist','history'] default=nil>>
      Does not have an :output method defined
      Does not have a :output key
      Going on to next node in linked list
    Searching for :output in #<Pry::Config::Default:0x3fc260419878 local_keys=['gist','history'] default=nil>
      Found a :output method!
    => #<IO:<STDOUT>>
    
  • Okay, so it doesn't find it in the first or second node, and finally it gets down to the last node, which is a Pry::Config::Default, and has an output method
    defined here
    which came from here

  • So if we don't override it, it finds this method, we haven't overridden the key, so it instance evals the proc, which returns $stdin

  • And our pager then prints to it just fine.

  • Have you figured out the bug yet? We are relying on method_missing to delegate our config requests down the linked list.

  • If output were suddenly not missing, it would find that method instead.

  • Lets see how this happens with RSpec.

Understanding the problem point in RSpec

  • We `require 'rspec/expectations' which requires rspec/matchers

  • This makes the RSpec::Matchers module available, which we then include.

  • Since we are in main, the current lexical scope is object (pretty sure that's here, but not completely)

  • So when we include RSpec::Matchers, we are including them into Object, making them available to pretty much everything:

    require 'rspec/expectations'  # => true
    include RSpec::Matchers       # => Object
    "abc".output                  # => #<RSpec::Matchers::BuiltIn::Output:0x007fc891099dc0 @expected=nil, @stream_capturer=RSpec::Matchers::BuiltIn::NullCapture>
    123.output                    # => #<RSpec::Matchers::BuiltIn::Output:0x007fc891099bb8 @expected=nil, @stream_capturer=RSpec::Matchers::BuiltIn::NullCapture>
    Object.ancestors              # => [Object, RSpec::Matchers, PP::ObjectMixin, Kernel, BasicObject]
  • which means that Pry::Config now has RSpec::Matchers methods, including output

  • So when it calls _pry_.config.output, it finds this method and returns RSpec::Matchers::BuiltIn::Output.new(nil) instead of $stdout

  • Being intended for utterly different purposes, this then causes pry to die when it tries to print to it.

What Pry should do

  • Uhm. This Pry::Config class is pretty iffy, IMO.

  • With the Ruby community's apparent fervor for guerrilla patching the shit out of Object, it's not improbable that these issues will continue to manifest.

  • I don't know all the use cases of Config, but it seems to me that hash-style key access would work just fine.

  • If this line] changed to

    @lookup.fetch key do
      value = @default && default[key]
      if value # don't set nil keys
        @lookup[key] = value
      end
    }

    then you could get rid of the method missing. Of course, everyone would have to use hash-style access, but this seems better to me.

  • Alternatively, if there's some reason to use method style access (seems pretty questionable), could define the methods on the config's singleton class at initialization time.

What @ogrishman should do

  • Use extend instead of include

    $ pry -r rspec/expectations
    [1] pry(main)> extend RSpec::Matchers
    => main
    [2] pry(main)> expect(1).to eq 1
    => true
    [3] pry(main)> expect(1).to eq 2
    RSpec::Expectations::ExpectationNotMetError:
    expected: 2
         got: 1
    
    (compared using ==)
    from /Users/josh/.gem/ruby/2.1.1/gems/rspec-expectations-3.0.2/lib/rspec/expectations/fail_with.rb:30:in `fail_with'
    [4] pry(main)>

@ogrishman
Copy link
Author

Great explanation! Thank you so much for the code analysis. Learned a lot from it.

@JoshCheek
Copy link
Contributor

Can we close this? If we want to follow my preference of removing method_missing, we can open another issue about it and reference this.

@ConradIrwin ConradIrwin changed the title Got error to use pry with rspec Pry.config.* should not use method missing. Aug 16, 2014
@ConradIrwin
Copy link
Member

Updated the title to reflect your intention. I'd love to see method_missing removed.

@JoshCheek
Copy link
Contributor

Cool, I'm happy to look at it over the next few days/week, unless anyone else is really excited about it (happy to collaborate, too!)

I don't have a whole lot of context regarding use-cases, is there anything I should know?

@JoshCheek
Copy link
Contributor

I probably won't complete this

So, it's been a week, I started on this, but haven't looked at it since last weekend. It's rough because I'm trying to balance either (1) don't break existing stuff, (2) minimal breaks to existing stuff, and mitigate by finding what would break and just going and fixing them (ie download all gems, filter to candidates that are likely hit, send them PRs), (3) the concern that configuration can simply never be correct without breaking existing stuff.

Anyway, based on my excitement levels when considering various topics, it's very improbable that I'll complete this. Figured state that since it's shitty to say you'll do something and then just let it drag out forever with people waiting.

Some notes on the issue

In order to not have wasted a lot of effort, here's a handful of notes, I have more, but they're all over the place and likely useless.

A good Idea that I can't quite remember

Oh, and I think @rf- made a really suggestion on IRC about... uhm.. maybe defining methods on singleton class? Don't remember exactly, just remember being a little inspired, thinking it could work and dramatically minimize breaking the public api.

Mutability + nesting = situations with no correct answer

Anyway, here's some examples of the issues. I'm not sure if all the issues boil down to this, or if there's several around this theme, since I didn't finish consolidating my ideas.

require 'pry'             # => true
config = Pry::Config.new  # => #<Pry::Config:0x3fd53cd93db0� local_keys=[] default=#<Pry::Config:0x3fd53d92cf00� local_keys=['hooks'] default=#<Pry::Config::Default:0x3fd53d92d1bc� local_keys=['gist','history'] default=nil>>>

# mutating the child is seen by the parent
config.extra_sticky_locals[:a] = 1  # => 1
Pry.extra_sticky_locals             # => {:a=>1}

# assignments on the child are *NOT* seen by the parent
config.extra_sticky_locals = {a: 2}  # => {:a=>2}
Pry.extra_sticky_locals              # => {:a=>1}

It's common (for me, at least) to edit Pry.whatever rather than _pry_.config.whatever, initially this was ignorance, now it's mostly convenience, but it implies certain things, e.g. if something has overridden the config option (say editor), then my change will just not have any impact (because it's in a parent config). But in other situations, I'm explicitly expecting to be sheltered from changes to the global config because I have overridden it in my _pry_.config (this is the assumption the config seems to have been defined under). In general there is no way to know which is the case, so we can never universally do the correct thing.

Incorrect test?

Also, this test

local.default.should == local
seems erroneous. It seems it should return default, not itself (I assume it passes because they are just incidentally ==)

Direction I think it should go

I generally think that when we're ready to break the public api (ie next major release), we need to require that all access to config goes through the instance. And we need to lose the nestedness of the configurations because I think there are just some situations where there is simply no right answer (ie it depends on what the user intended in their brain). In my brain it looks something like "we'll pass the pry instance to all plugins at the appropriate time so they can initialize themselves and any config (all other hooks would belong on the individual config)" and then, if this is inconvenient to access for users, then we can provide a command to make it more convenient

A musing

Last thought, I expect everyone to hate this, but want to state it: I strongly prefer explicitness. With our current setup, assuming we remove the nesting and require all changes to happen to the pry instance's config (to mitigate the mutability issues), there's no reason to not just use a hash. But if we wanted to be more explicit, an object would make sense in that you could require config options to be declared. It then runs some validations like "this doesn't conflict with other config keys", and maybe allows for namespacing, IDK, and then you can use it, and it will blow up if you try to set an undeclared config key (under the assumption that it's a spelling error, probably telling you what you could have set in the error message). Keep in mind, though, that I've never seen this actually done, so I don't know that it would work out well, it just appeals to my design sense.

@0x1eef 0x1eef closed this as completed in 57f4278 Dec 13, 2014
0x1eef pushed a commit that referenced this issue Dec 13, 2014
0x1eef pushed a commit that referenced this issue Dec 13, 2014
@0x1eef
Copy link
Contributor

0x1eef commented Dec 13, 2014

hey guys, I fixed this issue in 27e9199 by using a similar strategy to OpenStruct and as mentioned by @rf-. @JoshCheek thanks for thorough investigation, it was helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants