Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Loading history is slow => config option to limit number of lines? #759

Open
balinterdi opened this Issue · 11 comments

4 participants

@balinterdi

I've just looked into why launching a rails console takes ~60 secs in a mid-sized Rails project and found the culprit was Pry loading its history.

Turned out my history file had ~17.000 lines and took about 40-50 seconds to load. This seems slow at first although I did not really dig into this. More importantly, I don't think a history file should keep as many elements by default as users will surely get bitten by this.

I think Pry.config.history.max_lines should have a sane default (e.g 100) and it should be configurable. I'll try to find the time to get a stab at this.

@kyrylo
Collaborator
~[master]% wc -l ~/.pry_history
65508 /home/curacao/.pry_history
~[master]% time pry -e 'exit'
pry -e 'exit'  1.50s user 0.14s system 96% cpu 1.690 total

Please, note that I have many plugins installed.

@kyrylo
Collaborator

Oh, and what is the version of your Pry? The latest available is 0.9.10. You can check your version from command line (pry -v) or directly within Pry (with help of pry-version command).

@banister
Owner

@balinterdi my history file is enormous and loads in no time, modern pry versions should load history quickly, very old versions of pry had a problem loading large files and were slow.

@balinterdi

Interesting. I also have 0.9.10 installed and yet experienced the slow loading.

» bundle exec pry -v                                                                                                                                                    
Pry version 0.9.10 on Ruby 1.9.3
@ghost
➜  ~P git:(master) ✗ cat ~/.pry_history | wc -l
    6676
➜  ~P git:(master) ✗ time pry -e 'exit'
pry -e 'exit'  0.31s user 0.09s system 68% cpu 0.581 total
➜  ~P git:(master) ✗ pry -v
Pry version 0.9.10 on Ruby 2.0.0➜  ~P git:(master) ✗ 

Maybe the bottleneck is somewhere else?

@ghost

I still think it is a good idea to have a cutoff number, I rarely use all of the history available to me but I might use 10-20 lines of history. 100 is large(relatively speaking), and loads fast.

@kyrylo
Collaborator

@balinterdi, how did you determine it was Pry history that caused such slowness?

@balinterdi

I profiled the below code by inserting print statements (ok, puts statements :) ) between each of the lines with a timestamp affiexed:

  def self.initial_session_setup

    return if !initial_session?

    # note these have to be loaded here rather than in pry_instance as
    # we only want them loaded once per entire Pry lifetime.
    load_rc if Pry.config.should_load_rc
    load_local_rc if Pry.config.should_load_local_rc
    load_plugins if Pry.config.should_load_plugins
    load_requires if Pry.config.should_load_requires
    load_history if Pry.config.history.should_load
    load_traps if Pry.config.should_trap_interrupts

    @initial_session = false
  end

and found that load_history took 40-50 seconds to load. Deleting ~/.pry_history and relaunching rails console brought it down to 8 secs (from the 50-60 seconds experienced before).

@balinterdi

After some more poor man's profiling I've found that it is pushing to the Readline::HISTORY that is slow:

In history.rb:

    def load
      @loader.call do |line|
        @pusher.call(line.chomp)
        @history << line.chomp
      end
      @saved_lines = @original_lines = @history.length
    end

If I comment out the @pusher.call line then history is loaded in no time.

@balinterdi

Readline::History is just an Enumerable object so I don't see any reason why pushing to it should be slow.

That said, is there a reason Pry loads the history lines one-by-one? Maybe we could cut down on startup time by loading all lines at once.

Regarding why others don't see the slowness I do, I suspect I might have an issue with the readline lib but I'm not sure how that works.

@rf-
Owner

Readline::HISTORY actually has some strange properties. For example, calling to_a on it when it gets big is very expensive (which is why we maintain a separate @history array). I've never seen pushing to it be slow, though, so this is an interesting problem.

@rf- rf- closed this
@rf- rf- reopened this
@rf- rf- modified the milestone: v1.0.0
@rf- rf- added the feature label
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.