Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entering a literal null byte causes a crash and subsequent session history-load errors #1789

Closed
owst opened this issue Oct 9, 2018 · 9 comments
Closed

Comments

@owst
Copy link
Contributor

@owst owst commented Oct 9, 2018

On Ubuntu 16.04 x64, with xterm and Pry version 0.11.3 on Ruby 2.4.4 I accidentally entered a literal null byte when typing a string value in pry (by hitting Ctrl-space) - pry immediately crashed, and when I reload it, refuses to load the history as it contains a null byte:

(I held ctrl while first typing the space below, so it appeared as if I hadn't added one)

$ pry             
[1] pry(main)> x = 'foo bar'
~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/history.rb:117:in `<<': string contains null byte (ArgumentError)
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/history.rb:117:in `push_to_readline'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/history.rb:48:in `call'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/history.rb:48:in `push'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/pry_instance.rb:267:in `handle_line'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/pry_instance.rb:243:in `block (2 levels) in eval'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/pry_instance.rb:242:in `catch'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/pry_instance.rb:242:in `block in eval'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/pry_instance.rb:241:in `catch'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/pry_instance.rb:241:in `eval'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/repl.rb:77:in `block in repl'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/repl.rb:67:in `loop'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/repl.rb:67:in `repl'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/repl.rb:38:in `block in start'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/input_lock.rb:61:in `__with_ownership'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/input_lock.rb:79:in `with_ownership'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/repl.rb:38:in `start'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/repl.rb:13:in `start'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/pry_class.rb:192:in `start'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-byebug-3.6.0/lib/pry-byebug/pry_ext.rb:11:in `start_with_pry_byebug'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/lib/pry/cli.rb:116:in `start'
from ~/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/pry-0.11.3/bin/pry:12:in `<top (required)>'
from ~/.rbenv/versions/2.4.4/bin/pry:23:in `load'
from ~/.rbenv/versions/2.4.4/bin/pry:23:in `<main>'

and now, pry apparently refuses to load my history:

$ pry
History file not loaded: string contains null byte
[1] pry(main)>     

On master as of today (ef80932bc66ca8a03f970d24280a77af30293489) pry crashes rather than reporting that it can't load history:

$ bundle exec bin/pry
bundler: failed to load command: bin/pry (bin/pry)
ArgumentError: string contains null byte
  ~/code/pry/lib/pry/history.rb:117:in `<<'
  ~/code/pry/lib/pry/history.rb:117:in `push_to_readline'
  ~/code/pry/lib/pry/history.rb:37:in `call'
  ~/code/pry/lib/pry/history.rb:37:in `block in load'
  ~/code/pry/lib/pry/history.rb:108:in `block in read_from_file'
  ~/code/pry/lib/pry/history.rb:108:in `foreach'
  ~/code/pry/lib/pry/history.rb:108:in `read_from_file'
  ~/code/pry/lib/pry/history.rb:36:in `call'
  ~/code/pry/lib/pry/history.rb:36:in `load'
  ~/code/pry/lib/pry/pry_class.rb:243:in `load_history'
  ~/code/pry/lib/pry/pry_class.rb:147:in `final_session_setup'
  ~/code/pry/lib/pry/cli.rb:84:in `parse_options'
  bin/pry:11:in `<top (required)>'
owst added a commit to bambooengineering/pry that referenced this issue Oct 9, 2018
Fixes pry#1789.

Readline is unable to add lines to its history that contain a null byte;
we should therefore avoid saving such lines to the history file, and
ignore any such lines that are already present in the file.
@kyrylo
Copy link
Member

@kyrylo kyrylo commented Oct 10, 2018

Can you please help me reproduce this issue? I am trying:

% echo $'foo\0bar' >> ~/.pry_history
% pry
[1] pry(main)>

I am on OSX, so I Ctrl-Space doesn't work for me.

@owst
Copy link
Contributor Author

@owst owst commented Oct 10, 2018

Interesting - on my MacOS laptop, with iTerm2 (3.2.3) I can reproduce using your example:

~ pry
[1] pry(main)>
~ echo $'foo\0bar' >> ~/.pry_history
~ pry
History file not loaded: string contains null byte
[1] pry(main)>
~ tail -n-1 ~/.pry_history | xxd -
00000000: 666f 6f00 6261 720a                      foo.bar.
~ pry --version
Pry version 0.11.3 on Ruby 2.4.4

Just to check: are you running on the branch of my PR?

@kyrylo
Copy link
Member

@kyrylo kyrylo commented Oct 10, 2018

I'm running pry 0.11.3 on iTerm2 3.1.6, will try to upgrade iTerm.

~% echo $'foo\0bar' >> ~/.pry_history
~% pry
zsh: command not found: pry
~% gem i pry
Fetching: pry-0.11.3.gem (100%)
Successfully installed pry-0.11.3
1 gem installed
~% pry
zsh: correct 'pry' to 'pr' [nyae]? n
[1] pry(main)>
~% cat ~/.pry_history | xxd -
00000000: 666f 6f00 6261 720a                      foo.bar.
~% pry --version
Pry version 0.11.3 on Ruby 2.4.2
@kyrylo
Copy link
Member

@kyrylo kyrylo commented Oct 10, 2018

Upgraded iTerm2 to 3.2.3. Still nothing.

What's your Readline version? I'm on 7.0:

[1] pry(main)> Readline::VERSION
=> "7.0"
@owst
Copy link
Contributor Author

@owst owst commented Oct 10, 2018

[...]
~% cat ~/.pry_history | xxd -
00000000: 666f 6f00 6261 720a                      foo.bar.
[...]

So the NULL byte is there - but you're not seeing the error. Which version of Readline do you have?

I have:

$ pry
History file not loaded: string contains null byte
[1] pry(main)> Readline::VERSION
=> "7.0"
@kyrylo
Copy link
Member

@kyrylo kyrylo commented Oct 10, 2018

I don't know if this is of any importance but I'm on High Sierra. 😁

@owst
Copy link
Contributor Author

@owst owst commented Oct 10, 2018

It's a change in Ruby 2.4.2 => Ruby 2.4.4 (I missed that you were on a different version of Ruby!)): specifically, 2.4.4 includes this change.

After that change, hist_push (i.e. Readline::HISTORY#<<) uses OutputStringValue, which was changed to use StringValueCStr, which is

#define StringValueCStr(v) rb_string_value_cstr(&(v))

Finally, rb_string_value_cstr raises if the underlying C-string contains a NULL byte

@owst
Copy link
Contributor Author

@owst owst commented Oct 10, 2018

Just to confirm, if I use Ruby 2.4.2, I also don't see the warning about a NULL byte - but, as expected due to the way C strings are handled, my history entry is truncated at the first NULL byte:

> ruby --version
ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-darwin17]
> tail -n-1 ~/.pry_history | xxd -
00000000: 666f 6f00 6261 720a                      foo.bar.
> pry # Then hit up to see last history entry...
[1] pry(main)> foo
@kyrylo
Copy link
Member

@kyrylo kyrylo commented Oct 12, 2018

Thanks for digging into this. I did notice the Ruby version mismatch we have but neglected it (thought it was just a TEENY version difference, so no problems here).

@kyrylo kyrylo closed this in #1790 Oct 12, 2018
kyrylo added a commit that referenced this issue Oct 12, 2018
kyrylo added a commit that referenced this issue Oct 15, 2018
kyrylo added a commit that referenced this issue Oct 17, 2018
kyrylo added a commit that referenced this issue Oct 19, 2018
kyrylo added a commit that referenced this issue Oct 19, 2018
kyrylo added a commit that referenced this issue Oct 19, 2018
kyrylo added a commit that referenced this issue Oct 20, 2018
kyrylo added a commit that referenced this issue Oct 21, 2018
kyrylo added a commit that referenced this issue Oct 28, 2018
kyrylo added a commit that referenced this issue Oct 28, 2018
kyrylo added a commit that referenced this issue Nov 1, 2018
kyrylo added a commit that referenced this issue Nov 3, 2018
kyrylo added a commit that referenced this issue Nov 3, 2018
kyrylo added a commit that referenced this issue Nov 4, 2018
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Dec 17, 2018
pkgsr change:
* Remove @Prefix@ from ALTERNATIVES file.

### [v0.12.2][v0.12.2] (November 12, 2018)

#### Bug fixes

* Restore removed deprecations, which were removed by accident due to a bad
  rebase.

### [v0.12.1][v0.12.1] (November 12, 2018)

#### Bug fixes

* Stopped creating a new hash each time `Pry::Prompt#[]` is invoked
  ([#1855](pry/pry#1855))
* Fixed `less` pager not working when it's available
  ([#1861](pry/pry#1861))

### [v0.12.0][v0.12.0] (November 5, 2018)

#### Major changes

* Dropped support for Rubinius ([#1785](pry/pry#1785))

#### Features

* Added a new command, `clear-screen`, that clears the content of the screen Pry
  is running in regardless of platform (Windows or UNIX-like)
  ([#1723](pry/pry#1723))
* Added a new command, `gem-stat`, that prints gem statistics such as gem
  dependencies and downloads ([#1707](pry/pry#1707))
* Added support for nested exceptions for the `wtf` command
  ([#1791](pry/pry#1791))
* Added support for dynamic prompt names
  ([#1833](pry/pry#1833))

  ```rb
  # pryrc
  Pry.config.prompt_name = Pry.lazy { rand(100) }

  # Session
  [1] 80(main)>
  [2] 87(main)>
  [3] 30(main)>
  ```
* Added support for XDG Base Directory Specification
  ([#1609](pry/pry#1609),
  [#1844](pry/pry#1844),
  ([#1848](pry/pry#1848)))
* Removed the `simple-prompt`. Use `change-prompt simple` instead. The
  `list-prompt` command was removed and embedded as `change-prompt --list`
  ([#1849](pry/pry#1849))

#### API changes

* The following methods started accepting the new optional `config` parameter
  ([#1809](pry/pry#1809)):
  * `Pry::Helpers.tablify(things, line_length, config = Pry.config)`
  * `Pry::Helpers.tablify_or_one_line(heading, things, config = Pry.config)`
  * `Pry::Helpers.tablify_to_screen_width(things, options, config = Pry.config)`
  * `Pry::Helpers::Table.new(items, args, config = Pry.config)`

  You are expected to pass a session-local `_pry_.config` instead of the global
  one.

* Added new method `Pry::Config.assign`, for creating a Config non-recursively
  ([#1725](pry/pry#1725))
* Added `Pry.lazy`, which is a helper method for values that need to be
  calculated dynamically. Currently, only `config.prompt_name` supports it
  ([#1833](pry/pry#1833))
* `Pry::Prompt` responds to `.[]`, `.all` & `.add` now. The `Pry::Prompt.add`
  method must be used for implementing custom prompts. See the API in the
  documentation for the class ([#1846](pry/pry#1846))

#### Breaking changes

* Deleted the `Pry::Helpers::Text.bright_default` alias for
  `Pry::Helpers::Text.bold` ([#1795](pry/pry#1795))
* `Pry::Helpers.tablify_to_screen_width(things, options, config = Pry.config)`
  requires `options` or `nil` in place of them.
* `Pry::Helpers::Table.new(items, args, config = Pry.config)` requires `args`
  or `nil` in place of them.
* Completely revamped `Pry::HistoryArray`
  ([#1818](pry/pry#1818)).
  * It's been renamed to `Pry::Ring`
    ([#1817](pry/pry#1817))
  * The implementation has changed and as result, the following methods were
    removed:
    * `Pry::Ring#length` (use `Pry::Ring#count` instead)
    * `#empty?`, `#each`, `#inspect`, `#pop!`, `#to_h`
  * To access old Enumerable methods convert the ring to Array with `#to_a`
  * Fixed indexing for elements (e.g. `_pry_.input_ring[0]` always return some
    element and not `nil`)
* Renamed `Pry.config.prompt_safe_objects` to `Pry.config.prompt_safe_contexts`
* Removed deprecated `Pry::CommandSet#before_command` &
  `Pry::CommandSet#after_command` ([#1838](pry/pry#1838))

#### Deprecations

* Deprecated `_pry_.input_array` & `_pry_.output_array` in favour of
  `_pry_.input_ring` & `_pry_.output_ring` respectively
  ([#1814](pry/pry#1814))
* Deprecated `Pry::Command#text`. Please use `#black`, `#white`, etc. directly
  instead (as you would with helper functions from `BaseHelpers` and
  `CommandHelpers`) ([#1701](pry/pry#1701))
* Deprecated `_pry_.input_array` & `_pry_.output_array` in favour of
  `_pry_.input_ring` and `_pry_.output_ring` respectively
  ([#1817](pry/pry#1817))
* Deprecated `Pry::Platform`. Use `Pry::Helpers::Platform` instead. Note that
  `Pry::Helpers::BaseHelpers` still includes the `Platform` methods but emits a
  warning. You must switch to `Pry::Helpers::Platform` in your code
  ([#1838](pry/pry#1838),
  ([#1845](pry/pry#1845)))
* Deprecated `Pry::Prompt::MAP`. You should use `Pry::Prompt.all` instead to
  access the same map ([#1846](pry/pry#1846))

#### Bug fixes

* Fixed a bug where `cd Hash.new` reported `self` as an instance of Pry::Config
  in the prompt ([#1725](pry/pry#1725))
* Silenced the `Could not find files for the given pattern(s)` error message
  coming from `where` on Windows, when `less` or another pager is not installed
  ([#1767](pry/pry#1767))
* Fixed possible double loading of Pry plugins' `cli.rb` on Ruby (>= 2.4) due to
  [the `realpath` changes while invoking
  `require`](https://bugs.ruby-lang.org/issues/10222)
  ([#1762](pry/pry#1762),
  [#1774](pry/pry#1762))
* Fixed `NoMethodError` on code objects that have a comment but no source when
  invoking `show-source` ([#1779](pry/pry#1779))
* Fixed `negative argument (ArgumentError)` upon pasting code with tabs, which
  used to confuse automatic indentation
  ([#1771](pry/pry#1771))
* Fixed Pry not being able to load history on Ruby 2.4.4+ when it contains the
  null character ([#1789](pry/pry#1789))
* Fixed Pry raising errors on `cd`'ing into some objects that redefine
  `method_missing` and `respond_to?`
  ([#1811](pry/pry#1811))
* Fixed bug when indentation leaves parts of input after pressing enter when
  Readline is enabled with mode indicators for vi mode
  ([#1813](pry/pry#1813),
  [#1820](pry/pry#1820),
  [#1825](pry/pry#1825))
* Fixed `edit` not writing to history
  ([#1749](pry/pry#1749))

#### Other changes

* Deprecated the `Data` constant to match Ruby 2.5 in the `ls` command
  ([#1731](pry/pry#1731))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants