Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RVM changes shebangs which break rake/rdoc/ri on TruffleRuby #4408

Closed
eregon opened this issue Jun 18, 2018 · 9 comments · Fixed by #4409
Closed

RVM changes shebangs which break rake/rdoc/ri on TruffleRuby #4408

eregon opened this issue Jun 18, 2018 · 9 comments · Fixed by #4409

Comments

@eregon
Copy link
Contributor

eregon commented Jun 18, 2018

Description

RVM changes shebangs which break rake/rdoc/ri on TruffleRuby, because it changes the shebang line from

#!/usr/bin/env bash

to

#!/usr/bin/env truffleruby_executable_hooks

The original being https://github.com/oracle/truffleruby/blob/vm-1.0.0-rc2/bin/rake.
TruffleRuby does this so even if somebody ran truffleruby/bin/rake without adapting PATH or using a Ruby manager it would still pick truffleruby automatically.

The same happens for ri and rdoc (they are also broken), but not for gem, irb, testrb (they seem to work fine).

How I can disable these changes to shebangs of RVM for TruffleRuby?
I would like something reliable with no extra wrappers or changes to installed files, as I have no idea what they might break.

cc @mpapis @havenwood

Steps to reproduce

rvm install truffleruby
rvm use truffleruby
rake --version

Expected behavior

rake, version 12.0.0

Actual behavior

rake --version
/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin/truffleruby_executable_hooks:15:in `<main>': /home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin/rake:5: `$(' is not allowed as a global variable name (SyntaxError)
zsh: exit 1     rake --version

Environment info

rvm info rvm info

truffleruby-1.0.0-rc2:

system:
uname: "Linux localhost.localdomain 4.13.12-200.fc26.x86_64 #1 SMP Wed Nov 8 16:47:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux"
name: "Fedora"
version: "26"
architecture: "x86_64"
bash: "/usr/bin/bash => GNU bash, version 4.4.12(1)-release (x86_64-redhat-linux-gnu)"
zsh: "/usr/bin/zsh => zsh 5.3.1 (x86_64-redhat-linux-gnu)"
remote_path: "fedora/26/x86_64"

rvm:
version: "1.29.3 (master)"
updated: "12 hours 49 minutes 47 seconds ago"
path: "/home/eregon/.rvm"
autolibs: "[4] Allow RVM to use package manager if found, install missing dependencies, install package manager (only OS X)."

ruby:
interpreter: "truffleruby"
version: "1.0.0-rc2,"
date: "truffleruby 1.0.0-rc2, like ruby 2.4.4, GraalVM CE Native [x86_64-linux]"
platform: "x86_64-linux"
patchlevel: "truffleruby 1.0.0-rc2, like ruby 2.4.4, GraalVM CE Native [x86_64-linux]"
full_version: "truffleruby 1.0.0-rc2, like ruby 2.4.4, GraalVM CE Native [x86_64-linux]"

homes:
gem: "/home/eregon/.rvm/gems/truffleruby-1.0.0-rc2"
ruby: "/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2"

binaries:
ruby: "/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin/ruby"
irb: "/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin/irb"
gem: "/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin/gem"
rake: "/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin/rake"

environment:
PATH: "/home/eregon/.rvm/gems/truffleruby-1.0.0-rc2/bin:/home/eregon/.rvm/gems/truffleruby-1.0.0-rc2@global/bin:/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin:/home/eregon/.rvm/bin:/home/eregon/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/eregon/code/mx:/home/eregon/code/mspec/bin:/opt/crystal/bin"
GEM_HOME: "/home/eregon/.rvm/gems/truffleruby-1.0.0-rc2"
GEM_PATH: "/home/eregon/.rvm/gems/truffleruby-1.0.0-rc2:/home/eregon/.rvm/gems/truffleruby-1.0.0-rc2@global"
MY_RUBY_HOME: "/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2"
IRBRC: "/home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/.irbrc"
RUBYOPT: ""
gemset: ""

@eregon
Copy link
Contributor Author

eregon commented Jun 18, 2018

There is actually a fair bit of thought behind this design for our launchers, as documented at https://github.com/oracle/truffleruby/blob/master/doc/contributor/launchers.md, so I don't think we'd want to change the design of our rake/rdoc/irb/etc launchers.

eregon added a commit to eregon/rvm that referenced this issue Jun 19, 2018
* Fixes rvm#4408
* Avoid changing the behavior of standard Ruby executables.
* No need to install rake, it is part of the standard library.
@eregon
Copy link
Contributor Author

eregon commented Jun 19, 2018

PR at #4409

@eregon eregon changed the title RVM generates wrappers which break rake/rdoc/ri on TruffleRuby RVM changes shebangs which break rake/rdoc/ri on TruffleRuby Jun 19, 2018
havenwood pushed a commit that referenced this issue Jun 21, 2018
* Fixes #4408
* Avoid changing the behavior of standard Ruby executables.
* No need to install rake, it is part of the standard library.
@eregon
Copy link
Contributor Author

eregon commented Jun 21, 2018

~/.rvm/rubies/truffleruby-1.0.0-rc2/bin/truffleruby_executable_hooks contains this:

#!/usr/bin/env ruby

title = "ruby #{ARGV*" "}"
$0    = ARGV.shift
Process.setproctitle(title) if Process.methods.include?(:setproctitle)

require 'rubygems'
begin
  require 'executable-hooks/hooks'
  Gem::ExecutableHooks.run($0)
rescue LoadError
  warn "unable to load executable-hooks/hooks" if ENV.key?('ExecutableHooks_DEBUG')
end unless $0.end_with?('/executable-hooks-uninstaller')

eval File.read($0), binding, $0

The last line, eval File.read("bin/rake") doesn't work for TruffleRuby, because TruffleRuby's bin/rake is both an executable bash script and a Ruby executable (see below).

Ruby itself skips until it finds a shebang containing ruby when doing ruby bin/rake.
This is basically the -x flag of MRI, except it also happens automatically when the first shebang doesn't contain ruby.

So to make rubygems-bundler work with TruffleRuby, we'd probably need to replicate that logic in the generated truffleruby_executable_hooks or in some method of the gem, which is not so nice.
Both load and require also don't have the behavior to skip until a ruby shebang, it seems only the main script has this logic.

Another unfortunate side effect of rubygems-bundler changing the shebang of bin/rake is now it looks like a ruby shebang (truffleruby_executable_hooks contains ruby), and so even truffleruby .../bin/rake doesn't work anymore.

Here is the full unmodified ~/.rvm/rubies/truffleruby-1.0.0-rc2/bin/rake:

#!/usr/bin/env bash
# ignored by Ruby interpreter

# get the absolute path of the executable and resolve symlinks
SELF_PATH=$(cd "$(dirname "$0")" && pwd -P)/$(basename "$0")
while [ -h "$SELF_PATH" ]; do
  # 1) cd to directory of the symlink
  # 2) cd to the directory of where the symlink points
  # 3) get the pwd
  # 4) append the basename
  DIR=$(dirname "$SELF_PATH")
  SYM=$(readlink "$SELF_PATH")
  SELF_PATH=$(cd "$DIR" && cd "$(dirname "$SYM")" && pwd)/$(basename "$SYM")
done
exec "$(dirname $SELF_PATH)/ruby" "$SELF_PATH" "$@"

#!ruby
# ^ marks start of Ruby interpretation

#
# This file was generated by RubyGems.
#
# The application 'rake' is installed as part of a gem, and
# this file is here to facilitate running it.
#

require 'rubygems'

version = ">= 0.a"

if ARGV.first
  str = ARGV.first
  str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
  if str =~ /\A_(.*)_\z/ and Gem::Version.correct?($1) then
    version = $1
    ARGV.shift
  end
end

if Gem.respond_to?(:activate_bin_path)
load Gem.activate_bin_path('rake', 'rake', version)
else
gem "rake", version
load Gem.bin_path("rake", "rake", version)
end

@eregon
Copy link
Contributor Author

eregon commented Jun 21, 2018

One interesting workaround is, if rake is gem installed, and the generated bin/rake overwrites the original bin/rake then it works fine.

$ rvm use truffleruby
$ gem install rubygems-bundler
$ rake --version # => broken as above
bin/rake:5: `$(' is not allowed as a global variable name (SyntaxError)

$ gem install rake
Fetching: rake-12.3.1.gem (100%)
rake's executable "rake" conflicts with /home/eregon/.rvm/rubies/truffleruby-1.0.0-rc2/bin/rake
Overwrite the executable? [yN]  y
Successfully installed rake-12.3.1
1 gem installed

$ rake --version
rake, version 12.3.1

This doesn't fix ri/rdoc though, until a new rdoc is installed.

@eregon
Copy link
Contributor Author

eregon commented Jun 21, 2018

I suppose one way to fix this although very hacky would be to remove the Bash preludes of known launchers (rake/rdoc/ri) when installing TruffleRuby in RVM.
Since that installation of TruffleRuby is inside RVM, the right ruby binary should be used anyway.

Another way would be to write the Bash code in such a way that it can be executed as Ruby code but has no side effects. This seems challenging, but might be possible.

@eregon
Copy link
Contributor Author

eregon commented Jun 22, 2018

Another way would be to write the Bash code in such a way that it can be executed as Ruby code but has no side effects. This seems challenging, but might be possible.

@mame found a way to do this, a script that can be executed both as Bash or Ruby code, and allows arbitrary Ruby code and complex Bash code:

#!/bin/bash

echo "#{<<exit#" > /dev/null

echo "in bash"
exec ruby -x $0 $@

exit
}" if false

#!ruby
puts "in ruby"
p ARGV
#!/bin/bash

echo "#{<<exit#" > /dev/null

echo "in bash"
exec ruby -x $0 $@

exit
}" if false

#!ruby
puts "in ruby"
p ARGV
$ ruby -e 'eval File.read "tt"' 1 2 3
in ruby
["1", "2", "3"]

$ bash tt 1 2 3
in bash
in ruby
["1", "2", "3"]

If we write TruffleRuby's bin/rake, bin/ri, etc launchers this way then the Bash part should also be executable as Ruby code, which should solve this problem nicely (although in a rather surprising manner).

@mpapis
Copy link
Member

mpapis commented Jun 24, 2018

@eregon I just did this https://github.com/rvm/executable-hooks/blob/ae1b00c51a31f5a994bb60421ef977f1d56d8bb0/bin/ruby_executable_hooks - could you check if that works for you too?

@mpapis mpapis reopened this Jun 24, 2018
@eregon
Copy link
Contributor Author

eregon commented Jun 24, 2018

rvm/executable-hooks#32 seems to work well.
That's actually easier than I thought, that's a nice and easy solution 😃

I tested by removing gemsets/truffleruby/global.gems, reinstalling truffleruby, then rake --version fails as expected,
then changing ~/.rvm/gems/truffleruby-1.0.0-rc2@global/gems/executable-hooks-1.4.2/bin/ruby_executable_hooks to the version in the PR,
and then gem regenerate_binstubs + gem install rubygems-bundler
and rake --version works again.

I only have 1 TruffleRuby spec failing, which is:

1)
The launcher 'ri' in `RbConfig::CONFIG['bindir']` directory runs when symlinked FAILED
Expected "linktori 5.0.0\n"
 to match /^ri 5\.0\.0$/

~/code/truffleruby-tests/spec/truffle/launcher_spec.rb:43:in `block (6 levels) in <top (required)>'
~/code/truffleruby-tests/spec/truffle/launcher_spec.rb:39:in `chdir'
~/code/truffleruby-tests/spec/truffle/launcher_spec.rb:39:in `block (5 levels) in <top (required)>'
~/.rvm/rubies/truffleruby-1.0.0-rc2/lib/mri/tmpdir.rb:89:in `mktmpdir'
~/code/truffleruby-tests/spec/truffle/launcher_spec.rb:38:in `block (4 levels) in <top (required)>'
~/code/truffleruby-tests/spec/truffle/launcher_spec.rb:12:in `<top (required)>'

So if one does:

$ ln -s ~/.rvm/rubies/truffleruby-1.0.0-rc2/bin/ri linktori
$ ./linktori --version
linktori 5.0.0

ri uses File.basename $0 and [RUBY_ENGINE]_executable_hooks does $0 = ARGV.shift.
If instead it did $0 = File.realpath(ARGV.shift) or $0 = File.basename(File.realpath(ARGV.shift)) then this would print ri 5.0.0 as expected.
This is a small matter, but seems worth fixing separately.

In any case, let's merge rvm/executable-hooks#32 and I think we can revert 45f40f1 once there is a release of executable-hooks.

@eregon
Copy link
Contributor Author

eregon commented Jun 25, 2018

With executable-hooks 1.5.0 released and 0233c51, this issue can be closed, thank you @mpapis!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants