Initial support for installing default gems. #566

Merged
merged 4 commits into from Jun 26, 2013

Conversation

Projects
None yet
2 participants
Contributor

headius commented Jun 4, 2013

The logic here is pulled from MRI's rbinstall.rb default gem logic
and the actual logic for installing gems. The combined logic works
as follows:

  • The normal dependency chain is followed, unlike the rbinstall
    logic.
  • The gem's full specification (with complete file list) is dumped
    to GEM_HOME/specifications/default/GEM_NAME.gemspec. This covers
    the bulk of the rbinstall logic.
  • Only the bin files from the gem are extracted to the normal gem
    install location. This allows a stub binscript to boot from a
    combination of gem's bin and stdlib's libraries, as in the
    rbinstall logic.

Remaining work to do:

  • Unpack gem lib and ext contents into appropriate location for
    the stdlib. This will overwrite existing files, and is generally
    intended to do a dev-time update of a given default gem's
    contents in stdlib.
  • Install a binstub that works with both stdlib (default gem)
    libraries and installed gems, as in Ruby 2.0's rake, rdoc, ri
    commands. Primarily, this binstub needs to fallback to just a
    simple require when gem loading fails.
Initial support for installing default gems.
The logic here is pulled from MRI's rbinstall.rb default gem logic
and the actual logic for installing gems. The combined logic works
as follows:

* The normal dependency chain is followed, unlike the rbinstall
  logic.
* The gem's full specification (with complete file list) is dumped
  to GEM_HOME/specifications/default/GEM_NAME.gemspec. This covers
  the bulk of the rbinstall logic.
* Only the bin files from the gem are extracted to the normal gem
  install location. This allows a stub binscript to boot from a
  combination of gem's bin and stdlib's libraries, as in the
  rbinstall logic.

Remaining work to do:

* Unpack gem lib and ext contents into appropriate location for
  the stdlib. This will overwrite existing files, and is generally
  intended to do a dev-time update of a given default gem's
  contents in stdlib.
* Install a binstub that works with both stdlib (default gem)
  libraries and installed gems, as in Ruby 2.0's rake, rdoc, ri
  commands. Primarily, this binstub needs to fallback to just a
  simple require when gem loading fails.
Contributor

headius commented Jun 4, 2013

Worth pointing out:

rbinstall.rb does not unpack gem contents into stdlib nor does it create the default-aware binstubs. I mentioned that as work to-do in my original PR mostly because it seems logical to have a way to pull down a gem and replace local ext/lib contents with updated gem contents.

It would be bonus work to implement this.

lib/rubygems/commands/install_command.rb
+ add_option(:"Install/Update", '--default',
+ 'Add the gem\'s full specification to',
+ 'specifications/default and extract only its bin') do |v,o|
+ o[:default] = v
@drbrain

drbrain Jun 4, 2013

Owner

:default seems to ambiguous. How about :install_as_default?

@headius

headius Jun 4, 2013

Contributor

Ok, will fix.

headius added some commits Jun 4, 2013

Additional fixes for default gems:
* Use :install_as_default to be less ambiguous in options
* Add logic to ensure specification/default dir is created
Contributor

headius commented Jun 5, 2013

I realize now there's a "flaw" in this PR that actually should probably be fixed in RubyGems.

Default gem logic works by reading in specs from specifications/default and using the file list as triggers to attempt to boot those gems. However, the list of files it expects needs to be the require names rather than the file list normally provided by gems.

My PR uses the actual specification files from the gem, so it includes files not normally requirable (README, bin/, etc), and includes prefixes on those that are requirable (lib/, ext/*).

I believe RG default gem support needs to be modified to be able to use a proper, ummodified gemspec (as in my PR) and to build its list of require file hooks based on spec.files and spec/require_paths.

The logic would be:

  • Read in list of files and require paths
  • Filter to only include the files prefixed with those require paths
  • Remove those prefixes to use for default gem hooks.

Comments welcome...I may try to implement this, but would welcome help.

I'm also not sure it can be done as a non-breaking change since MRI 2.0 has shipped with the old logic and the old modified gemspec files with stripped paths.

Modify default gemspec registration to support unmodified specs.
MRI 2.0 shipped with its own default spec generator that rewrites
the gemspec file to contain a list of bare require names in
spec.files. This required extra logic in rbinstall.rb and makes
it impossible to simply copy a gemspec from a gem into the default
directory to register that gem as a default gem.

PR #566 adds --default functionality to `gem install` that copies
the gem's unmodified gemspec to the default directory, but since
unmodified gemspec contains a list of gem-relative filenames, it
could not use the old default specification logic.

This commit modifies default spec registration to support both
formats. In the "old" style, specifications are registered with
their raw filenames; we detect old style by checking the first
file to see if it is prefixed with any spec.require_paths. In the
"new" style, spec.require_paths are stripped off the spec's
filenames before registration, and only files that start with a
require_path are registered.

Both the old format and the new format continue to work with this
change, and we can migrate toward default specs being the
unmodified originals rather than custom-built as in MRI 2.0.
Contributor

headius commented Jun 5, 2013

I have added a commit that modifies Gem.register_default_spec to support both the old and new gemspec formats.

You can see the commit entry for more details, but in short... old style (MRI 2.0 custom-build specs) register as they do now, using all entries from spec.files; new style (unmodified gemspec) works by stripping require paths from files and only registering requirable names.

headius added a commit to jruby/jruby that referenced this pull request Jun 5, 2013

Owner

drbrain commented Jun 5, 2013

Regarding the flaw, master won't be merged to Ruby trunk as it has new features that should appear in Ruby 2.1.

Contributor

headius commented Jun 5, 2013

I'm hoping we can include this functionality in JRuby 1.7.5, so either the PR can be modified to work on RG 2.0 or hopefully 2.x will be released before then. Backporting my changes won't be difficult.

Contributor

headius commented Jun 24, 2013

I'm confused...I do not see a 2.0 branch in git. Is there a way I can rework this patch to get into a RubyGems 2.0 release, or will we have to wait until 2.1 (and if the latter, how long will that be?).

Owner

drbrain commented Jun 24, 2013

If we need to do another 2.0.x release I'll start from 2.0.3 for the branch. What is your JRuby 1.7.5 timeframe?

I hope to have a 2.1 release in the next three to four week timeframe. Currently there are 22 open issues on the 2.1 milestone and I've been working on those (I suspect a few more fixes/features will show up in there). I've been working the issues from oldest to newest and this is currently third-newest.

If JRuby 1.7.5 is imminent I can apply and backport this more quickly.

Contributor

headius commented Jun 24, 2013

I'd need to confer with @enebo but I don't think 1.7.5 is imminent. We're probably looking at a release in late July at the earliest, and before JRubyConfEU (mid-August) at the latest. I'd really like to get MRI 2.0-style default gem support into that release...so let me know if there's anything I can do to help make that happen.

drbrain added a commit that referenced this pull request Jun 26, 2013

Merge pull request #566 from headius/default_gems
Initial support for installing default gems.

@drbrain drbrain merged commit 4a97046 into rubygems:master Jun 26, 2013

1 check failed

default The Travis CI build failed
Details

drbrain added a commit that referenced this pull request Jun 26, 2013

Contributor

headius commented Jun 26, 2013

BTW, the flaw workaround logic will probably have to live for a long time (forever) since all installed MRI 2.0 have the old default specification format created by rbinstall.rb.

Owner

drbrain commented Jun 26, 2013

Yep, but that's fine, it'll be just like the 1.8.7 and 1.9.1 baggage we're still hauling around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment