Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gem server seems to not REALLY be doing what it says the defaults are #1303

Closed
thoraxe opened this issue Jul 3, 2015 · 10 comments
Closed

gem server seems to not REALLY be doing what it says the defaults are #1303

thoraxe opened this issue Jul 3, 2015 · 10 comments

Comments

@thoraxe
Copy link

@thoraxe thoraxe commented Jul 3, 2015

[thoraxe@t440 ~]$ gem server --help
...
  Defaults:
    --port 8808 --dir /home/thoraxe/.rvm/gems/ruby-2.1.2 --no-daemon

If I run "gem server" with no options, I end up getting errors when trying to pull gems from this server:

[thoraxe@t440 ~]$ gem server --debug
NOTE:  Debugging mode prints all exceptions even when rescued
Using Ext extension for JSON.
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/site_ruby/2.1.0/rubygems/server.rb:604 - `/api/v1/dependencies' not found.
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /api/v1/dependencies HTTP/1.1" 404 293
- -> /api/v1/dependencies
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /specs.4.8.gz HTTP/1.1" 200 2641
- -> /specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /prerelease_specs.4.8.gz HTTP/1.1" 200 83
- -> /prerelease_specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/rack-1.6.4.gemspec.rz HTTP/1.1" 200 562
- -> /quick/Marshal.4.8/rack-1.6.4.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz HTTP/1.1" 200 1022
- -> /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz HTTP/1.1" 200 512
- -> /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:30 EDT] "GET /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz HTTP/1.1" 200 276
- -> /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpservlet/filehandler.rb:331 - `/gems/rack-1.6.4.gem' not found.
172.17.0.3 - - [03/Jul/2015:18:57:30 EDT] "GET /gems/rack-1.6.4.gem HTTP/1.1" 404 293
- -> /gems/rack-1.6.4.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError

There is definitely something there:

[thoraxe@t440 ~]$ ll /home/thoraxe/.rvm/gems/ruby-2.1.2/gems | grep rack-1.6.4 | wc -l
1

If I explicitly specify a directory, it works:

[thoraxe@t440 ~]$ gem server --port 8808 --dir /home/thoraxe/.rvm/gems/ruby-2.1.2 --no-daemon --debug                                                                                                              
NOTE:  Debugging mode prints all exceptions even when rescued
Using Ext extension for JSON.
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/site_ruby/2.1.0/rubygems/server.rb:604 - `/api/v1/dependencies' not found.
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /api/v1/dependencies HTTP/1.1" 404 293
- -> /api/v1/dependencies
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /specs.4.8.gz HTTP/1.1" 200 2567
- -> /specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /prerelease_specs.4.8.gz HTTP/1.1" 200 83
- -> /prerelease_specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/rack-1.6.4.gemspec.rz HTTP/1.1" 200 562
- -> /quick/Marshal.4.8/rack-1.6.4.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz HTTP/1.1" 200 1022
- -> /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz HTTP/1.1" 200 512
- -> /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz HTTP/1.1" 200 276
- -> /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/rack-1.6.4.gem HTTP/1.1" 200 228864
- -> /gems/rack-1.6.4.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/rack-protection-1.5.3.gem HTTP/1.1" 200 18432
- -> /gems/rack-protection-1.5.3.gem
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/tilt-2.0.1.gem HTTP/1.1" 200 44544
- -> /gems/tilt-2.0.1.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/sinatra-1.4.6.gem HTTP/1.1" 200 355840
- -> /gems/sinatra-1.4.6.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
2.4.8
@copiousfreetime
Copy link
Contributor

@copiousfreetime copiousfreetime commented Apr 30, 2016

I've confirmed that this does still exist.

% ruby -v && gem -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin15]
2.6.4

Starting up gem server

% gem server --debug
NOTE:  Debugging mode prints all exceptions even when rescued
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1726 - invalid option: no-rdoc
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1536 - invalid option: --no-rdoc
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1726 - invalid option: no-ri
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1536 - invalid option: --no-ri
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Using Ext extension for JSON.

Look for a gem

% gem list -q --remote --clear-sources -s http://localhost:8808/ launchy
launchy (2.4.3)

# Server output
::1 - - [30/Apr/2016:15:11:27 MDT] "GET /latest_specs.4.8.gz HTTP/1.1" 200 357
- -> /latest_specs.4.8.gz
Exception `WEBrick::HTTPStatus::EOFError' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/webrick/httpserver.rb:82 - WEBrick::HTTPStatus::EOFError

And attempt to download it

% gem fetch -q  --clear-sources -s http://localhost:8808/ launchy
ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
    bad response Not Found  404 (http://localhost:8808/gems/launchy-2.4.3.gem)

# Server output
::1 - - [30/Apr/2016:15:12:56 MDT] "GET /latest_specs.4.8.gz HTTP/1.1" 200 357
- -> /latest_specs.4.8.gz
Exception `WEBrick::HTTPStatus::NotFound' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/webrick/httpservlet/filehandler.rb:332 - `/gems/launchy-2.4.3.gem' not found.

Specifically using the -d option on server does result in the expected behavior.

The cause of this bug is that the default the value of options[:gemdir] is Gem.path which is array of paths. In my experiments this is:

% ruby -e 'puts Gem.path'
/Users/jeremy/Clients/ruby-together/.gem/ruby/2.3.1
/Users/jeremy/.gem/ruby/2.3.1
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0

When gem server instantiates an instance of Gem::Server this array of paths is passed in as the first parameter -- https://github.com/rubygems/rubygems/blob/master/lib/rubygems/commands/server_command.rb#L81-L84

During the setup and mounting of paths, Gem::Server loops over that array and adds items to the webrick mount table -- https://github.com/rubygems/rubygems/blob/master/lib/rubygems/commands/server_command.rb#L81-L84

Unfortunately, it mounts them all at the same point, and since MountTable uses a hash internally, the last one wins. Which in this case is /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0 and that is not where the gem files that we have installed in /Users/jeremy/Clients/ruby-together/.gem/ruby/2.3.1 live.

I see 2 potential solutions for this:

  1. Change the default options[:gemdir] to be Gem.dir instead of Gem.path which would avoid the whole issue since we would have a default of a single directory.
  2. Change how the /gems route is mounted and write a proc handler to do the lookup instead of a WEBrick::HTTPServlet::FileHandler.

Thoughts anyone? I don't think (2) would be all that difficult. The approach would be:

  1. Create a new method Gem::Server#gems that would search for the proper gem file
  2. Mount the method using @server.mount_proc "/gems", method(:gems)

@andrewhood125
Copy link

@andrewhood125 andrewhood125 commented Jul 15, 2020

Just hit this myself settings up a private gem server. Thank goodness for this issue. 🙏 Adding the directory option fixed the issue.

@deivid-rodriguez
Copy link
Member

@deivid-rodriguez deivid-rodriguez commented Oct 1, 2020

Hi! We're actually going to deprecate the gem server command because we believe it hardly has any realworld usage. Do you still use this command?

@thoraxe
Copy link
Author

@thoraxe thoraxe commented Oct 1, 2020

What would the replacement solution be for running a gem server?

@hsbt
Copy link
Member

@hsbt hsbt commented Oct 1, 2020

I have α plan to extract it to rubygems-server plugin. You can use it after removing gem server command from rubygems.

@deivid-rodriguez
Copy link
Member

@deivid-rodriguez deivid-rodriguez commented Oct 1, 2020

What would the replacement solution be for running a gem server?

Something like gemstash I believe. @bronzdoc @indirect Is gemstash the alternative to gem server?

@bronzdoc
Copy link
Member

@bronzdoc bronzdoc commented Oct 5, 2020

@deivid-rodriguez Yes

We added documentation in rubygems.org but the PR hasn't landed yet

@deivid-rodriguez
Copy link
Member

@deivid-rodriguez deivid-rodriguez commented Oct 5, 2020

Oh, nice! I didn't know those docs existed. Should we start by removing documentation for gem server from guides.rubygems.org and then maybe deprecating the command at a later stage? Just thinking that going from "gem server is that easiest way of running your own gem server" to "gem server is deprecated" might be a bit abrupt?

@thoraxe
Copy link
Author

@thoraxe thoraxe commented Oct 5, 2020

If there's a valid and supported alternative, definitely it's fine with me to nuke this / CLOSEDWONTFIX.

@deivid-rodriguez
Copy link
Member

@deivid-rodriguez deivid-rodriguez commented Oct 6, 2020

Ok, let's close this then 👍.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants
@copiousfreetime @hsbt @thoraxe @andrewhood125 @deivid-rodriguez @bronzdoc and others