`gem server` seems to not REALLY be doing what it says the defaults are #1303

thoraxe · 2015-07-03T22:59:28Z

[thoraxe@t440 ~]$ gem server --help
...
  Defaults:
    --port 8808 --dir /home/thoraxe/.rvm/gems/ruby-2.1.2 --no-daemon

If I run "gem server" with no options, I end up getting errors when trying to pull gems from this server:

[thoraxe@t440 ~]$ gem server --debug
NOTE:  Debugging mode prints all exceptions even when rescued
Using Ext extension for JSON.
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/site_ruby/2.1.0/rubygems/server.rb:604 - `/api/v1/dependencies' not found.
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /api/v1/dependencies HTTP/1.1" 404 293
- -> /api/v1/dependencies
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /specs.4.8.gz HTTP/1.1" 200 2641
- -> /specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /prerelease_specs.4.8.gz HTTP/1.1" 200 83
- -> /prerelease_specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/rack-1.6.4.gemspec.rz HTTP/1.1" 200 562
- -> /quick/Marshal.4.8/rack-1.6.4.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz HTTP/1.1" 200 1022
- -> /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz HTTP/1.1" 200 512
- -> /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:30 EDT] "GET /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz HTTP/1.1" 200 276
- -> /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpservlet/filehandler.rb:331 - `/gems/rack-1.6.4.gem' not found.
172.17.0.3 - - [03/Jul/2015:18:57:30 EDT] "GET /gems/rack-1.6.4.gem HTTP/1.1" 404 293
- -> /gems/rack-1.6.4.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError

There is definitely something there:

[thoraxe@t440 ~]$ ll /home/thoraxe/.rvm/gems/ruby-2.1.2/gems | grep rack-1.6.4 | wc -l
1

If I explicitly specify a directory, it works:

[thoraxe@t440 ~]$ gem server --port 8808 --dir /home/thoraxe/.rvm/gems/ruby-2.1.2 --no-daemon --debug                                                                                                              
NOTE:  Debugging mode prints all exceptions even when rescued
Using Ext extension for JSON.
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/site_ruby/2.1.0/rubygems/server.rb:604 - `/api/v1/dependencies' not found.
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /api/v1/dependencies HTTP/1.1" 404 293
- -> /api/v1/dependencies
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /specs.4.8.gz HTTP/1.1" 200 2567
- -> /specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /prerelease_specs.4.8.gz HTTP/1.1" 200 83
- -> /prerelease_specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/rack-1.6.4.gemspec.rz HTTP/1.1" 200 562
- -> /quick/Marshal.4.8/rack-1.6.4.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz HTTP/1.1" 200 1022
- -> /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz HTTP/1.1" 200 512
- -> /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz HTTP/1.1" 200 276
- -> /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/rack-1.6.4.gem HTTP/1.1" 200 228864
- -> /gems/rack-1.6.4.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/rack-protection-1.5.3.gem HTTP/1.1" 200 18432
- -> /gems/rack-protection-1.5.3.gem
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/tilt-2.0.1.gem HTTP/1.1" 200 44544
- -> /gems/tilt-2.0.1.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/sinatra-1.4.6.gem HTTP/1.1" 200 355840
- -> /gems/sinatra-1.4.6.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError

ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
2.4.8

The text was updated successfully, but these errors were encountered:

copiousfreetime · 2016-04-30T21:48:20Z

I've confirmed that this does still exist.

% ruby -v && gem -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin15]
2.6.4

Starting up gem server

% gem server --debug
NOTE:  Debugging mode prints all exceptions even when rescued
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1726 - invalid option: no-rdoc
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1536 - invalid option: --no-rdoc
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1726 - invalid option: no-ri
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1536 - invalid option: --no-ri
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Using Ext extension for JSON.

Look for a gem

% gem list -q --remote --clear-sources -s http://localhost:8808/ launchy
launchy (2.4.3)

# Server output
::1 - - [30/Apr/2016:15:11:27 MDT] "GET /latest_specs.4.8.gz HTTP/1.1" 200 357
- -> /latest_specs.4.8.gz
Exception `WEBrick::HTTPStatus::EOFError' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/webrick/httpserver.rb:82 - WEBrick::HTTPStatus::EOFError

And attempt to download it

% gem fetch -q  --clear-sources -s http://localhost:8808/ launchy
ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
    bad response Not Found  404 (http://localhost:8808/gems/launchy-2.4.3.gem)

# Server output
::1 - - [30/Apr/2016:15:12:56 MDT] "GET /latest_specs.4.8.gz HTTP/1.1" 200 357
- -> /latest_specs.4.8.gz
Exception `WEBrick::HTTPStatus::NotFound' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/webrick/httpservlet/filehandler.rb:332 - `/gems/launchy-2.4.3.gem' not found.

Specifically using the -d option on server does result in the expected behavior.

The cause of this bug is that the default the value of options[:gemdir] is Gem.path which is array of paths. In my experiments this is:

% ruby -e 'puts Gem.path'
/Users/jeremy/Clients/ruby-together/.gem/ruby/2.3.1
/Users/jeremy/.gem/ruby/2.3.1
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0

When gem server instantiates an instance of Gem::Server this array of paths is passed in as the first parameter -- https://github.com/rubygems/rubygems/blob/master/lib/rubygems/commands/server_command.rb#L81-L84

During the setup and mounting of paths, Gem::Server loops over that array and adds items to the webrick mount table -- https://github.com/rubygems/rubygems/blob/master/lib/rubygems/commands/server_command.rb#L81-L84

Unfortunately, it mounts them all at the same point, and since MountTable uses a hash internally, the last one wins. Which in this case is /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0 and that is not where the gem files that we have installed in /Users/jeremy/Clients/ruby-together/.gem/ruby/2.3.1 live.

I see 2 potential solutions for this:

Change the default options[:gemdir] to be Gem.dir instead of Gem.path which would avoid the whole issue since we would have a default of a single directory.
Change how the /gems route is mounted and write a proc handler to do the lookup instead of a WEBrick::HTTPServlet::FileHandler.

Thoughts anyone? I don't think (2) would be all that difficult. The approach would be:

Create a new method Gem::Server#gems that would search for the proper gem file
Mount the method using @server.mount_proc "/gems", method(:gems)

andrewhood125 · 2020-07-15T03:27:32Z

Just hit this myself settings up a private gem server. Thank goodness for this issue. 🙏 Adding the directory option fixed the issue.

deivid-rodriguez · 2020-10-01T10:45:34Z

Hi! We're actually going to deprecate the gem server command because we believe it hardly has any realworld usage. Do you still use this command?

thoraxe · 2020-10-01T11:07:28Z

What would the replacement solution be for running a gem server?

hsbt · 2020-10-01T11:34:35Z

I have α plan to extract it to rubygems-server plugin. You can use it after removing gem server command from rubygems.

deivid-rodriguez · 2020-10-01T11:42:15Z

What would the replacement solution be for running a gem server?

Something like gemstash I believe. @bronzdoc @indirect Is gemstash the alternative to gem server?

bronzdoc · 2020-10-05T04:32:08Z

@deivid-rodriguez Yes

We added documentation in rubygems.org but the PR hasn't landed yet

deivid-rodriguez · 2020-10-05T16:11:39Z

Oh, nice! I didn't know those docs existed. Should we start by removing documentation for gem server from guides.rubygems.org and then maybe deprecating the command at a later stage? Just thinking that going from "gem server is that easiest way of running your own gem server" to "gem server is deprecated" might be a bit abrupt?

thoraxe · 2020-10-05T23:27:50Z

If there's a valid and supported alternative, definitely it's fine with me to nuke this / CLOSEDWONTFIX.

deivid-rodriguez · 2020-10-06T10:46:56Z

Ok, let's close this then 👍.

copiousfreetime added the status: triage label Jan 8, 2016

copiousfreetime added category: other type: bug report and removed status: triage labels Apr 30, 2016

copiousfreetime added the status: ready label Apr 30, 2016

bronzdoc self-assigned this Apr 30, 2018

deivid-rodriguez added status: feedback required and removed status: ready labels Oct 1, 2020

deivid-rodriguez closed this as completed Oct 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`gem server` seems to not REALLY be doing what it says the defaults are #1303

`gem server` seems to not REALLY be doing what it says the defaults are #1303

thoraxe commented Jul 3, 2015

copiousfreetime commented Apr 30, 2016

andrewhood125 commented Jul 15, 2020

deivid-rodriguez commented Oct 1, 2020

thoraxe commented Oct 1, 2020

hsbt commented Oct 1, 2020

deivid-rodriguez commented Oct 1, 2020

bronzdoc commented Oct 5, 2020 •

edited

Loading

deivid-rodriguez commented Oct 5, 2020

thoraxe commented Oct 5, 2020

deivid-rodriguez commented Oct 6, 2020

gem server seems to not REALLY be doing what it says the defaults are #1303

gem server seems to not REALLY be doing what it says the defaults are #1303

Comments

thoraxe commented Jul 3, 2015

copiousfreetime commented Apr 30, 2016

andrewhood125 commented Jul 15, 2020

deivid-rodriguez commented Oct 1, 2020

thoraxe commented Oct 1, 2020

hsbt commented Oct 1, 2020

deivid-rodriguez commented Oct 1, 2020

bronzdoc commented Oct 5, 2020 • edited Loading

deivid-rodriguez commented Oct 5, 2020

thoraxe commented Oct 5, 2020

deivid-rodriguez commented Oct 6, 2020

`gem server` seems to not REALLY be doing what it says the defaults are #1303

`gem server` seems to not REALLY be doing what it says the defaults are #1303

bronzdoc commented Oct 5, 2020 •

edited

Loading