`gem server` seems to not REALLY be doing what it says the defaults are

[thoraxe]

[thoraxe@t440 ~]$ gem server --help
...
  Defaults:
    --port 8808 --dir /home/thoraxe/.rvm/gems/ruby-2.1.2 --no-daemon

If I run "gem server" with no options, I end up getting errors when trying to pull gems from this server:

[thoraxe@t440 ~]$ gem server --debug
NOTE:  Debugging mode prints all exceptions even when rescued
Using Ext extension for JSON.
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/site_ruby/2.1.0/rubygems/server.rb:604 - `/api/v1/dependencies' not found.
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /api/v1/dependencies HTTP/1.1" 404 293
- -> /api/v1/dependencies
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /specs.4.8.gz HTTP/1.1" 200 2641
- -> /specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /prerelease_specs.4.8.gz HTTP/1.1" 200 83
- -> /prerelease_specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/rack-1.6.4.gemspec.rz HTTP/1.1" 200 562
- -> /quick/Marshal.4.8/rack-1.6.4.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz HTTP/1.1" 200 1022
- -> /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:29 EDT] "GET /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz HTTP/1.1" 200 512
- -> /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:57:30 EDT] "GET /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz HTTP/1.1" 200 276
- -> /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpservlet/filehandler.rb:331 - `/gems/rack-1.6.4.gem' not found.
172.17.0.3 - - [03/Jul/2015:18:57:30 EDT] "GET /gems/rack-1.6.4.gem HTTP/1.1" 404 293
- -> /gems/rack-1.6.4.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError

There is definitely something there:

[thoraxe@t440 ~]$ ll /home/thoraxe/.rvm/gems/ruby-2.1.2/gems | grep rack-1.6.4 | wc -l
1

If I explicitly specify a directory, it works:

[thoraxe@t440 ~]$ gem server --port 8808 --dir /home/thoraxe/.rvm/gems/ruby-2.1.2 --no-daemon --debug                                                                                                              
NOTE:  Debugging mode prints all exceptions even when rescued
Using Ext extension for JSON.
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Exception `WEBrick::HTTPStatus::NotFound' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/site_ruby/2.1.0/rubygems/server.rb:604 - `/api/v1/dependencies' not found.
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /api/v1/dependencies HTTP/1.1" 404 293
- -> /api/v1/dependencies
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /specs.4.8.gz HTTP/1.1" 200 2567
- -> /specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /prerelease_specs.4.8.gz HTTP/1.1" 200 83
- -> /prerelease_specs.4.8.gz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/rack-1.6.4.gemspec.rz HTTP/1.1" 200 562
- -> /quick/Marshal.4.8/rack-1.6.4.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz HTTP/1.1" 200 1022
- -> /quick/Marshal.4.8/rack-protection-1.5.3.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz HTTP/1.1" 200 512
- -> /quick/Marshal.4.8/sinatra-1.4.6.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:02 EDT] "GET /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz HTTP/1.1" 200 276
- -> /quick/Marshal.4.8/tilt-2.0.1.gemspec.rz
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/rack-1.6.4.gem HTTP/1.1" 200 228864
- -> /gems/rack-1.6.4.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/rack-protection-1.5.3.gem HTTP/1.1" 200 18432
- -> /gems/rack-protection-1.5.3.gem
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/tilt-2.0.1.gem HTTP/1.1" 200 44544
- -> /gems/tilt-2.0.1.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
172.17.0.3 - - [03/Jul/2015:18:59:03 EDT] "GET /gems/sinatra-1.4.6.gem HTTP/1.1" 200 355840
- -> /gems/sinatra-1.4.6.gem
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError
Exception `WEBrick::HTTPStatus::EOFError' at /home/thoraxe/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:80 - WEBrick::HTTPStatus::EOFError

ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
2.4.8

[copiousfreetime]

I've confirmed that this does still exist.

% ruby -v && gem -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin15]
2.6.4

Starting up gem server

% gem server --debug
NOTE:  Debugging mode prints all exceptions even when rescued
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1726 - invalid option: no-rdoc
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1536 - invalid option: --no-rdoc
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1726 - invalid option: no-ri
Exception `OptionParser::InvalidOption' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/optparse.rb:1536 - invalid option: --no-ri
Server started at http://0.0.0.0:8808
Server started at http://[::]:8808
Using Ext extension for JSON.

Look for a gem

% gem list -q --remote --clear-sources -s http://localhost:8808/ launchy
launchy (2.4.3)

# Server output
::1 - - [30/Apr/2016:15:11:27 MDT] "GET /latest_specs.4.8.gz HTTP/1.1" 200 357
- -> /latest_specs.4.8.gz
Exception `WEBrick::HTTPStatus::EOFError' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/webrick/httpserver.rb:82 - WEBrick::HTTPStatus::EOFError

And attempt to download it

% gem fetch -q  --clear-sources -s http://localhost:8808/ launchy
ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
    bad response Not Found  404 (http://localhost:8808/gems/launchy-2.4.3.gem)

# Server output
::1 - - [30/Apr/2016:15:12:56 MDT] "GET /latest_specs.4.8.gz HTTP/1.1" 200 357
- -> /latest_specs.4.8.gz
Exception `WEBrick::HTTPStatus::NotFound' at /opt/rubies/ruby-2.3.1/lib/ruby/2.3.0/webrick/httpservlet/filehandler.rb:332 - `/gems/launchy-2.4.3.gem' not found.

Specifically using the -d option on server does result in the expected behavior.

The cause of this bug is that the default the value of options[:gemdir] is Gem.path which is array of paths. In my experiments this is:

% ruby -e 'puts Gem.path'
/Users/jeremy/Clients/ruby-together/.gem/ruby/2.3.1
/Users/jeremy/.gem/ruby/2.3.1
/opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0

When gem server instantiates an instance of Gem::Server this array of paths is passed in as the first parameter -- https://github.com/rubygems/rubygems/blob/master/lib/rubygems/commands/server_command.rb#L81-L84

During the setup and mounting of paths, Gem::Server loops over that array and adds items to the webrick mount table -- https://github.com/rubygems/rubygems/blob/master/lib/rubygems/commands/server_command.rb#L81-L84

Unfortunately, it mounts them all at the same point, and since MountTable uses a hash internally, the last one wins. Which in this case is /opt/rubies/ruby-2.3.1/lib/ruby/gems/2.3.0 and that is not where the gem files that we have installed in /Users/jeremy/Clients/ruby-together/.gem/ruby/2.3.1 live.

I see 2 potential solutions for this:

1. Change the default options[:gemdir] to be Gem.dir instead of Gem.path which would avoid the whole issue since we would have a default of a single directory.
2. Change how the /gems route is mounted and write a proc handler to do the lookup instead of a WEBrick::HTTPServlet::FileHandler.

Thoughts anyone? I don't think (2) would be all that difficult. The approach would be:

1. Create a new method Gem::Server#gems that would search for the proper gem file
2. Mount the method using @server.mount_proc "/gems", method(:gems)

[andrewhood125]

Just hit this myself settings up a private gem server. Thank goodness for this issue. 🙏 Adding the directory option fixed the issue.

[deivid-rodriguez]

Hi! We're actually going to deprecate the gem server command because we believe it hardly has any realworld usage. Do you still use this command?

[thoraxe]

What would the replacement solution be for running a gem server?

[hsbt]

I have α plan to extract it to rubygems-server plugin. You can use it after removing gem server command from rubygems.

[deivid-rodriguez]

What would the replacement solution be for running a gem server?

Something like gemstash I believe. @bronzdoc @indirect Is gemstash the alternative to gem server?

[bronzdoc]

@deivid-rodriguez Yes

We added documentation in rubygems.org but the PR hasn't landed yet