Bundler.require(:default) is requiring all groups.

[jcoyne]

I have a program that has this Gemfile:

# frozen_string_literal: true

source 'https://rubygems.org'

gem 'activesupport', require: 'active_support'
gem 'aws-sdk-s3'
gem 'dry-struct'
gem 'dry-types'
gem 'faraday'
gem 'faraday-encoding'
gem 'faraday-http-cache'
gem 'faraday_middleware'
gem 'json-ld'
gem 'marc'
gem 'sparql'
gem 'zeitwerk'

group :development, :test do
  gem 'pry-byebug'
  gem 'rdf-turtle'
  gem 'rspec'
  gem 'rspec_junit_formatter' # used by CircleCI
  gem 'rubocop'
  gem 'rubocop-rspec'
  gem 'simplecov', '~> 0.21'
  gem 'vcr'
  gem 'webmock'
end

And in the program itself begins with:

require 'bundler'
Bundler.require(:default) # this is line 4 of lib/rdf2marc.rb

And In my deployed environment this error is raised:

    "errorMessage": "Could not find byebug-11.1.3 in any of the sources",
    "errorType": "Init<Bundler::GemNotFound>",
    "stackTrace": [
        "/var/runtime/gems/bundler-2.2.23/lib/bundler/spec_set.rb:91:in `block in materialize'",
        "/var/runtime/gems/bundler-2.2.23/lib/bundler/spec_set.rb:86:in `map!'",
        "/var/runtime/gems/bundler-2.2.23/lib/bundler/spec_set.rb:86:in `materialize'",
        "/var/runtime/gems/bundler-2.2.23/lib/bundler/definition.rb:190:in `specs'",
        "/var/runtime/gems/bundler-2.2.23/lib/bundler/definition.rb:259:in `specs_for'",
        "/var/runtime/gems/bundler-2.2.23/lib/bundler/runtime.rb:20:in `setup'",
        "/var/runtime/gems/bundler-2.2.23/lib/bundler.rb:151:in `setup'",
        "/var/runtime/gems/bundler-2.2.23/lib/bundler.rb:174:in `require'",
        "/var/task/lib/rdf2marc.rb:4:in `<top (required)>'",
        "/var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
        "/var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
        "/var/task/lambda_function.rb:4:in `<top (required)>'",
        "/var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
        "/var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'"
    ]

I'm confused as why bundler is even looking for byebug since it is not in the default group.

[rnhurt]

I'm seeing the same thing.

Gemfile:

# frozen_string_literal: true

source 'https://rubygems.org'
ruby '~> 2.7.0'

gem 'aws-sdk-cognitoidentityprovider'
gem 'aws-sdk-ssm'
gem 'httparty'

group :development, :test do
  gem 'aws-sdk-cloudformation'
  gem 'aws-sdk-cloudwatchlogs'
  gem 'aws-sdk-inspector'
  gem 'aws-sdk-s3'
  gem 'pry'
  gem 'rake'
  gem 'rubyzip', require: 'zip'
end

Ruby program begins with:

require 'bundler'
Bundler.require(:default)
require 'net/http'
require 'base64'

And errors with:

{
  "errorMessage": "Could not find aws-sdk-cloudformation-1.58.0 in any of the sources",
  "errorType": "Init<Bundler::GemNotFound>",
  "stackTrace": [
    "/var/runtime/gems/bundler-2.2.23/lib/bundler/spec_set.rb:91:in `block in materialize'",
    "/var/runtime/gems/bundler-2.2.23/lib/bundler/spec_set.rb:86:in `map!'",
    "/var/runtime/gems/bundler-2.2.23/lib/bundler/spec_set.rb:86:in `materialize'",
    "/var/runtime/gems/bundler-2.2.23/lib/bundler/definition.rb:190:in `specs'",
    "/var/runtime/gems/bundler-2.2.23/lib/bundler/definition.rb:259:in `specs_for'",
    "/var/runtime/gems/bundler-2.2.23/lib/bundler/runtime.rb:20:in `setup'",
    "/var/runtime/gems/bundler-2.2.23/lib/bundler.rb:151:in `setup'",
    "/var/runtime/gems/bundler-2.2.23/lib/bundler.rb:174:in `require'",
    "/var/task/main.rb:2:in `<top (required)>'",
    "/var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
    "/var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'"
  ]
}

UPDATE

If I remove the "test" and "development" gems from the Gemfile, everything works perfectlly:

# frozen_string_literal: true

source 'https://rubygems.org'
ruby '~> 2.7.0'

gem 'aws-sdk-cognitoidentityprovider'
gem 'aws-sdk-ssm'
gem 'httparty'

[deivid-rodriguez]

I will need to look into this more closely, but the culprit I've identified in the past for this kind of issue is that Bundler.require (or Bundler.setup) had been called previously with other groups, and once that happens that's memoized and subsequent invocations with other groups have no effect and are essentially no-ops.

Can you add some logging to Bundler.setup and figure out how many times it's being called, and which groups are being passed each time?

[rnhurt]

I will need to look into this more closely, but the culprit I've identified in the past for this kind of issue is that Bundler.require (or Bundler.setup) had been called previously with other groups, and once that happens that's memoized and subsequent invocations with other groups have no effect and are essentially no-ops.

Unfortunately (or fortunately) I wasn't able to reproduce the problem in a simple example. But this comment makes me think that my problem might be with calling Bundler.require multiple times. I'll have to look at it a bit more closely, but as it's an AWS Lambda function it's a bit more tricky to reproduce/test.

[deivid-rodriguez]

I can imagine. I think we can probably print a warning if Bundler.setup is called multiple times with different groups, since that's a sign of something unexpected going on.