Could not find gem in any of the sources w/t require "bundler/setup" #36

Mongey · 2021-02-22T15:33:59Z

Describe the bug

To auto require gems, we specific the following.

  require "rubygems"
  require "bundler/setup"

I can remove these requires, and it works, but, wondering why it doesn't

Error Stack Trace

Init error when loading handler handler.Handler.list
2021-02-22T14:55:03.403+00:00	{
2021-02-22T14:55:03.403+00:00	"errorMessage": "Could not find public_suffix-3.1.1 in any of the sources",
2021-02-22T14:55:03.403+00:00	"errorType": "Init<Bundler::GemNotFound>",
2021-02-22T14:55:03.403+00:00	"stackTrace": [
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/spec_set.rb:87:in `block in materialize'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/spec_set.rb:81:in `map!'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/spec_set.rb:81:in `materialize'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/definition.rb:175:in `specs'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/definition.rb:245:in `specs_for'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/definition.rb:227:in `requested_specs'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/runtime.rb:91:in `block in definition_method'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/runtime.rb:20:in `setup'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler.rb:148:in `setup'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/setup.rb:20:in `block in <top (required)>'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/ui/shell.rb:136:in `with_level'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/ui/shell.rb:88:in `silence'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/setup.rb:20:in `<top (required)>'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/task/config/initializers/autoloader.rb:11:in `<top (required)>'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/task/handler.rb:1:in `<top (required)>'",

The text was updated successfully, but these errors were encountered:

navarasu · 2021-02-23T16:42:51Z

Can you share yours serverless.yml (Need ruby_layer config info to check whether you are using local bundler or docker version)?
If local give your local bundler version and ruby version.

Mongey · 2021-02-23T16:50:44Z

@navarasu I'm using the docker version, running via CircleCI.

provider:
  name: aws
  runtime: ruby2.5

plugins:
  - serverless-ruby-layer

package:
  include:
    - Gemfile
    - Gemfile.lock
    - handler.rb
    - config/**
    - app/**
    - ./

custom:
  rubyLayer:
    use_docker: true


RUBY VERSION
   ruby 2.5.8p224

BUNDLED WITH
   2.1.4

navarasu · 2021-02-23T17:16:38Z

Thanks. Is your aws runtime version 2.5? or 2.7?

provider:
  name: aws
  runtime: ruby2.5

Mongey · 2021-02-23T17:30:46Z

Whoops, I forgot that part, It's 2.5

provider:
  name: aws
  runtime: ruby2.5

jagthedrummer · 2021-04-02T20:54:15Z

I think this is the same problem I'm seeing in #40, and it should hopefully be fixed by #41. (I haven't quite figured out how to run a local copy of a serverless plugin, so I haven't tested it directly. 😬 )

navarasu · 2021-05-02T21:22:36Z

@Mongey I am not sure that issue #40 is related to your issue. Currently, I have created an example for your case and tried to reproduce the issue in my local with the 1.4.0 version.

With the existing serverless version (1.4.0) itself, I have tried the requires all example.

require 'rubygems'
require 'bundler/setup'
Bundler.require(:default)

It is working without any issue

Below is the example I have tried. The folder contains the following four files,

➜  basic-requires-all tree
.
├── Gemfile
├── Gemfile.lock
├── handler.rb
└── serverless.yml

0 directories, 4 files

Gemfile

source 'https://rubygems.org'
gem 'httparty'
gem 'nokogiri'

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    httparty (0.18.1)
      mime-types (~> 3.0)
      multi_xml (>= 0.5.2)
    mime-types (3.3.1)
      mime-types-data (~> 3.2015)
    mime-types-data (3.2021.0225)
    mini_portile2 (2.5.1)
    multi_xml (0.6.0)
    nokogiri (1.11.3)
      mini_portile2 (~> 2.5.0)
      racc (~> 1.4)
    racc (1.5.2)

PLATFORMS
  ruby

DEPENDENCIES
  httparty
  nokogiri

BUNDLED WITH
   2.1.4

handler.rb

require 'rubygems'
require 'bundler/setup'
Bundler.require(:default)

def hello(event:, context:)
  body = HTTParty.get("https://github.com").body
  doc = Nokogiri::HTML(body)

  { statusCode: 200, body: doc.title }
end

serverless.yml

service: basic 

plugins:
  - serverless-ruby-layer
provider:
  name: aws
  runtime: ruby2.5

custom:
  rubyLayer:
    use_docker: true

package:
  include:
    - Gemfile
    - Gemfile.lock

functions:
  hello:
    handler: handler.hello

I am not able to reproduce your issue. As @jagthedrummer said, the mismatch between Gemfile.lock of your local and Gems packed in docker might vary and it might reproduce your issue. So #40 fix might solve your issue.

So please share the appropriate Gemfile and Gemfile.lock with me to reproduce and verify your issue. If that is the case, then I can do a few other changes to support requires all gem option
Thanks

navarasu · 2021-05-03T11:30:50Z

Hi @Mongey, I am able to reproduce the issue by replacing it with a different version for a dependency gem in Gemfile.lock.
. In the above example, I have replaced it with the following Gemfile.lock

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    httparty (0.18.1)
      mime-types (~> 3.0)
      multi_xml (>= 0.5.2)
    mime-types (3.3.1)
      mime-types-data (~> 3.2015)
    mime-types-data (3.2020.1104)
    mini_portile2 (2.5.1)
    multi_xml (0.6.0)
    nokogiri (1.11.3)
      mini_portile2 (~> 2.5.0)
      racc (~> 1.4)
    racc (1.5.2)

PLATFORMS
  ruby

DEPENDENCIES
  httparty
  nokogiri

BUNDLED WITH
   2.1.4

In Gemfile.lock, the mime-types-data gem version is 3.2020.1104. But the packed gem contains 3.2021.0225. I deployed the change. Now I get the below error,

{
    "errorMessage": "Could not find mime-types-data-3.2020.1104 in any of the sources",
    "errorType": "Init<Bundler::GemNotFound>",
    "stackTrace": [
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/spec_set.rb:89:in `block in materialize'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/spec_set.rb:82:in `map!'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/spec_set.rb:82:in `materialize'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/definition.rb:185:in `specs'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/definition.rb:255:in `specs_for'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/definition.rb:237:in `requested_specs'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/runtime.rb:91:in `block in definition_method'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/runtime.rb:20:in `setup'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler.rb:148:in `setup'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/setup.rb:20:in `block in <top (required)>'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/ui/shell.rb:136:in `with_level'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/ui/shell.rb:88:in `silence'",
        "/var/runtime/gems/bundler-2.2.15/lib/bundler/setup.rb:20:in `<top (required)>'",
        "/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
        "/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
        "/var/task/handler.rb:2:in `<top (required)>'",
        "/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
        "/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'"
    ]
}

…nd Gemfile.lock in package if lock file is present

navarasu · 2021-05-03T12:26:56Z

Added example and test for bundler.require all.
Include Gemfile and Gemfile.lock in the function package if the lock file is present.

So the below config not required

package:
  include:
    - Gemfile
    - Gemfile.lock

It will auto-include Gemfile and Gemfile.lock if Gemfile.lock is present. Refer example here

Added test case for Gemfile and Gemfile.lock auto include

navarasu · 2021-05-03T12:28:27Z

Fix #40 solves this ticket. I will update the doc and will create a new release

#36 Added example and test for bundler.require all. Include Gemfile and Gemfile.lock in package if lock file is present

navarasu · 2021-05-15T04:54:29Z

@Mongey This fix is included in release 1.5.0. Please update the npm version and verify.

npm i serverless-ruby-layer

Mongey added the bug Something isn't working label Feb 22, 2021

navarasu self-assigned this May 2, 2021

navarasu added this to the 1.5.0 milestone May 2, 2021

navarasu added a commit that referenced this issue May 3, 2021

#36 Added example and test for bundler.require all. Include Gemfile a…

e712610

…nd Gemfile.lock in package if lock file is present

navarasu added a commit that referenced this issue May 3, 2021

#36 Updated docs with bundler.require all example

05fe853

navarasu linked a pull request May 3, 2021 that will close this issue

#36 Added example and test for bundler.require all. Include Gemfile and Gemfile.lock in package if lock file is present #48

Merged

navarasu closed this as completed in #48 May 3, 2021

navarasu added a commit that referenced this issue May 3, 2021

Merge pull request #48 from navarasu/fix_36_requires_all

973d462

#36 Added example and test for bundler.require all. Include Gemfile and Gemfile.lock in package if lock file is present

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not find gem in any of the sources w/t require "bundler/setup" #36

Could not find gem in any of the sources w/t require "bundler/setup" #36

Mongey commented Feb 22, 2021

navarasu commented Feb 23, 2021 •

edited

Loading

Mongey commented Feb 23, 2021 •

edited

Loading

navarasu commented Feb 23, 2021 •

edited

Loading

Mongey commented Feb 23, 2021 •

edited

Loading

jagthedrummer commented Apr 2, 2021

navarasu commented May 2, 2021 •

edited

Loading

navarasu commented May 3, 2021

navarasu commented May 3, 2021

navarasu commented May 3, 2021

navarasu commented May 15, 2021

Could not find *gem* in any of the sources w/t require "bundler/setup" #36

Could not find *gem* in any of the sources w/t require "bundler/setup" #36

Comments

Mongey commented Feb 22, 2021

navarasu commented Feb 23, 2021 • edited Loading

Mongey commented Feb 23, 2021 • edited Loading

navarasu commented Feb 23, 2021 • edited Loading

Mongey commented Feb 23, 2021 • edited Loading

jagthedrummer commented Apr 2, 2021

navarasu commented May 2, 2021 • edited Loading

navarasu commented May 3, 2021

navarasu commented May 3, 2021

navarasu commented May 3, 2021

navarasu commented May 15, 2021

Could not find gem in any of the sources w/t require "bundler/setup" #36

Could not find gem in any of the sources w/t require "bundler/setup" #36

navarasu commented Feb 23, 2021 •

edited

Loading

Mongey commented Feb 23, 2021 •

edited

Loading

navarasu commented Feb 23, 2021 •

edited

Loading

Mongey commented Feb 23, 2021 •

edited

Loading

navarasu commented May 2, 2021 •

edited

Loading