New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS SDK Uninitialized Constant Error #3645

Closed
smythey21 opened this Issue Feb 4, 2016 · 15 comments

Comments

Projects
None yet
6 participants
@smythey21

Using the latest version of aws-sdk, we started receiving the error uninitialized constant Aws::Client::Errors with jruby-9.0.5.0 (though it works fine with 9.0.4.0) when calling .exists? on an s3 resource. Wondering if anything has changed in 9.0.5.0 that would cause this?

[2016-02-04 16:09:23 -0500] [d5ef5747-ebee-40] uninitialized constant Aws::Client::Errors
** [Raven] Event not sent due to excluded environment: development
[2016-02-04 16:09:23 -0500] [d5ef5747-ebee-40] ["/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:25:in `const_get'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:25:in `error'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:9:in `block in call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/response.rb:43:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/response.rb:43:in `block in on'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:139:in `block in on_success'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:166:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:166:in `block in listener'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:130:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:130:in `on_done'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/http/response.rb:137:in `on_success'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/response.rb:42:in `on'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/xml/error_handler.rb:8:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_request_signer.rb:64:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_redirects.rb:15:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/retry_errors.rb:87:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_md5s.rb:33:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_expect_100_continue.rb:21:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_bucket_dns.rb:31:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/rest/handler.rb:7:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/user_agent.rb:12:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/plugins/endpoint.rb:41:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/param_validator.rb:21:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:18:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/plugins/response_target.rb:21:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/seahorse/client/request.rb:70:in `send_request'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/poller.rb:61:in `send_request'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/poller.rb:47:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:104:in `block in poll'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:101:in `loop'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:101:in `poll'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:91:in `block in wait'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:90:in `catch'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:90:in `block in wait'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:89:in `catch'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/waiters/waiter.rb:89:in `wait'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-core-2.2.0/lib/aws-sdk-core/client_waiters.rb:110:in `wait_until'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-resources-2.2.0/lib/aws-sdk-resources/operations.rb:214:in `call'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-resources-2.2.0/lib/aws-sdk-resources/operation_methods.rb:19:in `block in wait_until_exists'",
 "/Users/kevinsmyth/.rbenv/versions/jruby-9.0.5.0/lib/ruby/gems/shared/gems/aws-sdk-resources-2.2.0/lib/aws-sdk-resources/resource.rb:131:in `exists?'",
@torrancew

This comment has been minimized.

Show comment
Hide comment
@torrancew

torrancew Feb 4, 2016

@smythey21 I see the aws-sdk README notes that this is a common error when unintentionally upgrading from v1.x to v2.x of their API. Is it possible you're impacted by that?

Edit Seems that that's the opposite error message from what is presented here (v1 used AWS, v2 Aws)

@smythey21 I see the aws-sdk README notes that this is a common error when unintentionally upgrading from v1.x to v2.x of their API. Is it possible you're impacted by that?

Edit Seems that that's the opposite error message from what is presented here (v1 used AWS, v2 Aws)

@smythey21

This comment has been minimized.

Show comment
Hide comment
@smythey21

smythey21 Feb 5, 2016

@torrancew we are using aws-sdk 2.2.13 and jruby 9.0.4.0 in production with no issues, but thanks for pointing that out :)

@torrancew we are using aws-sdk 2.2.13 and jruby 9.0.4.0 in production with no issues, but thanks for pointing that out :)

@jonp

This comment has been minimized.

Show comment
Hide comment
@jonp

jonp Feb 9, 2016

@smythey21 I'm getting the same thing trying to use S3.

It seems that the call to const_set in the aws-sdk-core gem doesn't sit well with jruby 9. In old jruby 1.7, that would actually set the const in the module and Aws::S3 would correctly think that its name was "Aws::S3". In jruby 9, however, it doesn't seem to "take" in that you can still use Aws::S3, but it thinks that its name is a module instance (see below). So, in the line above your error the aws-sdk-core gem is (a bit sloppily) trying to do some tricks with the name of the module to try to decide what handler to use for the error, only to have an unexpected name to parse. It was expecting something like Aws::S3::Client, from which it wants to pull "S3", but instead gets something like #<#<Module:0x9addea4>::Client:0x5341eab4>, from which it pulls "Client."

So, there are two bugs: first: jruby's const_set is acting odd in this case and second: the aws gem is playing fast and loose with names.

The simple repro for this is:

» irb
jruby-9.0.5.0 :001 > require 'aws-sdk'; Aws::S3
 => #<Module:0x4ff4357f>

The return should be Aws::S3 and not #<Module:0x4ff4357f>.

The good news is that a second call to const_set does seem to work, so if you tuck this line in your app's startup somewhere after where you require 'aws-sdk' you'll have a workaround:

module Aws; const_set(:S3, Aws::S3); end # or whichever AWS service you're trying to use

Unfortunately, I can't seem to repro this without using the aws-sdk gem, so I can't offer conclusive proof that it's really jruby's fault. OTOH, the fact that it behaves correctly on jruby-1.7 is a strong indicator.

jonp commented Feb 9, 2016

@smythey21 I'm getting the same thing trying to use S3.

It seems that the call to const_set in the aws-sdk-core gem doesn't sit well with jruby 9. In old jruby 1.7, that would actually set the const in the module and Aws::S3 would correctly think that its name was "Aws::S3". In jruby 9, however, it doesn't seem to "take" in that you can still use Aws::S3, but it thinks that its name is a module instance (see below). So, in the line above your error the aws-sdk-core gem is (a bit sloppily) trying to do some tricks with the name of the module to try to decide what handler to use for the error, only to have an unexpected name to parse. It was expecting something like Aws::S3::Client, from which it wants to pull "S3", but instead gets something like #<#<Module:0x9addea4>::Client:0x5341eab4>, from which it pulls "Client."

So, there are two bugs: first: jruby's const_set is acting odd in this case and second: the aws gem is playing fast and loose with names.

The simple repro for this is:

» irb
jruby-9.0.5.0 :001 > require 'aws-sdk'; Aws::S3
 => #<Module:0x4ff4357f>

The return should be Aws::S3 and not #<Module:0x4ff4357f>.

The good news is that a second call to const_set does seem to work, so if you tuck this line in your app's startup somewhere after where you require 'aws-sdk' you'll have a workaround:

module Aws; const_set(:S3, Aws::S3); end # or whichever AWS service you're trying to use

Unfortunately, I can't seem to repro this without using the aws-sdk gem, so I can't offer conclusive proof that it's really jruby's fault. OTOH, the fact that it behaves correctly on jruby-1.7 is a strong indicator.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Feb 13, 2016

Member

Wow, this is really weird. It works fine in a simple script:

$ jruby -e 'module Foo; m = Module.new { extend Enumerable }; const_set :Bar, m; end; p Foo::Bar'
Foo::Bar

But I can confirm the const_set in aws-sdk-core.rb in add_service does not cause it to be set.

This is likely related to another bug report that I can't find right now.

After some investigation, it appears that 02efc4d by @kares broke this. Many paths now cause constants to be defined as "hidden", which skips setting their names.

@kares Can you look into this and repair your patch?

Member

headius commented Feb 13, 2016

Wow, this is really weird. It works fine in a simple script:

$ jruby -e 'module Foo; m = Module.new { extend Enumerable }; const_set :Bar, m; end; p Foo::Bar'
Foo::Bar

But I can confirm the const_set in aws-sdk-core.rb in add_service does not cause it to be set.

This is likely related to another bug report that I can't find right now.

After some investigation, it appears that 02efc4d by @kares broke this. Many paths now cause constants to be defined as "hidden", which skips setting their names.

@kares Can you look into this and repair your patch?

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Feb 13, 2016

Member

I'm rather surprised this didn't regress any tests. We need to add some ruby specs for this.

Member

headius commented Feb 13, 2016

I'm rather surprised this didn't regress any tests. We need to add some ruby specs for this.

@kares

This comment has been minimized.

Show comment
Hide comment
@kares

kares Feb 13, 2016

Member

yy, can look into this next week (unless its fixed by than). thanks.

Member

kares commented Feb 13, 2016

yy, can look into this next week (unless its fixed by than). thanks.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Feb 14, 2016

Member

@kares I may have a look at it during my flights home today. Can you point me toward any tests you used to verify your fixes, so I don't regress anything?

Member

headius commented Feb 14, 2016

@kares I may have a look at it during my flights home today. Can you point me toward any tests you used to verify your fixes, so I don't regress anything?

@kares

This comment has been minimized.

Show comment
Hide comment
@kares

kares Feb 14, 2016

Member

did it around constant "hiding" in Java int (e.g. private inner classes) thus spec:ji should be green ...
mostly needed hidden flag to be part of the const storing API (have some local unfinished JI tunings that consume it) ... if all specs/tests pass do not worry about changing anything else.

Member

kares commented Feb 14, 2016

did it around constant "hiding" in Java int (e.g. private inner classes) thus spec:ji should be green ...
mostly needed hidden flag to be part of the const storing API (have some local unfinished JI tunings that consume it) ... if all specs/tests pass do not worry about changing anything else.

@headius

This comment has been minimized.

Show comment
Hide comment
Member

headius commented Feb 14, 2016

@kares Ok.

@kares

This comment has been minimized.

Show comment
Hide comment
@kares

kares Feb 16, 2016

Member

the issue relates to autoloading ... able to reproduce in a small script. what seems specific here is the auto-loading part doing a callback :

  module GH3645
    class << self

      def add_mod(name) # call this from autoloaded script
        mod = Module.new { extend GH3645 }
        const_set(name, mod)
        mod
      end

    end
    autoload :S3, File.expand_path('GH-3645_autoload', File.dirname(__FILE__))
  end
Member

kares commented Feb 16, 2016

the issue relates to autoloading ... able to reproduce in a small script. what seems specific here is the auto-loading part doing a callback :

  module GH3645
    class << self

      def add_mod(name) # call this from autoloaded script
        mod = Module.new { extend GH3645 }
        const_set(name, mod)
        mod
      end

    end
    autoload :S3, File.expand_path('GH-3645_autoload', File.dirname(__FILE__))
  end
@kares

This comment has been minimized.

Show comment
Hide comment
@kares

kares Feb 16, 2016

Member

OK, thing I grasp what's going on. the regression is related (and expected) due previous autoload logic.
regression came in with fixing autoload not to load on const_defined? checks b216c5d ... previously the const_set would always initialize the autoloaded constant "early" and thus set its parent module, whether after the change it does not since there's logic to wait for autoload-ing to finish loading the file. the "finish" autoload logic simply swaps the constant in the table.

I believe the side effect of the previous logic to set the parent (and name) early on was better than to add it to finishAutoload ... but will look into MRI's test suite for more on how/whether this is tested.

Member

kares commented Feb 16, 2016

OK, thing I grasp what's going on. the regression is related (and expected) due previous autoload logic.
regression came in with fixing autoload not to load on const_defined? checks b216c5d ... previously the const_set would always initialize the autoloaded constant "early" and thus set its parent module, whether after the change it does not since there's logic to wait for autoload-ing to finish loading the file. the "finish" autoload logic simply swaps the constant in the table.

I believe the side effect of the previous logic to set the parent (and name) early on was better than to add it to finishAutoload ... but will look into MRI's test suite for more on how/whether this is tested.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Feb 16, 2016

Member

@kares Ok I'll let you take it from here. I did try the patching setConstant to default to non-hidden, which didn't break any JI tests (did you add some for the hidden stuff?)...but it also did not fix the issue.

Member

headius commented Feb 16, 2016

@kares Ok I'll let you take it from here. I did try the patching setConstant to default to non-hidden, which didn't break any JI tests (did you add some for the hidden stuff?)...but it also did not fix the issue.

kares added a commit that referenced this issue Feb 16, 2016

make sure auto-load does set-up parent module and name (see #3645)
... previously this worked as a side effect of fetchConstant "loading" auto-loads
but since (b216c5d) `autoload` has been fixed to not initialize prematurely

kares added a commit that referenced this issue Feb 16, 2016

set parent module and name early on even for auto-load (closing #3645)
... this is not crucial but restores the same behavior as in JRuby < 9.0.5.0
@kares

This comment has been minimized.

Show comment
Hide comment
@kares

kares Feb 16, 2016

Member

@headius yes there's a spec testing hiding non public inner classes as constants
... as noted this is not related to the commit you linked but a side effect of another (that is why it did not fix).

now fixed and expected to behave as before.

Member

kares commented Feb 16, 2016

@headius yes there's a spec testing hiding non public inner classes as constants
... as noted this is not related to the commit you linked but a side effect of another (that is why it did not fix).

now fixed and expected to behave as before.

@kares kares closed this Feb 16, 2016

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Feb 16, 2016

Member

@kares Excellent, thank you!

Member

headius commented Feb 16, 2016

@kares Excellent, thank you!

@dcrec1

This comment has been minimized.

Show comment
Hide comment
@dcrec1

dcrec1 Nov 17, 2016

@jonp solution worked for me.

dcrec1 commented Nov 17, 2016

@jonp solution worked for me.

mhoroschun pushed a commit to spinify/logstash-input-cloudwatch-logs that referenced this issue Feb 3, 2017

jamtur01 added a commit to EmpaticoOrg/logstash-input-cloudwatch-logs that referenced this issue Mar 25, 2017

This patch is related to the autoloading and ruby
When I run the plugin on Logstash 5.2.2 I see:

`Aws::Client::Errors isn't found`

I did some exploring and found the S3 plugins had some similar issues.
This was their fix.

```
The fix exist in jruby 9k but not in the current jruby, not sure when or
it will be backported jruby/jruby#3645

AWS is doing tricky name discovery in the module to generate the correct
error class and this strategy is bogus in jruby and `eager_autoload`
don't fix this issue.

This will be a short lived patch since AWS is removing the need.  see:
aws/aws-sdk-ruby#1301 (comment)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment