Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aws-sdk-1.42.0 crash on http_proxy #33

Closed
pda opened this issue Oct 7, 2015 · 3 comments
Closed

aws-sdk-1.42.0 crash on http_proxy #33

pda opened this issue Oct 7, 2015 · 3 comments

Comments

@pda
Copy link
Collaborator

pda commented Oct 7, 2015

Here's a crash from an app using aws-sdk-1.42.0 (~18 months old) and Ruby 2.2.3 (latest) failing to parse http_proxy="127.0.0.1:53486" as a URI.

aws-vault v2.3.2

$ aws-vault exec profilename -- bundle exec ./publish
/Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/uri/rfc3986_parser.rb:66:in `split': bad URI(is not URI?): 127.0.0.1:53486 (URI::InvalidURIError)
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/uri/rfc3986_parser.rb:72:in `parse'
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/uri/common.rb:226:in `parse'
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/uri/generic.rb:1554:in `find_proxy'
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/net/http.rb:1034:in `proxy_uri'
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/net/http.rb:1019:in `proxy?'
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/net/http.rb:869:in `connect'
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/net/http.rb:863:in `do_start'
        from /Users/pda/.rubies/ruby-2.2.3/lib/ruby/2.2.0/net/http.rb:858:in `start'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:399:in `get_credentials'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:51:in `block in set?'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:50:in `synchronize'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:50:in `set?'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:130:in `block in credentials'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:129:in `each'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:129:in `credentials'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/credential_providers.rb:61:in `access_key_id'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:549:in `build_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:491:in `block (3 levels) in client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/response.rb:175:in `call'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/response.rb:175:in `build_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/response.rb:114:in `initialize'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:203:in `new'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:203:in `new_response'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:490:in `block (2 levels) in client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:391:in `log_client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:477:in `block in client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:373:in `return_or_raise'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/core/client.rb:476:in `client_request'
        from (eval):3:in `put_object'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/s3/s3_object.rb:1752:in `write_with_put_object'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-1.42.0/lib/aws/s3/s3_object.rb:607:in `write'
        from ./publish:23:in `block in <main>'
        from ./publish:16:in `each'
        from ./publish:16:in `<main>'
exit status 1

Note that the error comes out of ruby 2.2.3 stdlib net/http trying to auto-detect the proxy. However, I failed to reproduce it in isolation:

$ http_proxy="127.0.0.1:54179" ruby -r net/http -r uri -e 'puts Net::HTTP.start("example.org", 80) { |h| h.request(Net::HTTP::Get.new(URI("http://example.org/"))) }'
#<Net::HTTPOK:0x007f8334069e88>
@pda
Copy link
Collaborator Author

pda commented Oct 7, 2015

Follow-up after upgrading to aws-sdk 1.66.0, the latest in the v1 series;

$ aws-vault exec profilename -- bundle exec ./publish
project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/credential_providers.rb:140:in `credentials':  (AWS::Errors::MissingCredentialsError)
Missing Credentials.

Unable to find AWS credentials.  You can configure your AWS credentials
a few different ways:

* Call AWS.config with :access_key_id and :secret_access_key

* Export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to ENV

* On EC2 you can run instances with an IAM instance profile and credentials
  will be auto loaded from the instance metadata service on those
  instances.

* Call AWS.config with :credential_provider.  A credential provider should
  either include AWS::Core::CredentialProviders::Provider or respond to
  the same public methods.

= Ruby on Rails

In a Ruby on Rails application you may also specify your credentials in
the following ways:

* Via a config initializer script using any of the methods mentioned above
  (e.g. RAILS_ROOT/config/initializers/aws-sdk.rb).

* Via a yaml configuration file located at RAILS_ROOT/config/aws.yml.
  This file should be formated like the default RAILS_ROOT/config/database.yml
  file.

        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/credential_providers.rb:62:in `access_key_id'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:549:in `build_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:491:in `block (3 levels) in client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/response.rb:175:in `call'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/response.rb:175:in `build_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/response.rb:114:in `initialize'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:203:in `new'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:203:in `new_response'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:490:in `block (2 levels) in client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:391:in `log_client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:477:in `block in client_request'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:373:in `return_or_raise'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:476:in `client_request'
        from (eval):3:in `put_object'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3_object.rb:1765:in `write_with_put_object'
        from project/vendor/bundle/ruby/2.2.0/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3_object.rb:611:in `write'
        from ./publish:23:in `block in <main>'
        from ./publish:16:in `each'
        from ./publish:16:in `<main>'
exit status 1

@lox lox closed this as completed in de376f3 Oct 11, 2015
lox added a commit that referenced this issue Oct 11, 2015
@lox
Merge pull request #36 from 99designs/add-protocol-to-proxy
0f14e84 
Added protocol to http_proxy, fixes #33
@lox
Copy link
Collaborator

lox commented Oct 11, 2015

I've merged a fix and will put out a release. Could you test this with 1.66.0 and open a different bug if it persists @pda?

@pda
Copy link
Collaborator Author

pda commented Oct 12, 2015

v1.42.0 no longer crashes during proxy parsing, and now successfully authenticates.
v1.66.0 still fails with that same AWS::Errors::MissingCredentialsError. I'll create a new issue for it.

@pda pda mentioned this issue Oct 12, 2015
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lox @pda