Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checksum validation failed since aws-sdk-s3 1.132.0 #2888

Closed
dentarg opened this issue Jul 31, 2023 · 1 comment · Fixed by #2889
Closed

Checksum validation failed since aws-sdk-s3 1.132.0 #2888

dentarg opened this issue Jul 31, 2023 · 1 comment · Fixed by #2889
Assignees
Labels
bug This issue is a bug. investigating Issue is being investigated

Comments

@dentarg
Copy link

dentarg commented Jul 31, 2023

Describe the bug

Since aws-sdk-s3 1.132.0 (#2886 I suspect) the following code fails with the following error:

$ cat tmp/aws-sdk-s3.rb
require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "nokogiri"
  gem "aws-sdk-s3", ENV.fetch("SDK_VERSION", "1.132.0")
end

require "aws-sdk-s3"
require "tempfile"

Aws.config[:s3] = { stub_responses: true }

path = Tempfile.new.path
s3_bucket = Aws::S3::Resource.new.bucket("BUCKET")
s3_bucket.object("KEY").download_file(path)
$ SDK_VERSION=1.132.0 ruby tmp/aws-sdk-s3.rb
Fetching gem metadata from https://rubygems.org/......
Resolving dependencies...
Using aws-eventstream 1.2.0
Using aws-partitions 1.794.0
Using aws-sigv4 1.6.0
Using bundler 2.4.13
Using racc 1.7.1
Using jmespath 1.6.2
Using nokogiri 1.15.3 (arm64-darwin)
Using aws-sdk-core 3.180.0
Using aws-sdk-kms 1.71.0
Using aws-sdk-s3 1.132.0
/Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:218:in `block in add_verify_response_checksum_handlers': Checksum validation failed on x-amz-checksum-SHA1 computed: 2jmj7l5rSw0yVb/vlWAYkK/YBwk=, expected: ChecksumSHA1 (Aws::Errors::ChecksumError)
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/http/response.rb:146:in `block in on_success'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/http/response.rb:173:in `block in listener'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/http/response.rb:182:in `block in emit'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/http/response.rb:182:in `each'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/http/response.rb:182:in `emit'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/http/response.rb:112:in `signal_done'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/stub_responses.rb:98:in `signal_http'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/stub_responses.rb:68:in `apply_stub'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/stub_responses.rb:57:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/plugins/content_length.rb:24:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/streaming_retry.rb:71:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/s3_signer.rb:73:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/s3_host_id.rb:17:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/xml/error_handler.rb:10:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/sign.rb:49:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/transfer_encoding.rb:26:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:12:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/s3_signer.rb:48:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/redirects.rb:20:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/user_agent.rb:37:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/http_checksum.rb:19:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/endpoint_pattern.rb:30:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:136:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/request_compression.rb:94:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/client_stubs.rb:36:in `block in setup_stubbing'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/expect_100_continue.rb:23:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/bucket_name_restrictions.rb:21:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/rest/handler.rb:10:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/recursion_detection.rb:18:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/endpoints.rb:41:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/endpoint_discovery.rb:84:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/plugins/endpoint.rb:47:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/param_validator.rb:26:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/plugins/raise_response_errors.rb:16:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/skip_whole_multipart_get_checksums.rb:18:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:24:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/dualstack.rb:21:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/accelerate.rb:43:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:111:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:16:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/plugins/response_target.rb:24:in `call'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/seahorse/client/request.rb:72:in `send_request'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/client.rb:6081:in `get_object'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/file_downloader.rb:163:in `single_request'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/file_downloader.rb:77:in `multipart_download'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/file_downloader.rb:41:in `block in download'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/user_agent.rb:28:in `feature'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/file_downloader.rb:39:in `download'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/customizations/object.rb:512:in `block in download_file'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-core-3.180.0/lib/aws-sdk-core/plugins/user_agent.rb:28:in `feature'
	from /Users/dentarg/.arm64_rubies/3.2.2/lib/ruby/gems/3.2.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/customizations/object.rb:511:in `download_file'
	from tmp/aws-sdk-s3.rb:16:in `<main>'

Expected Behavior

No failure, like it was using aws-sdk-s3 1.131.0

$ SDK_VERSION=1.131.0 ruby tmp/aws-sdk-s3.rb
Fetching gem metadata from https://rubygems.org/......
Resolving dependencies...
Using aws-eventstream 1.2.0
Using aws-partitions 1.794.0
Using jmespath 1.6.2
Using aws-sigv4 1.6.0
Using bundler 2.4.13
Using racc 1.7.1
Using aws-sdk-core 3.180.0
Using nokogiri 1.15.3 (arm64-darwin)
Using aws-sdk-kms 1.71.0
Using aws-sdk-s3 1.131.0

Current Behavior

See "Describe the bug"

Reproduction Steps

See "Describe the bug"

Possible Solution

No response

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-s3

Environment details (Version of Ruby, OS environment)

Ruby 3.2.2

@dentarg dentarg added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 31, 2023
@alextwoods alextwoods added investigating Issue is being investigated and removed needs-triage This issue or PR still needs to be triaged. labels Jul 31, 2023
@alextwoods alextwoods self-assigned this Jul 31, 2023
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. investigating Issue is being investigated
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants