NoMethodError: undefined method `bytesize'

[blackjiro]

Hi Janko, thank you for your awesome job!

We encountered NoMethodError: undefined method 'bytesize' error from down/chunked_io.rb:173:in `readpartial'.

We use Shrine gem which depends on your gem and got error as traceback below.
aws-sdk-core gem's update bellow seems to be related this issue?
Much appreciate if you could look on this issue.

aws-sdk-core update

aws/aws-sdk-ruby#2357

traceback

NoMethodError: undefined method `bytesize' for #<Array:0x00007f8c067a8688>
  from down/chunked_io.rb:173:in `readpartial'
  from shrine/uploaded_file.rb:146:in `copy_stream'
  from shrine/uploaded_file.rb:146:in `block in stream'
  from shrine/uploaded_file.rb:98:in `open'
  from shrine/uploaded_file.rb:146:in `stream'
  from shrine/uploaded_file.rb:122:in `download'
  from shrine/plugins/derivatives.rb:272:in `process_derivatives'
  from shrine/plugins/derivatives.rb:184:in `create_derivatives'
  from shrine/plugins/derivatives.rb:44:in `block in define_model_methods'
  from (eval):4:in `block (3 levels) in run_file'
  from active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
  from active_record/relation/batches.rb:70:in `each'
  from active_record/relation/batches.rb:70:in `block in find_each'
  from active_record/relation/batches.rb:136:in `block in find_in_batches'
  from active_record/relation/batches.rb:238:in `block in in_batches'
  from active_record/relation/batches.rb:222:in `loop'
  from active_record/relation/batches.rb:222:in `in_batches'
  from active_record/relation/batches.rb:135:in `find_in_batches'
  from active_record/relation/batches.rb:69:in `find_each'
  from active_record/querying.rb:21:in `find_each'
  from (eval):3:in `block (2 levels) in run_file'
  from seed-fu/runner.rb:46:in `eval'
  from seed-fu/runner.rb:46:in `block (2 levels) in run_file'
  from seed-fu/runner.rb:58:in `block in open'
  from seed-fu/runner.rb:57:in `open'
  from seed-fu/runner.rb:57:in `open'
  from seed-fu/runner.rb:36:in `block in run_file'
  from active_record/connection_adapters/abstract/database_statements.rb:280:in `block in transaction'
  from active_record/connection_adapters/abstract/transaction.rb:280:in `block in within_new_transaction'
  from active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
  from active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
  from active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
  from active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
  from active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
  from active_record/connection_adapters/abstract/transaction.rb:278:in `within_new_transaction'
  from active_record/connection_adapters/abstract/database_statements.rb:280:in `transaction'
  from active_record/transactions.rb:212:in `transaction'
  from seed-fu/runner.rb:35:in `run_file'
  from seed-fu/runner.rb:26:in `block in run'
  from seed-fu/runner.rb:25:in `each'
  from seed-fu/runner.rb:25:in `run'
  from seed-fu.rb:29:in `seed'
  from tasks/seed_fu.rake:36:in `block (2 levels) in <main>'
  from rake/task.rb:281:in `block in execute'
  from rake/task.rb:281:in `each'
  from rake/task.rb:281:in `execute'
  from rake/task.rb:219:in `block in invoke_with_call_chain'
  from monitor.rb:235:in `mon_synchronize'
  from rake/task.rb:199:in `invoke_with_call_chain'
  from rake/task.rb:188:in `invoke'
  from rake/application.rb:160:in `invoke_task'
  from rake/application.rb:116:in `block (2 levels) in top_level'
  from rake/application.rb:116:in `each'
  from rake/application.rb:116:in `block in top_level'
  from rake/application.rb:125:in `run_with_threads'
  from rake/application.rb:110:in `top_level'
  from rails/commands/rake/rake_command.rb:23:in `block in perform'
  from rake/application.rb:186:in `standard_exception_handling'
  from rails/commands/rake/rake_command.rb:20:in `perform'
  from rails/command.rb:48:in `invoke'
  from rails/commands.rb:18:in `<main>'
  from bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
  from bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
  from bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
  from bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
  from bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
  from active_support/dependencies.rb:324:in `block in require'
  from active_support/dependencies.rb:291:in `load_dependency'
  from active_support/dependencies.rb:324:in `require'
  from bin/rails:12:in `<main>'

[janko]

Thanks for the report and pointing to the relevant change in aws-sdk-s3. Yeah, it seems we need to update the S3 storage in Shrine to handle the additional yielded headers in S3#open.

Would you like to take a stab at a pull request? Otherwise I should be able to get to it next week.

[andrewzhuk]

@janko thanks for earlier we are waiting for

[blackjiro]

Hi @janko
Do you mind if you take care of this? I am not so experienced and not sure the best way to modify this...

I tried to fix this but only come up with below code which just keep hacky way to get headers because I thought keep supporting older versions of aws-sdk-s3 might be necessary and Down::ChunkedIO receives chunks and content-length separately.
If this is fine I would like to make a pull request.

https://github.com/shrinerb/shrine/blob/master/lib/shrine/storage/s3.rb#L296

chunks         = Enumerator.new { |y| loop { y << body.next } }

↓

# just ignore additional headers object
chunks          = Enumerator.new { |y| loop { y << Array(body.next).first } }

[janko]

The commit above should fix it. I cannot yet release a new version, as Shrine master has some changes that still need to be finished, so for now you can pull from GitHub. I plan to release a new Shrine version shortly.