Strange issue with 3.1, background jobs, Heroku and HTML email attachments (#117)

[jontyD]

This seems to have re-occurred in rails 5.0.2, ruby 2.4.0
mime_types version 3.1

calling MIME::Types.of(filename) causes an error: undefined method sort' for nil:NilClass /app/vendor/bundle/ruby/2.4.0/gems/mime-types-3.1/lib/mime/types.rb:152:in type_for'
/app/vendor/bundle/ruby/2.4.0/gems/mime-types-3.1/lib/mime/types/registry.rb:32:in `type_for'

filename is some_name.pdf and is definitely present at this point

This works locally including in a local delayed job but on heroku worker in a delayed job I get this error which seems like issue 117

NB dj running as a daemon

[halostatue]

… That‘s really weird. It’s pointed out something that I didn’t know ([].inject(:+) => nil), but the result array shouldn’t be empty because the extension index should be fully populated before this code gets run. This is going to take some time to figure out, and I’m afraid that I don’t see having that time in the next two weeks.

[johnsonj]

I'm hitting this trying to update this code from mime-types 2.6.2 to 3.1 with ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-linux].

Will update if I find more info.

Stacktrace:

  1) VCAP::CloudController::BuildpacksController#delete with async false does not fail if no buildpack bits were ever uploaded
     Failure/Error: mime_type = MIME::Types.of(source_path).first.try(:content_type)
     
     NoMethodError:
       undefined method `sort' for nil:NilClass
     # /usr/local/google/home/jrjohnson/.rvm/gems/ruby-2.3.3/gems/mime-types-3.1/lib/mime/types.rb:152:in `type_for'
     # /usr/local/google/home/jrjohnson/.rvm/gems/ruby-2.3.3/gems/mime-types-3.1/lib/mime/types/registry.rb:32:in `type_for'
     # ./lib/cloud_controller/blobstore/fog/fog_client.rb:63:in `block in cp_to_blobstore'
     # ./lib/cloud_controller/blobstore/fog/fog_client.rb:59:in `open'
     # ./lib/cloud_controller/blobstore/fog/fog_client.rb:59:in `cp_to_blobstore'
     # ./lib/cloud_controller/blobstore/retryable_client.rb:48:in `block in cp_to_blobstore'
     # ./lib/cloud_controller/blobstore/retryable_client.rb:139:in `with_retries'
     # ./lib/cloud_controller/blobstore/retryable_client.rb:42:in `cp_to_blobstore'
     # ./lib/cloud_controller/blobstore/fog/error_handling_client.rb:55:in `block in cp_to_blobstore'
     # ./lib/cloud_controller/blobstore/fog/error_handling_client.rb:61:in `error_handling'
     # ./lib/cloud_controller/blobstore/fog/error_handling_client.rb:55:in `cp_to_blobstore'
     # ./spec/unit/controllers/runtime/buildpacks_controller_spec.rb:143:in `block (4 levels) in <module:CloudController>'

Repro:

1. Clone my fork

git clone https://github.com/johnsonj/cloud_controller_ng
cd cloud_controller_ng

2. Create postgres user/database:

sudo -u postgres psql -c "create user pguser with password 'pguser' superuser;"
sudo -u postgres psql -c "create database cc_test;"

3. Run spec:

DB=postgres PGPASSWORD="pguser" PGUSER="pguser" bundle exec rspec spec/unit/controllers/runtime/buildpacks_controller_spec.rb

[FX-HAO]

+1

[johnsonj]

In our case the issue isn't that @extensions_index is nil, it's that we were passing in a bad parameter that it was previously quiet about.

The calling code essentially did this:

  MIME::Types.of(Tempfile.new).first.try(:content_type)

Which is used to create an empty array inside of type_for

  a = Array(Tempfile.new) # => []
  a.flat_map { |fn| puts 'never called!'; fn } # => []
  a.compact # => []
  a.inject(:+) # => nil

So ruby learnings from this bug:

• [].inject(:+) => nil
• [Tempfile.new] != Array(Tempfile.new)

[halostatue]

This is because Array(Tempfile.new) calls Tempfile#to_a, delegated from File#to_a via Enumerable#to_a.

Sounds like I might need to go back to [arg].flatten instead of Array(arg).

[halostatue]

Resolved with edf3cd7.