ActiveStorage variant raises Errno::ENAMETOOLONG

[marcusg]

Steps to reproduce

# storage.yml
local:
  service: Disk
  root: <%= Rails.root.join("storage") %>

# user.rb
class User < ApplicationRecord
  has_one_attached :avatar
end

f = File.open(Rails.root.join("spec", "assets", "png.png"))
u = User.first
attachment = u.avatar.attach io: f, filename: "my avatar", content_type: "image/png"
variant = attachment.variant thumbnail: "100x100>", background: :white, gravity: :center, extent: "100x100"
variant.processed.service_url
# => Errno::ENAMETOOLONG: File name too long @ rb_sysopen - /home/myname/Work/repos/github/project/storage/va/ri/variants/XYnB7EN9Yq6agpMwzmQQJ8HP/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaDdDVG9PZEdoMWJXSnVZV2xzU1NJTk1UQXdlREV3TUQ0R09nWkZWRG9QWW1GamEyZHliM1Z1WkRvS2QyaHBkR1U2REdkeVlYWnBkSGs2QzJObGJuUmxjam9MWlhoMFpXNTBTU0lNTVRBd2VERXdNQVk3QmxRPSIsImV4cCI6bnVsbCwicHVyIjoidmFyaWF0aW9uIn19--b8a505ddd21c7236e0f88d343c26ae1d17bcfbd5
	from (irb):17

Expected behavior

no Errno::ENAMETOOLONG exception

Actual behavior

Errno::ENAMETOOLONG exception is raised because the file name is too long

System configuration

Rails version:
master (07bac9e)

Ruby version:
2.3.5

OS
Ubuntu 16.04

[marcusg]

Testing results: using a smaller set of options passed to attachment.variant will not raise an exception. So it looks like the hash passed to #variant grows and reaches the max file name length.

[marcusg]

Here is a failing test https://github.com/hausgold/rails/commit/1ca248bb016786c12eb17db335865097403abb9c

[khall]

Not sure what the best approach here is. The filesystem has a maximum limit for filenames, we're encoding the passed data via Base64.strict_encode64, which raises the mentioned exception when it exceeds the filesystem limit. More data passed into the variant = more to encode = the limit is exceeded.

Ideas I have:

• Drop some variant data: But if we drop passed variant data, won't filenames will collide?
• Use a more concise encoding: But eventually we will run into this problem once enough data is passed.
• Figure out if Errno::ENAMETOOLONG would be raised, and raise our own exception limiting the amount of variant data that can be passed. This seems to be the only worthwhile idea (aside from doing nothing).

[marcusg]

@khall Do you think splitting the variant#key into subdirectories can help?

  # variant model
  def key
    "variants/#{blob.key}/#{variation.key.scan(/.{1,255}/).join("/")}"
  end

[khall]

That's a good idea, and that'd certainly help until the 1024 character path limit is reached. We'd have to reassemble the parts of the key prior to calling Base64::strict_decode64, but that seems doable.

[Jack12816]

@khall I was wondering where you get the 1024 path limit from? From file system perspective it's not limited (ext2-4, btrfs, xfs) in nesting. They all share the same 255 characters filename limit. I did not found a concrete string length limit for ruby.

[khall]

@Jack12816 I noticed it here https://github.com/rails/rails/blob/master/activesupport/lib/active_support/cache/file_store.rb#L20 and assumed it was accurate. There's references to a maximum path length in ruby as well, but there isn't one single answer, I've seen 1024, 4096, and 8192. Here are a couple links to them:

https://github.com/ruby/ruby/search?utf8=%E2%9C%93&q=PATH_MAX&type=
https://github.com/ruby/ruby/search?utf8=%E2%9C%93&q=MAXPATHLEN&type=