Skip to content

Commit

Permalink
Use MiniMime for MIME type lookup by filename/extension
Browse files Browse the repository at this point in the history
Co-authored-by: John Hawthorn <john@hawthorn.email>
  • Loading branch information
georgeclaghorn and jhawthorn committed Mar 24, 2021
1 parent cce58f2 commit 5040f48
Show file tree
Hide file tree
Showing 6 changed files with 35 additions and 7 deletions.
6 changes: 4 additions & 2 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@ PATH
specs:
marcel (0.3.3)
mimemagic (~> 0.3.2)

This comment has been minimized.

Copy link
@AFU92

AFU92 Mar 24, 2021

Should we update it?

This comment has been minimized.

Copy link
@mcdave029

mcdave029 Mar 25, 2021

@georgeclaghorn @jhawthorn follow up updates for this mimemagic regarding this rails/rails#41750. Thank you

This comment has been minimized.

Copy link
@ssaunier

ssaunier Mar 25, 2021

@AFU92 I guess this dependency will be removed as soon as the mini_mime replacement has been tested by Basecamp / GitHub / etc. internal test suites 🤔

mini_mime (~> 1.0.2)

GEM
remote: https://rubygems.org/
specs:
mimemagic (0.3.2)
mimemagic (0.3.6)

This comment has been minimized.

Copy link
@rvanbuijtenen

rvanbuijtenen Mar 25, 2021

Please keep in mind that mimemagic 0.3.6 is still under the GPL-2 licenses, and is therefore not allowed to be used in commercial applications.

This comment has been minimized.

Copy link
@q3k

q3k Mar 25, 2021

GPL 2 does not prevent you from using your product in commercial applications.

This comment has been minimized.

Copy link
@rvanbuijtenen

rvanbuijtenen Mar 25, 2021

My apologies, I meant to say closed-source applications (which still contain a large number of commercial applications). This is indeed not the case for all uses of software under GPL-2, but since mimemagic 0.3.6 includes a modified copy of a GPL-2 licensed project, including it would still lead to a license violation for those applications. See https://news.ycombinator.com/item?id=26566925 for a discussion on the legal implications.

There is a fork that redistributes it under an MIT licence: https://github.com/jellybob/mimemagic,. However, this version requires the installation of the Freedesktop.org shared-mime-info database used by mimemagic separately, in order to be compliant with the license.

I must admit, it's not a favourable fix, but the only one available right now (as far as I'm aware). I hope the author of the original mimemagic dependency releases a version with an appropriate fix soon

This comment has been minimized.

Copy link
@rvanbuijtenen

rvanbuijtenen Mar 25, 2021

Anyway, if according the above comment by @ssaunier mimemagic is replaced entirely by mini_mime, then that's a lot easier and (in my opinion) a better fix

This comment has been minimized.

Copy link
@loadkpi

loadkpi Mar 25, 2021

0.3.6 is yanked now

This comment has been minimized.

Copy link
@rvanbuijtenen

rvanbuijtenen Mar 25, 2021

yes indeed, and so has 4.0

Instead, the fork under MIT that I mentioned has now been released as the one and only v 0.4.1
On debian-based systems, simply run apt install shared-mime-info and then install mimemagic 0.4.1 and it works (for our docker-based builds at least). Allthough I can imagine it is still more desirable to use a dependency for mimetypes that does not require an external dependency.

This comment has been minimized.

Copy link
@CarlosCD

CarlosCD Mar 25, 2021

It seems to be a 0.3.8 pushed today (March 25): https://rubygems.org/gems/mimemagic/versions

This comment has been minimized.

Copy link
@masha256

masha256 Mar 25, 2021

FYI - https://github.com/mimemagicrb/mimemagic is recommending brew install shared-mime-info for macOS users.

This comment has been minimized.

Copy link
@izzergh
mini_mime (1.0.2)
minitest (5.11.3)
rack (2.1.4)
rake (13.0.1)
Expand All @@ -23,4 +25,4 @@ DEPENDENCIES
rake (~> 13.0)

BUNDLED WITH
1.16.4
1.17.2
26 changes: 21 additions & 5 deletions lib/marcel/mime_type.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,15 @@
require "mini_mime"

class Marcel::MimeType
BINARY = "application/octet-stream"

@ext_overrides = {}

OverrideInfo = Struct.new(:content_type)

class << self
attr_reader :ext_overrides

def extend(type, extensions: [], parents: [], magic: nil)
existing = MimeMagic::TYPES[type] || [[], [], ""]

Expand All @@ -10,6 +18,12 @@ def extend(type, extensions: [], parents: [], magic: nil)
comment = existing[2]

MimeMagic.add(type, extensions: extensions, magic: magic, parents: parents, comment: comment)

info = MiniMime.lookup_by_content_type(type)
info ||= OverrideInfo.new(type)
extensions.each do |ext|
@ext_overrides[ext] = info
end
end

def for(pathname_or_io = nil, name: nil, extension: nil, declared_type: nil)
Expand All @@ -36,16 +50,18 @@ def for_data(pathname_or_io)

def for_name(name)
if name
if magic = MimeMagic.by_path(name)
magic.type.downcase
end
extension = File.extname(name)
return if extension.empty?
for_extension(extension)
end
end

def for_extension(extension)
if extension
if magic = MimeMagic.by_extension(extension)
magic.type.downcase
extension = extension.gsub(/\A\./, "").downcase

if info = (Marcel::MimeType.ext_overrides[extension] || MiniMime.lookup_by_extension(extension))
info.content_type.downcase
end
end
end
Expand Down
9 changes: 9 additions & 0 deletions lib/marcel/mime_type/definitions.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,12 @@

Marcel::MimeType.extend "image/heif", magic: [[4, "ftypmif1"]], extensions: %w( heif )
Marcel::MimeType.extend "image/heic", magic: [[4, "ftypheic"]], extensions: %w( heic )

# Backwards compat overrides from MiniMime migration
Marcel::MimeType.extend "text/x-log", extensions: %w( log )
Marcel::MimeType.extend "application/sql", extensions: %w( sql )
Marcel::MimeType.extend "audio/flac", extensions: %w( flac )
Marcel::MimeType.extend "image/x-eps", extensions: %w( eps )
Marcel::MimeType.extend "video/mp4", extensions: %w( mp4 )
Marcel::MimeType.extend "image/bmp", extensions: %w( bmp )
Marcel::MimeType.extend "application/vnd.adobe.flash.movie", extensions: %w( swf )
1 change: 1 addition & 0 deletions marcel.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Gem::Specification.new do |spec|
spec.required_ruby_version = '>= 2.2'

spec.add_dependency 'mimemagic', '~> 0.3.2'
spec.add_dependency 'mini_mime', '~> 1.0.2'

spec.add_development_dependency 'minitest', '~> 5.11'
spec.add_development_dependency 'bundler', '>= 1.7'
Expand Down
File renamed without changes.
File renamed without changes.

0 comments on commit 5040f48

Please sign in to comment.