Dependency on mimemagic 0.3.x no longer valid

[pedantic-git]

The gem mimemagic (see mimemagicrb/mimemagic#97 & mimemagicrb/mimemagic#98 ) has resolved a licensing issue today by yanking all builds prior to 0.4.0, but Rails itself (activestorage) has a dependency on mimemagic (~> 0.3.2).

Would it be possible to release new patches of 6.1 (and others?) that relax this dependency to include 0.4.x?

Steps to reproduce

Attempt to install any version of Rails!

Expected behavior

Gem will install

Actual behavior

Your bundle is locked to mimemagic (0.3.5), but that version could not be found
in any of the sources listed in your Gemfile. If you haven't changed sources,
that means the author of mimemagic (0.3.5) has removed it. You'll need to update
your bundle to a version other than mimemagic (0.3.5) that hasn't been removed
in order to install.

System configuration

Rails version: 6.1.3

Ruby version: 3.0.0

[pedantic-git]

This is no longer a breaking issue because the maintainer of mimemagic has released a compatible version called 0.3.6, but the dependency probably still might need updating. mimemagicrb/mimemagic#98

[morgoth]

Also, please note that the mimemagic gem is now on GPL2 license mimemagicrb/mimemagic@c0f7b6b

[minad]

See my comment here: mimemagicrb/mimemagic#98 (comment)

Note that the GPL-2.0 license may not be desirable under all circumstances. If there is interest in the alternative approach proposed in mimemagicrb/mimemagic#97 (reading the freedesktop.xml file at startup) and someone prepares a PR a new version could be released under the MIT license again.

[minad]

For now I decided to archive the project. The last released versions are 0.3.6 and 0.4.0 released under the GPL-2.0.

[pedrofurtado]

I did not understand the root cause of yanked versions, but what is the solution for this scenario? Force everyone in the world to upgrade to 0.3.6? I will ask this in mimemagic repo, but the repo was archived (that I also not understand why was archived). Crazy day! 😆

[coding-bunny]

How will this affect projects running older Rails versions?
E.g Commercial legacy projects etc?

[gberdugo]

I did not understand the root cause of yanked versions, but what is the solution for this scenario? Force everyone in the world to upgrade to 0.3.6? I will ask this in mimemagic repo, but the repo was archived (that I also not understand why was archived). Crazy day!

I think so

How will this affect projects running older Rails versions?
E.g Commercial legacy projects etc?

I'm not a lawyer, but in my understanding, the mimemagic change to GPL licence forces rails to be distributed in GPL also, which forces all projects that are using Rails to be open-sourced

[jellybob]

Updating to 0.4.x would be a substantial issue for many people, as it will result in a GPL2 dependency being pulled in, which a large proportion of companies have policies against using.

[coding-bunny]

yeah GPL definitely is not an option for us.

[minad]

The best way forward is probably to create a version of the gem which is licensed under the MIT license and loads the mime database at startup. This is the approach proposed by the maintainer of shared-mime-info, see mimemagicrb/mimemagic#97.

[nicholalexander]

does someone want to take on or self nominate setting up some official fork and license change that rails can use? i would volunteer but not sure what the rails team might want to do with this...

[minad]

@nicholalexander If the rails teams nominates an official maintainer, please notify me. Then we can also arrange the takeover process for the gem at rubygems.org

[jk779]

I don't want to party poop but because 0.3.6 is already GPL-2, deploying rails right now brings everybody using a not compatible license in a sticky situation i guess...

[coding-bunny]

I'm willing to help out/contribute, but will need some good explanation on what the possibilities are that can be done to remedy this problem correctly. Although I'd rather see the solution in the hands of the Rails team

[olafura]

Another solutions is to use: https://pagure.io/mailcap/blob/master/f/mime.types like https://github.com/elixir-plug/mime

Seems to be under Public Domain from Redhat, derives from http://www.iana.org/assignments/media-types/media-types.xhtml

[erran]

I don't want to party poop but because 0.3.6 is already GPL-2 which brings everybody using rails in a sticky situation right now i guess...

👍🏽 Since @minad is a fan of yanking gems maybe they’ll yank the GPL-2 license as a non-breaking gem version since it poisons the license for anyone using rails too as far as I can tell. /s

[minad]

@olafura This is a good solution too.

[jellybob]

I'm not convinced @olafura's solution is a great one. Freedesktop.org's mime types file is much more exhaustive than Red Hat's, particularly around identifying a file type via matching byte patterns within a file. Red Hat's version is solely based on file extensions.

[minad]

@erran This is not a good proposal. Do you prefer to have no 0.3.x version available at all? The fastest way to sort this out is:

1. Generate tables.rb from https://pagure.io/mailcap/blob/master/f/mime.types and distribute that as 0.3.7 and 0.4.1. But @jellybob's objection is true.
2. Load the freedesktop.org.xml at startup and don't distribute tables.rb as part of the mimemagic gem. Then we can also create a 0.3.7 and 0.4.1 version under the MIT license.

[coding-bunny]

I'm going to try and make a PR to replace the gem. Even if it's not ideal, it's a good starting point for me and can be rejected/updated by others

[jellybob]

As I see it there are two options, both with drawbacks, so as others have said some input from the Rails team is probably needed:

Option 1: Download the freedesktop.org mime type descriptions at runtime. This probably results in a minimum of changes to mimemagic, and doesn't impact how that gem behaves. However, this then means potential issues for people deploying in tightly regulated environments as additional communication with third party servers will need to be approved - in some cases there's potential for pulling in dependencies from third parties at run time simply not being approved at all.

Option 2: Move to using Red Hat's public domain licensed mime type descriptions. Doesn't have issues with pulling in resources at run time, but it will mean a reduction in accuracy during mime type sniffing.

A third option might be to support configuring the source of the freedesktop.org mime types at run time via an environment variable or similar, which I believe in most cases would allow pulling that via the operating system's package manager at build time. This is going to be easier to get approval for in a regulated environment as it's simply installing a dependency via the same path as other dependencies. I don't have enough knowledge of software licensing to be able to say whether depending on that file causes the GPL 2 license to be inherited by anything depending on that file, but my gut feel is that it wouldn't.

[georgeclaghorn]

Using a database that doesn’t contain magic signatures isn’t a viable path. We (via Marcel) use MimeMagic specifically for this feature; if we don’t need that, we already have multiple other MIME type DBs available in Rails with varying completeness (but we do need it).

[minad]

@coding-bunny Please notify me when you have something ready.
@jellybob Yes, that's right. Pulling/Loading at runtime is not without impact. I think it needs a bit of effort to get this right. Maybe it is best to require the file being present at startup by requiring manual installation of the file.

[matthewford]

As I see it there are two options, both with drawbacks, so as others have said some input from the Rails team is probably needed:

Option 1: Download the freedesktop.org mime type descriptions at runtime. This probably results in a minimum of changes to mimemagic, and doesn't impact how that gem behaves. However, this then means potential issues for people deploying in tightly regulated environments as additional communication with third party servers will need to be approved - in some cases there's potential for pulling in dependencies from third parties at run time simply not being approved at all.

Option 2: Move to using Red Hat's public domain licensed mime type descriptions. Doesn't have issues with pulling in resources at run time, but it will mean a reduction in accuracy during mime type sniffing.

You could take a similar approach to faraday, where there is a common interface but two implementations

[sergey-alekseev]

Not only Rails depends on mimemagic. The gem is used by 576,971 Repositories and 172 Packages on Github ATM. It would be more correct for @minad to unarchive https://github.com/minad/mimemagic and move this conversation there IMO.

[minad]

@sergey-alekseev I disagree. The Rails dependency is certainly the most impactful one. It is best if we find a solution which works for Rails and which is sanctioned by the Rails team.

[coding-bunny]

I've started here : #41751

[samcdavid]

For anyone else that runs into this error when trying to build their project, I updated my Rails 5 project to 5.2.5 and this resolved the issue and mimemagic is no longer in the Gemfile.lock.

I personally have not tried this for Rails 6, but I suspect that if you are on Rails 6 and upgrade to 6.1.3.1 that this will also resolve your build issues and remove mimemagic from your lock file.