Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency on mimemagic 0.3.x no longer valid #41750

Closed
pedantic-git opened this issue Mar 24, 2021 · 115 comments
Closed

Dependency on mimemagic 0.3.x no longer valid #41750

pedantic-git opened this issue Mar 24, 2021 · 115 comments

Comments

@pedantic-git
Copy link
Contributor

@pedantic-git pedantic-git commented Mar 24, 2021

The gem mimemagic (see mimemagicrb/mimemagic#97 & mimemagicrb/mimemagic#98 ) has resolved a licensing issue today by yanking all builds prior to 0.4.0, but Rails itself (activestorage) has a dependency on mimemagic (~> 0.3.2).

Would it be possible to release new patches of 6.1 (and others?) that relax this dependency to include 0.4.x?

Steps to reproduce

Attempt to install any version of Rails!

Expected behavior

Gem will install

Actual behavior

Your bundle is locked to mimemagic (0.3.5), but that version could not be found
in any of the sources listed in your Gemfile. If you haven't changed sources,
that means the author of mimemagic (0.3.5) has removed it. You'll need to update
your bundle to a version other than mimemagic (0.3.5) that hasn't been removed
in order to install.

System configuration

Rails version: 6.1.3

Ruby version: 3.0.0

@pedantic-git
Copy link
Contributor Author

@pedantic-git pedantic-git commented Mar 24, 2021

This is no longer a breaking issue because the maintainer of mimemagic has released a compatible version called 0.3.6, but the dependency probably still might need updating. mimemagicrb/mimemagic#98

@morgoth
Copy link
Member

@morgoth morgoth commented Mar 24, 2021

Also, please note that the mimemagic gem is now on GPL2 license mimemagicrb/mimemagic@c0f7b6b

@minad
Copy link

@minad minad commented Mar 24, 2021

See my comment here: mimemagicrb/mimemagic#98 (comment)

Note that the GPL-2.0 license may not be desirable under all circumstances. If there is interest in the alternative approach proposed in mimemagicrb/mimemagic#97 (reading the freedesktop.xml file at startup) and someone prepares a PR a new version could be released under the MIT license again.

@minad
Copy link

@minad minad commented Mar 24, 2021

For now I decided to archive the project. The last released versions are 0.3.6 and 0.4.0 released under the GPL-2.0.

@pedrofurtado
Copy link
Member

@pedrofurtado pedrofurtado commented Mar 24, 2021

I did not understand the root cause of yanked versions, but what is the solution for this scenario? Force everyone in the world to upgrade to 0.3.6? I will ask this in mimemagic repo, but the repo was archived (that I also not understand why was archived). Crazy day! 😆

@coding-bunny
Copy link

@coding-bunny coding-bunny commented Mar 24, 2021

How will this affect projects running older Rails versions?
E.g Commercial legacy projects etc?

@gberdugo
Copy link

@gberdugo gberdugo commented Mar 24, 2021

I did not understand the root cause of yanked versions, but what is the solution for this scenario? Force everyone in the world to upgrade to 0.3.6? I will ask this in mimemagic repo, but the repo was archived (that I also not understand why was archived). Crazy day!

I think so

How will this affect projects running older Rails versions?
E.g Commercial legacy projects etc?

I'm not a lawyer, but in my understanding, the mimemagic change to GPL licence forces rails to be distributed in GPL also, which forces all projects that are using Rails to be open-sourced

@jellybob
Copy link

@jellybob jellybob commented Mar 24, 2021

Updating to 0.4.x would be a substantial issue for many people, as it will result in a GPL2 dependency being pulled in, which a large proportion of companies have policies against using.

@coding-bunny
Copy link

@coding-bunny coding-bunny commented Mar 24, 2021

yeah GPL definitely is not an option for us.

@minad
Copy link

@minad minad commented Mar 24, 2021

The best way forward is probably to create a version of the gem which is licensed under the MIT license and loads the mime database at startup. This is the approach proposed by the maintainer of shared-mime-info, see mimemagicrb/mimemagic#97.

@nicholalexander
Copy link

@nicholalexander nicholalexander commented Mar 24, 2021

does someone want to take on or self nominate setting up some official fork and license change that rails can use? i would volunteer but not sure what the rails team might want to do with this...

@minad
Copy link

@minad minad commented Mar 24, 2021

@nicholalexander If the rails teams nominates an official maintainer, please notify me. Then we can also arrange the takeover process for the gem at rubygems.org

@jk779
Copy link

@jk779 jk779 commented Mar 24, 2021

I don't want to party poop but because 0.3.6 is already GPL-2, deploying rails right now brings everybody using a not compatible license in a sticky situation i guess...

@coding-bunny
Copy link

@coding-bunny coding-bunny commented Mar 24, 2021

I'm willing to help out/contribute, but will need some good explanation on what the possibilities are that can be done to remedy this problem correctly. Although I'd rather see the solution in the hands of the Rails team

@olafura
Copy link

@olafura olafura commented Mar 24, 2021

Another solutions is to use: https://pagure.io/mailcap/blob/master/f/mime.types like https://github.com/elixir-plug/mime

Seems to be under Public Domain from Redhat, derives from http://www.iana.org/assignments/media-types/media-types.xhtml

@erran
Copy link

@erran erran commented Mar 24, 2021

I don't want to party poop but because 0.3.6 is already GPL-2 which brings everybody using rails in a sticky situation right now i guess...

👍🏽 Since @minad is a fan of yanking gems maybe they’ll yank the GPL-2 license as a non-breaking gem version since it poisons the license for anyone using rails too as far as I can tell. /s

@minad
Copy link

@minad minad commented Mar 24, 2021

@olafura This is a good solution too.

@jellybob
Copy link

@jellybob jellybob commented Mar 24, 2021

I'm not convinced @olafura's solution is a great one. Freedesktop.org's mime types file is much more exhaustive than Red Hat's, particularly around identifying a file type via matching byte patterns within a file. Red Hat's version is solely based on file extensions.

@minad
Copy link

@minad minad commented Mar 24, 2021

@erran This is not a good proposal. Do you prefer to have no 0.3.x version available at all? The fastest way to sort this out is:

  1. Generate tables.rb from https://pagure.io/mailcap/blob/master/f/mime.types and distribute that as 0.3.7 and 0.4.1. But @jellybob's objection is true.
  2. Load the freedesktop.org.xml at startup and don't distribute tables.rb as part of the mimemagic gem. Then we can also create a 0.3.7 and 0.4.1 version under the MIT license.
@coding-bunny
Copy link

@coding-bunny coding-bunny commented Mar 24, 2021

I'm going to try and make a PR to replace the gem. Even if it's not ideal, it's a good starting point for me and can be rejected/updated by others

@jellybob
Copy link

@jellybob jellybob commented Mar 24, 2021

As I see it there are two options, both with drawbacks, so as others have said some input from the Rails team is probably needed:

Option 1: Download the freedesktop.org mime type descriptions at runtime. This probably results in a minimum of changes to mimemagic, and doesn't impact how that gem behaves. However, this then means potential issues for people deploying in tightly regulated environments as additional communication with third party servers will need to be approved - in some cases there's potential for pulling in dependencies from third parties at run time simply not being approved at all.

Option 2: Move to using Red Hat's public domain licensed mime type descriptions. Doesn't have issues with pulling in resources at run time, but it will mean a reduction in accuracy during mime type sniffing.

A third option might be to support configuring the source of the freedesktop.org mime types at run time via an environment variable or similar, which I believe in most cases would allow pulling that via the operating system's package manager at build time. This is going to be easier to get approval for in a regulated environment as it's simply installing a dependency via the same path as other dependencies. I don't have enough knowledge of software licensing to be able to say whether depending on that file causes the GPL 2 license to be inherited by anything depending on that file, but my gut feel is that it wouldn't.

@georgeclaghorn
Copy link
Member

@georgeclaghorn georgeclaghorn commented Mar 24, 2021

Using a database that doesn’t contain magic signatures isn’t a viable path. We (via Marcel) use MimeMagic specifically for this feature; if we don’t need that, we already have multiple other MIME type DBs available in Rails with varying completeness (but we do need it).

@minad
Copy link

@minad minad commented Mar 24, 2021

@coding-bunny Please notify me when you have something ready.
@jellybob Yes, that's right. Pulling/Loading at runtime is not without impact. I think it needs a bit of effort to get this right. Maybe it is best to require the file being present at startup by requiring manual installation of the file.

@matthewford
Copy link

@matthewford matthewford commented Mar 24, 2021

As I see it there are two options, both with drawbacks, so as others have said some input from the Rails team is probably needed:

Option 1: Download the freedesktop.org mime type descriptions at runtime. This probably results in a minimum of changes to mimemagic, and doesn't impact how that gem behaves. However, this then means potential issues for people deploying in tightly regulated environments as additional communication with third party servers will need to be approved - in some cases there's potential for pulling in dependencies from third parties at run time simply not being approved at all.

Option 2: Move to using Red Hat's public domain licensed mime type descriptions. Doesn't have issues with pulling in resources at run time, but it will mean a reduction in accuracy during mime type sniffing.

You could take a similar approach to faraday, where there is a common interface but two implementations

@sergey-alekseev
Copy link
Contributor

@sergey-alekseev sergey-alekseev commented Mar 24, 2021

Not only Rails depends on mimemagic. The gem is used by 576,971 Repositories and 172 Packages on Github ATM. It would be more correct for @minad to unarchive https://github.com/minad/mimemagic and move this conversation there IMO.

@minad
Copy link

@minad minad commented Mar 24, 2021

@sergey-alekseev I disagree. The Rails dependency is certainly the most impactful one. It is best if we find a solution which works for Rails and which is sanctioned by the Rails team.

@coding-bunny
Copy link

@coding-bunny coding-bunny commented Mar 24, 2021

I've started here : #41751

Lloyd-K pushed a commit to DFE-Digital/get-help-with-tech that referenced this issue Mar 31, 2021
- This is to deal with licensing issues
- Further discussion can be found at rails/rails#41750
akofink added a commit to akofink/compliance-backend that referenced this issue Apr 1, 2021
See rails/rails#41750

Signed-off-by: Andrew Kofink <akofink@redhat.com>
longnd added a commit to nimblehq/rails-templates that referenced this issue Apr 2, 2021
To avoid the issue of failed build because of yanked dependencty (mimemagic 0.3.x)
rails/rails#41750
https://github.com/rails/rails/releases/tag/v6.1.3.1
https://github.com/rails/marcel/releases/tag/v1.0.0
james-wills-sf added a commit to SFDigitalServices/sf-dahlia-lap that referenced this issue Apr 5, 2021
Webapp ticket: [DAH-751]
Partners ticket: [DAH-752]

Webapp currently relies on mimemagic 3.5, which has been removed from the gem repository for licensing reasons. Rails 5.2.5 removes the mimemagic dependency.

See the [rails release](https://weblog.rubyonrails.org/2021/3/26/marcel-upgrade-releases/) and these issues [new mimemagic version released under MIT](rails/rails#41757), [dependency on mimemagic no longer valid](rails/rails#41750) for more info
microstudi added a commit to Som-Energia/decidim-som-energia-app that referenced this issue Apr 6, 2021
* Update mimemagic

rails/rails#41750 (comment)

* Duplicacio processos

* Smol refactor

* Add docs

* add homepage spec

* add configuration docs

* set proper locales

* fix lint md

Co-authored-by: Ivan Vergés <ivan@platoniq.net>
nasirkhanbeis pushed a commit to UKGovernmentBEIS/beis-opss-cosmetics that referenced this issue Apr 6, 2021
Due to some licensing issue[1], a gem version that Rails depends on was
yanked, breaking Rails installation.
This is currently blocking our deployment pipeline.

Mimemagic released a new 0.3.6 version[2] so we can manually enforce it
until a new patch on Rails fixing the dependency is released.

- [1]: rails/rails#41750
- [2]: mimemagicrb/mimemagic#98 (comment)
proctorj pushed a commit to InformedSolutions/JAQU-CAZ-Vehicle-Compliance-Checker-Web that referenced this issue Apr 6, 2021
Old version of mimemagic has been removed
rails/rails#41750
This commit bumps the version up to the
closest version possible.

The only change introduced in 0.3.6 is license
change (from MIT to GPL2)

Co-authored-by: Krzysztof Szromek <k.szromek@exlabs.co.uk>
ayuki-joto pushed a commit to codeforjapan/decidim-cfj that referenced this issue Apr 7, 2021
glacials added a commit to glacials/splits-io that referenced this issue Apr 8, 2021
lentschi added a commit to lentschi/foodsoft that referenced this issue Apr 9, 2021
This is the result of running `bundle update --conservative mimemagic`
as suggested by rails/rails#41750 (comment)
barnslig added a commit to barnslig/voctoweb that referenced this issue Apr 9, 2021
Rails 5.2.0 won't install anymore because it's dependency on mimemagic
0.3.5 is no longer available. Rails 5.2.5 fixes this problem.

See:
- rails/rails#41750
- https://weblog.rubyonrails.org/2021/3/26/marcel-upgrade-releases/
wvengen pushed a commit to foodcoops/foodsoft that referenced this issue Apr 9, 2021
This is the result of running `bundle update --conservative mimemagic`
as suggested by rails/rails#41750 (comment)
@samcdavid
Copy link

@samcdavid samcdavid commented Apr 12, 2021

For anyone else that runs into this error when trying to build their project, I updated my Rails 5 project to 5.2.5 and this resolved the issue and mimemagic is no longer in the Gemfile.lock.

I personally have not tried this for Rails 6, but I suspect that if you are on Rails 6 and upgrade to 6.1.3.1 that this will also resolve your build issues and remove mimemagic from your lock file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet