Mimemagic Rails dependency Licensing drama and next steps

[PhilCoggins]

The Rails community is reeling from a licensing issue from the gem mimemagic, which is required for a regular Rails gem install (it is a transitive dependency of ActiveStorage).

As of this morning, the v0.3.7 of the gem is available with all prior versions yanked. This new version of the gem requires freedesktop.org.xml, which looks in these locations. If the file is unavailable from these locations, an ENV variable can be provided to point to the correct location.

This file is not available on 2.7.2-slim-buster image, I have not checked other images, but given the fact that this file is pulled from a library that helps with desktop development, I imagine it would not be available on most Docker images. In order to satisfy this dependency, the new maintainer recommends adding the apt source, installing the entire shared-mime-info library, decompressing with 7zip CLI, moving the file to a suitable location, and setting proper ENV to point to the correct location.

Would it be prudent to make this file available in your base Docker images as a convenience to reduce headaches? The slim-buster images are extremely popular for Rails developers, and I'm sure DevOps engineers around the world are scrambling to update their Dockerfiles, which will now require an additional 3-4 steps (add sources, install shared-mime-info, install 7zip CLI, move the file, set ENV, cleanup) in their base step alone. It seems this would be very helpful for Rails developers, who likely represent a large portion of this image's users.

[tianon]

The dependencies on https://packages.debian.org/buster/shared-mime-info are pretty light -- any idea why it's recommended to pull from the source instead of just installing that package?

To be clear, we do not plan to add this to the slim images; it likely already exists in the non-slim images (and if it doesn't, it's a great candidate for adding to https://hub.docker.com/_/buildpack-deps, which is what the non-slim images are FROM). However, I think this issue can serve as a great reference point for users who are newly grappling with this problem.

Edit: looks like buildpack-deps did not have this package, so I've put up a PR at docker-library/buildpack-deps#120

Edit x2: however, ruby:latest does have it already 😄

[tianon]

After this operation, 43.1 MB of additional disk space will be used.

... I guess this is why. 😅

(However, as I noted in my edit above, buildpack-deps did not have this package, for which I've opened a PR, and ruby:latest already does include this package.)

[PhilCoggins]

@tianon Thank you for the quick response and turnaround!

If the package isn't suitable to add to the slim images, I'd like to lobby to at least get freedesktop.com.xml added to the slim images in the lookup location that mimemagic uses. Without this file, Rails DevOps engineers will be collectively adding 10s of thousands of identical, redundant lines to their Dockerfiles. The footprint of the file is heavy (2.3MB), which I understand probably isn't ideal for images intended to be "slim", but again, considering the popularity of this image among Rails developers, I think it's worth a second consideration.

Thanks again!

[PhilCoggins]

For future Rails Docker devs, here's what I've added to my Dockerfile to pull in freedesktop.com.xml, the first step is common to grab essentials that get a Rails app running, just ensure you add shared-mime-info. The second step copies the required file freedesktop.org.xml to current directory, removes shared-mime-info, purging any transitive dependencies, and copies it back to its original location, which mimemagic will find, without adding any ENV variables:

# Common dependencies
RUN apt-get update -qq \
  && apt-get install -yq --no-install-recommends \
    build-essential \
    curl \
    git \
    gnupg2 \
    less \
    shared-mime-info \
  && apt-get clean \
  && rm -rf /var/cache/apt/archives/* \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
  && truncate -s 0 /var/log/*log /var/log/apt/*log

RUN cp /usr/share/mime/packages/freedesktop.org.xml ./ \
  && apt-get remove -y --purge shared-mime-info \
  && mkdir -p /usr/share/mime/packages/ \
  && cp ./freedesktop.org.xml /usr/share/mime/packages/

There might be better ways to do this, but this is quick and only requires one additional step in my Dockerfile. I'm a novice Debian and Docker developer, so feel free to suggest improvements or optimizations.

I really hope we can see this file make it to the ruby-slim images to not require this step, but I'm also willing to accept the stringent standards that help keep those amazing images as small as possible.

[tianon]

Here's a minimal (and also straightforward) solution I've come up with, which results in a ~3.45MB layer (which seems pretty reasonable for a ~2.2MB file):

RUN set -eux; \
	apt-get update; \
	apt-get install -y --no-install-recommends shared-mime-info; \
	mkdir -pv /usr/local/share/mime/packages; \
	cp -v /usr/share/mime/packages/freedesktop.org.xml /usr/local/share/mime/packages/; \
	apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false shared-mime-info; \
	rm -rf /var/lib/apt/lists/*

This also puts the result into /usr/local/share/mime/packages, which is both not going to be clobbered by the package manager accidentally, and is in the list of directories which mimemagic will search for it in.

If you can tolerate a multi-image dependency, you could also trivially copy the file from the non-slim image at build:

COPY --from=ruby:3.0.0-buster /usr/share/mime/packages/freedesktop.org.xml /usr/local/share/mime/packages/

(Which then results in a ~2.28MB layer containing only the single file.)

Also, to be explicitly clear, we do not plan to add this to the slim Ruby images unless it is required for Ruby itself.

[jouve]

RUN \
 	apt-get download shared-mime-info; \
	dpkg-deb --fsys-tarfile shared-mime-info_*.deb | tar -C /usr/local -x --strip 2 ./usr/share/mime/packages/freedesktop.org.xml; \
	rm shared-mime-info_*.deb; \

[tianon]

This is as fixed as it's going to be from this image (closing accordingly).