Replace Imagick with something better

[enoch85]

EDIT (SEO): The PHP module "imagick" is not enabled although the theming app is. For favicon generation to work correctly, you need to install and enable this module.

A few days ago it was brought up to my attention that using Imagick could have very negative effects on security. The Nextcloud snap decided to not using it due to that fact, and I've now mitigated the same threat(s) as well by not using it in the Nextcloud VM.

Here are the discussion regarding the decision in the Nextcloud snap, and I think it totally makes sense not to use it in the Nextcloud Server as well.

The situation now though is that it's recomended and the setup checks will inform the user that the package is missing. As Nextcloud is advertising it's secure, then why use a package that is prune to a lot of CVEs in the past?

Regarding alternatives I think this post sums it up quite well.

Please consider removing the recommendation in future versions, and please also consider replacing the use of Imagick with something better and more secure.

EDIT 2: We now install Imaginary as a replacement for this in the Nextcloud VM.

[kesselb]

Ref #12821

I see the concerns and could imagine to move the warning into the theming app that some feature are not working because imagick is not present.

@skjnldsv @juliushaertl looks like @nextcloud/vm and @nextcloud/snap not going to add imagick. @nextcloud/docker added imagick a few days ago.

Edit: There is already a warning that some things does not work:

server/apps/theming/templates/settings-admin.php

Line 136 in 3068f07

<?php if (!$_['canThemeIcons']) { ?>

[skjnldsv]

Please consider removing the recommendation in future versions, and please also consider replacing the use of Imagick with something better and more secure.

Nothing allow us to properly convert images types (especially svg) with php other than imagick (that I'm aware of) unfortunately.

[kesselb]

Apart from this favicon svg generation the theming app works fine without imagick?

[skjnldsv]

@danielkesselberg avatars will be not as great looking without imagick.
Previews too :)

[J0WI]

How about the performance without imagick?
This does also affect the gallery app: https://github.com/nextcloud/gallery#supporting-more-media-types

[cyphar]

From what I've read, most of the ImageMagick CVEs come from individual filters or filetypes that we probably don't care about -- is there a way to whitelist policy.xml so that we only allow the few formats that we want to support? In addition, checking that the magic header of files is actually correct (for the formats we want to support) would protect against most malicious files.

ImageMagick (and GraphicsMagick to a lesser extent) have had quite a large number of CVEs, mostly due to the sheer amount of formats and features that users need to process images. I would say a good first step would be to figure out a whitelist hardening configuration that we can use across the board, and then we can evaluate switching away from ImageMagick if that's not sufficient.

[J0WI]

See also Imagick/imagick#262

[cyphar]

There are also a couple of ways we could restrict ImageMagick through seccomp or by putting it inside an empty network namespace (and a mount namespace which has everything mounted-over except /lib64 and the file that is being accessed -- though this will require a bit of work since it requires having some sort of unveil feature).

[enoch85]

@skjnldsv So something like https://github.com/flyimg/flyimg wouldn't work?

[skjnldsv]

@enoch85 that would require shell_exec. Yes, we could rely on external software (like inkscape for example). But this is not really recommended to do on php. @rullzer ?

EDIT: sorry, I thought it was another software. Yes, we can use an external docker as well. We actually have a PoC somewhere for that. But this is a really heavy dependency and this would not scale to every setup. Also, most people don't use docker :/

[kyrofa]

Thanks for this, @enoch85. It's probably no surprise that I completely agree on this issue. In the snap it's not even possible for people to use it, so folks will just see the warning forever and be unable to do anything. So at the very least, packagers should be able to disable this warning without triggering an integrity failure. Even better, Nextcloud should just stop suggesting it be installed if it's not. If one doesn't miss the functionality it provides, all it does is make the general populous less secure. Best yet: find an alternative so everyone can enjoy the functionality without trading security for it.

[skjnldsv]

Best yet: find an alternative so everyone can enjoy the functionality without trading security for it.

Let's be clear here, we all agree 😝
Having another php lib that could do the job would be ideal, but I don't have anything else to suggest unfortunately. I'm open to suggestion though :)

[kyrofa]

I'm glad we agree on that, but I'm also realistic: I don't know of an alternative off the top of my head either. While we look for one, can Nextcloud please stop complaining if imagick is not installed?

[bpcurse]

How about a temporary solution that shows this special warning as one that can be confirmed as read and then be permanently hidden?

[ghost]

Is it just SVG that GD didn't support? How about https://github.com/meyfa/php-svg?

[J0WI]

I'm not sure if a less common extensions with only few/one maintainers more secure (and more reliable for e.g. compatibility) than ImageMagick.

There are indeed more filetypes, but:

Technically Nextcloud can also generate the previews of other file types such as PDF, SVG or various office documents. Due to security concerns those providers have been disabled by default and are considered unsupported. While those providers are still available, we discourage enabling them, and they are not documented.

https://docs.nextcloud.com/server/stable/admin_manual/configuration_files/previews_configuration.html
https://github.com/nextcloud/gallery#supporting-more-media-types

[LEDfan]

@enoch85

So something like https://github.com/flyimg/flyimg wouldn't work?

It seems like that's more like an API/microservice to manipulate images, but under the hood it's also using ImageMagick https://github.com/flyimg/flyimg#technology-stack:

Image manipulation: ImageMagick

[juliusknorr]

I'm glad we agree on that, but I'm also realistic: I don't know of an alternative off the top of my head either. While we look for one, can Nextcloud please stop complaining if imagick is not installed?

It is not a hard error message, but a warning, so I see no reason why, we should hide this. It just informs admins that they are missing a dependency that might cause some features to not work properly. Regarding the security concerns, if previews for those files are not enabled we don't pass user provided files to imagemagick as far as I know. The theming app is limited to admins, so there is no attack vector here, since the admin is considered as trusted anyway.

[kyrofa]

As soon as imagick is available it can be used by any application, no? That's the big problem here. It's easy to use incorrectly and it's easy to install third-party apps that do so.

[ghost]

How about GraphicsMagick, a fork of IM. Does it have the same security problems?

http://www.graphicsmagick.org/

From their website:
Here are some reasons to prefer GraphicsMagick over ImageMagick:

GM is more efficient so it gets the job done faster using fewer resources.
GM is much smaller and lighter (3-5X smaller installation footprint).
GM is used to process billions of files at the world's largest photo sites (e.g. Flickr and Etsy).
GM does not conflict with other installed software.
GM suffers from fewer security issues and exploits.

[juliusknorr]

GraphicsMagick seems to be API compatible to imagemagick, so it could be a drop in replacement in any setup as far as I can tell.

[kesselb]

https://pecl.php.net/package/gmagick there is no stable release 😞

[J0WI]

The imagemagick/GraphicsMagick binary is only required for extended file types like SVG:
nextcloud/docker#594 (comment)
Is php-imagick also that worse?

[gessel]

The ffmpeg problem is, fortunately, patched now for FreeBSD. Hopefully will be committed soon, but for now:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256215

[szaimen]

Looks to me like graphicsmagick could be a good replacement to imagemagick in the meantime.
Would also probably prevent many security issues but is probably a lot of work to be implemented.

[szaimen]

cc @nextcloud/server @nextcloud/security

[J0WI]

Anyone familiar with libvips? The benchmarks and reviews look great.

[tianon]

My experience with it was in the context of Ghost via "sharp" requiring too new of a libvips (Debian Buster has 8.7 and they required 8.9+) which caused a host of issues for getting it successfully installed, so I'd suggest surveying what version of libvips is available in the expected target environments and ensuring the lowest common denominator meets the needs of the project before committing (but that's just my 2c; no real stake here 😇).

[enoch85]

I still think this is an ongoing discussion though.

Mainly this issue exist due to the security concerns, and whatever replacing Imagick needs to be better OR not produce a warning.

[PVince81]

also note: some research done with an external preview generator: #24166
(note: I don't have time right now to continue this, feel free to take over)

[adripo]

While someone will continue developing this feature, what do you think if we proceed by marking the warning as INFO in the Administration Overview as suggested by @kerberizer in nextcloud/docker/1414#issuecomment-945842317?

[solracsf]

Closing as per #24166

[Fuseteam]

oh man, that's awesome