Improve libvips support for Heroku?

[mokolabs]

I'm ultimately planning to deploy my new image_processing app to Heroku.

Since libvips isn't included in the default package list on Heroku stacks, I will need to install a custom buildpack before I deploy my app.

There are quite a few libvips buildpacks on Heroku already:
https://elements.heroku.com/search/buildpacks?q=libvips

But I'm not sure which is best. Plus they all seem outdated. The most popular uses version 7.42.3, but that's over five years old. The newest buildpack uses 8.0.0, but that's almost 3 years old.

I'd like to see...

• An updated buildpack with a current version of libvips
• A new "How To" guide on the wiki with complete steps for installing the buildpack on Heroku

Why do this? We should make it super easy to use libvips with image_processing on Heroku. Right now, our story is "libvips support is amazing, but it's kind of a pain to deploy to Heroku". If we create a new buildpack, the story will be "libvips is amazing and it's easy to deploy to Heroku using our custom buildpack".

The buildpack, of course, should be its own Github project. I'm happy to host it, but I'm certainly not an expert on Ubuntu package management and library compilation, so I'd love some help from others who may know more about that stuff than me. (I can also help with testing and documentation).

Thoughts?

[mokolabs]

This seems like the most recently updated buildpack:
https://github.com/kespry/heroku-buildpack-vips

The README says it's using libvips version 8.0.0, but a month ago, there was a commit that bumped it to version 8.6.2 (but they forgot to update the README).

The latest official libvips release is 8.6.3, so that's basically current.

[janko]

@mokolabs I agree, we should definitely do a writeup on that. Though I'm wondering whether a better place for Heroku instructions would be the ruby-vips wiki (where we would link to from here).

[mokolabs]

@janko-m Sweet. Yeah, it definitely makes more sense that the guide lives over in that repo.

For starters, I forked the most recently updated buildpack, then renamed it and updated the README:
https://github.com/mokolabs/heroku-buildpack-libvips

But I'm really thinking about starting from scratch with a brand new modern buildpack that isn't a fork.

[janko]

But I'm really thinking about starting from scratch with a brand new modern buildpack that isn't a fork.

Wow, that buildpack has some deep level of forks 😃. In any case, that sounds good to me!

[mokolabs]

Okay, so I've got two repos for this now.

The first is a slightly tweaked version of the most recently updated vips buildpack:
https://github.com/mokolabs/heroku-buildpack-vips

I'll be using that just for testing and to see how the existing buildpacks work in practice.

The second repo is for the brand new buildpack:
https://github.com/mokolabs/heroku-buildpack-libvips

I'm also going with heroku-buildpack-libvips as the name because it uses the more current libvips terminology and it won't be confused with the dozen or so existing heroku-buildpack-vips forks -- especially when browsing buildpacks on Heroku's Elements marketplace.

[zoras]

Hey guys, I've forked the buildpack at zoras/heroku-buildpack-libvips and made it work for myself.

Can you guys checkout if it works the same for you? Feedbacks appreciated :) https://elements.heroku.com/buildpacks/zoras/heroku-buildpack-libvips

[janko]

@zoras Cool, thanks a lot for making it! I will have a chance to try it out soon, and will let you know how it worked for me.

[adrienpoly]

@zoras I tried your package but I am getting this error during the build

remote: -----> Installing libglib2.0-0_2.56.4-0ubuntu0.18.04.3_amd64.deb
remote: -----> Installing libglib2.0-bin_2.56.4-0ubuntu0.18.04.3_amd64.deb
remote: -----> Installing libglib2.0-dev_2.56.4-0ubuntu0.18.04.3_amd64.deb
remote: -----> Installing libglib2.0-dev-bin_2.56.4-0ubuntu0.18.04.3_amd64.deb
remote: -----> Writing profile script
remote: -----> Rewrite package-config files
remote: -----> vips app detected
remote: -----> Vendoring binaries
remote:        Fetching wget https://kespry-packages.s3.amazonaws.com/vips/heroku-18/libvips-8.6.2.tgz
remote: 
remote: gzip: stdin: not in gzip format
remote: tar: Child returned status 1
remote: tar: Error is not recoverable: exiting now
remote:  !     Push rejected, failed to compile vips app.

I think I followed the instructions by installing both the apt and your package maybe I missed something

[bibendi]

Hey @zoras! You can try this one https://github.com/kinfamilies/heroku-buildpack-vips
It contains the latest version of libvips - 8.8.0
And this build is not using the S3.

[zoras]

@adrienpoly Duh! I linked the kespry github (fixed now). My fork is at https://github.com/zoras/heroku-buildpack-libvips and the buildpack registry should be https://buildpack-registry.s3.amazonaws.com/buildpacks/zoras/heroku-buildpack-libvips.tgz

@bibendi Awesome!

[janko]

Thanks for sharing your buildpacks! I’m going to close this issue since it seems we have working buildpacks now, but feel free to continue commenting here.

[bibendi]

If we have the launchpad PPA for all recent Ubuntu versions, then life would be easier. 😸 I have tried, but gave up.

[iraszl]

Using @zoras 's suggestion I tried to push to heroku, but I'm getting:

-----> vips app detected
-----> Vendoring binaries
       Fetching wget https://kespry-packages.s3.amazonaws.com/vips/heroku-18/libvips-8.6.2.tgz
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
 !     Push rejected, failed to compile vips app.
 !     Push failed

Looks like https://kespry-packages.s3.amazonaws.com/vips/heroku-18/libvips-8.6.2.tgz is no longer accessible. How to resolve this?

cc. @bibendi

Thank you!

[jcupitt]

I would download libvips tarballs from github, eg.:

https://github.com/libvips/libvips/releases/download/v8.8.1/vips-8.8.1.tar.gz

Please use the most recent version, it has various security fixes, see the changelog.

[iraszl]

Thank you @jcupitt and sorry for my ignorance, but how do I use this link? Do I fork the zoras' buildpack and replace the broken link with yours?

[jcupitt]

Sorry, no idea :( I've not done much with heroku. We need an expert to help.

I did make this:

https://github.com/jcupitt/docker-builds/tree/master/libvips-heroku18

Which builds a basic libvips for heroku18 in docker, but it would need some work to make it into a bullet or slug or whatever they call it.

I noticed @zoras (in his excellent buildpack) is adding cfitsio support. I would not enable this myself -- I don't think it has been fuzzed for untrusted files.

libvips is part of oss-fuzz now, so I'd stick to the formats that are being tested there.

https://github.com/google/oss-fuzz/tree/master/projects/libvips

[krnjn]

Not sure if this will solve your case @iraszl, but I had a similar issue with the chain of buildpacks that seemed built to support libvips on Heroku. After trial / error with a few, this is the one that worked for me:

# app.json
{
  ... # configs here for name, environment, etc...
  "stack": "heroku-18",
  "buildpacks": [
    {
      "url": "heroku-community/apt"
    },
    {
      "url": "https://github.com/brandoncc/heroku-buildpack-vips.git"
    },
    {
      "url": "heroku/nodejs"
    },
    {
      "url": "heroku/ruby"
    }
  ]
}

# Aptfile
libglib2.0-0
libglib2.0-dev
libpoppler-glib8

[iraszl]

@krnjn Worked! Thanks so much for your kind attention to the matter.

[jcupitt]

Oh, I'm glad Brandon made that. I would bump the libvips version though -- he's still using 8.7.1.

[iraszl]

Can you guys confirm the above fully work for you? For me the VIPS part works great, but now my app isn't loading CSS and JS. Locally it works, but not on Heroku, so I assume it's related to the buildpack configuration:

heroku buildpacks:add --index 1 heroku-community/apt
heroku buildpacks:add --index 2 https://github.com/brandoncc/heroku-buildpack-vips.git
heroku buildpacks:add --index 3 heroku/nodejs
heroku buildpacks:add --index 4 heroku/ruby

# application.js
//= require rails-ujs
//= require jquery
//= require turbolinks
//= require_tree .

# application.scss
 *= require rails_bootstrap_forms
 */
@import "bootstrap";

# application.html.erb
<!DOCTYPE html>
<html>
  <head>
    <%= csrf_meta_tags %>
    <%= csp_meta_tag %>
    <meta content="width=device-width, initial-scale=1" name="viewport"/>
    <%= stylesheet_link_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
    <%= javascript_include_tag 'application', 'data-turbolinks-track': 'reload' %>
  </head>
...

Do you guys have any ideas for me to try to fix it?

[reckerswartz]

@iraszl try heroku config:set RAILS_SERVE_STATIC_FILES=true

[matpowel]

@iraszl @krnjn thanks for all the updates on this, out of curiosity though do you know why we wouldn't just install the APT buildpack and then just install libvips, libvips-dev etc from the Aptfile? Wouldn't that automatically install the dependencies?

[maxence33]

hi @matpowel have you actually had any luck with installing through Apt buildpack ?

[matpowel]

@maxence33 no, I remember we tried the pure APT method and it didn't work. I can't remember exactly what the issue was. We ended up with this in the Aptfile:

libglib2.0-0
libglib2.0-dev
libpoppler-glib8
libheif

And then the official Heroku Apt and BrandonCC's Vips buildpack (https://github.com/brandoncc/heroku-buildpack-vips)

[maxence33]

Thank you @matpowel , will try to replicate your setup with Brandoncc buildpack

[schneems]

Has anyone managed to git vips installed on Heroku via an aptfile? The example by @matpowel doesn't look like it's working for me:

~ $ cat Aptfile
libglib2.0-0
libglib2.0-dev
libpoppler-glib8
libheif
~ $ vips -v
bash: vips: command not found

[matpowel]

@schneems we never got it working, we're moving to our own slug/docker image but at the moment we are still using the brandoncc buildpack and then APT to install some of the libs per my previous post

[schneems]

Scratch that, apparently this works:

~ $ vips -v
vips-8.4.5-Wed Jan 18 12:36:44 UTC 2017
~ $ cat Aptfile
libglib2.0-0
libglib2.0-dev
libpoppler-glib8
libheif
libvips-dev
libvips

[matpowel]

@schneems does that really work? That's the first attempt we made and it didn't work for ActiveStorage etc

[schneems]

Looks like it's working for me:

mktmpdir
echo "source 'https://rubygems.org'" > Gemfile
echo "gem 'image_processing'" >> Gemfile
bundle install
cat <<EOM > Aptfile
libglib2.0-0
libglib2.0-dev
libpoppler-glib8
libheif
libvips-dev
libvips
EOM
git init .; git add .; git commit -m first
heroku create
heroku buildpacks:clear
heroku buildpacks:add heroku-community/apt
heroku buildpacks:add heroku/ruby
git push heroku

$ heroku run which vips
Running which vips on ⬢ afternoon-sea-69907... up, run.2751 (Free)
/app/.apt/usr/bin/vips

[willnet]

I tried Aptfile @schneems wrote , but I got error like following. We need libvips 8.6+ but the Aptfile gets 8.4.5.

image_processing/vips requires libvips 8.6+ excluded from capture

[schneems]

@willnet if this was on plain linux or docker how would you get that version of libvips on the box? Heroku's "just linux" with some stuff on top of it. Heroku 18 is Ubuntu18 etc. Is there an apt repository that has taht version of libvips?

[schneems]

Worth mentioning that it looks like the above script no longer works on heroku-20. It looks like libheif needs to be libheif-dev.

[sedubois]

I just upgraded a Rails 6 app to Rails 7. The app was already using APT and Vips buildpacks + Aptfile as suggested in this thread.

Now with Rails 7 using Vips by default instead of mini magick, is this still needed or can the Heroku setup be simplified? It is not mentioned in the docs.

[etherbob]

I just upgraded a Rails 6 app to Rails 7. The app was already using APT and Vips buildpacks + Aptfile as suggested in this thread.

Now with Rails 7 using Vips by default instead of mini magick, is this still needed or can the Heroku setup be simplified? It is not mentioned in the docs.

I recently opened a support ticket with Heroku on this and got the following response:

This is an excellent question that's already being discussed by our language owners and stack image maintainers. This is something that we would very much like to support, however, at this stage I don't have an ETA for when this would be implemented or if it will be for that matter.

@sedubois if you have the time and inclination, please send them a support request. It'd be great to get libvips as part of the standard ruby stack at Heroku, but they apparently aren't in any rush without more input from their customers.

[maxence33]

Is there such thing as a Ruby stack ? Maybe they can add it to Heroku 22. I guess VIPS has become mainframe in a lot of environments.

[schneems]

We're looking to add it to the heroku-22 stack. Adding it to heroku-18 or heroku-20 isn't viable. Even putting it in heroku-22 comes with some non-trivial concerns. In the short term it seems there's a buildpack that can be used that's partially maintained by a libvips contributor heroku/heroku-buildpack-ruby#1200 (comment)

[mokolabs]

That would be awesome, @schneems!

[danielricecodes]

libglib2.0-0
libglib2.0-dev
libpoppler-glib8
libheif
libvips-dev
libvips

On Heroku-22, I did not need libheif in Aptfile.

libglib2.0-0
libglib2.0-dev
libpoppler-glib8
libvips-dev
libvips

[matpowel]

@schneems does that mean it got added to Heroku-22?

[schneems]

https://devcenter.heroku.com/articles/stack-packages has libheif-dev as 1.12.0-2build1 for heroku-22