API version 4

[kleisauke]

API version 4

Preface

As the amount of traffic increases, we come up with solutions to handle the traffic. Nowadays we handle 2 million requests per hour (4 million on peak), which is great, but it requires proper server capacity and effective software that scales with the amount of traffic that make this service reliable.

OpenResty, LuaJIT and lua-vips

With API version 4 we decided to rewrite to complete code base to Lua and switch to OpenResty, which turns Nginx and LuaJIT into a full-fledged scriptable web platform. For image handling and manipulation we still use the libvips image processing library under the hood. We have only moved from the PHP-binding to the Lua binding.

You may wonder why we did this; it's because OpenResty is robust, fast and scales with the amount of traffic we are processing nowadays. One of the core benefits of OpenResty is that it is fully asynchronous and that our code can be written directly inside Nginx without using PHP-FPM and FastCGI. The amazing just-in-time (JIT) compilation and the integrated foreign function interface (FFI) in LuaJIT have also been motives to move away from PHP.

Perhaps you've already noticed, https://images.weserv.nl/ already states that the API version 4 is running.
This is not a mistake, our A/B testing went so well that we decided (on time of writing) to forward all traffic to API version 4.

What has changed for the end user?

Absolutely nothing, all parameters listed on API 3 still work on API 4. The only thing you might notice is when you process an image, that it will load faster. If you found something odd, don't hesitate to contact us. We track support tickets, issues and feature requests using the GitHub issue tracker.

What has been added?

We introduced some new parameters and a more flexible URI parser.

Masking

Controls the visible and non-visible area of the image. Previously the &mask and &mtrim parameter were named as &shape and &strim, it was renamed to make room for new mask features.

• Mask type &mask=. Sets the mask type from a predefined list of shapes:
  • circle
  • ellipse
  • triangle
  • triangle-180: Triangle tilted upside down
  • pentagon
  • pentagon-180: Pentagon tilted upside down
  • hexagon
  • square: Square tilted 45 degrees
  • star: 5-point star
  • heart
• Mask trim &mtrim. Removes the remaining whitespace from the mask.
• Mask background &mbg=. Sets the background color of the mask.

Flexible URI parser

With API 4 we are supporting URI's starting with http:// and https://. Previously this would result in an error to prevent BBCode parsers to fail on the double http(s)://-part. Because Markdown is increasingly becoming the standard for user input, we decided to make the URI parser more flexible.
Note that we will still support ?url=ssl:… and ?url=//…-requests for backwards compatibility.

Where can I view the code?

The code is available on the 4.x branch, this will become the default branch in the future.

[andrieslouw]

Small note: You'll need libvips version 8.7+ to run API 4. As of now, there is only a release candidate available, which we run in production. But I'm sure all the amazing work of @jcupitt et al. will make it into a official release soon! 😄

[andrieslouw]

It seems that API 4 has some (small?) memory leaks, we're investigating if this issue has to do with Nginx, LuaJIT, Nginx, lua-vips or libvips. For more information: libvips/lua-vips#24

[andrieslouw]

For now we fixed the memory leaks by using jemalloc:

echo '/usr/lib64/libjemalloc.so' >> /etc/ld.so.preload
service openresty reload

This prevents actual RAM from being used, but it may show high committed memory.

[andrieslouw]

Also, libvips 8.7.0 has official been released! This is needed to run API 4, and also brings some improvements to the latest version of API 3. @jcupitt moved everything to a new organization too, so check out their new home: https://github.com/libvips/

[andrieslouw]

API 5 has been released, see #189.