[Feature] webp support

[nitriques]

I know it is not an easy piece of work, but offering webp support would be pretty cool.

Thanks.

[Kikobeats]

how can we do that?

[nitriques]

Well, how to you currently convert images to arrays of pixels ?

[akfish]

In browser, it's done by <canvas>. So it's just a matter of what formats the browser supports. See browser.ts.
In node.js, jimp is used, for it's a pure JavaScript implementation. I didn't want to introduce some binaries/dependencies that might break on some platforms into the default implementation. See node.ts.

node-vibrant is designed to be extendable. Image format support is provided via ImageClasss. One can implement his/her own ImageClass and use it by set Vibrant.DefaultOpts.ImageClass = YourCustomImageClass.

To implement one, one could

• Extend abstract class ImageBase.
• Or implements Image interface from scratch.

Output pixel array should be the same format as ImageData.data. That is "a one-dimensional array containing the data in the RGBA order, with integer values between 0 and 255 (included)".

It's really just a matter of finding a webp decoder package for node.

[nitriques]

Great! thanks! I am not that good in TypeScript, but I'll see what can be done!

[akfish]

Nice!

FYI. I'm refactoring node-vibrant into multiple small packages for version 3.1.0. All image classes are implemented as their own npm packages.

Check out @vibrant/image-node for reference implementation. It's still the same as described above. Except that now you will only need one @vibrant/image dependency from this project, instead of forking the entire repo. Hopefully that would simplify things.

[nitriques]

Great !

[favna]

I've been using this package in my Discord bot which is starting to move to displaying all its images in webp format by default due to its many advantages so this feature is starting to be a huge thing for me. I see a wontfix label so that really worries me.

@nitriques have you done any of your "seeing what can be done" yet? or are you going to work on it after all @akfish ?

[akfish]

@favna It won't be fixed in the main package/repo. But it is fixable by implementing a custom ImageClass (and published as a separate npm package). WebP libs for node.js (that I've seen) require external binaries or native modules, which could be messy to support across platforms. The main package is intended to work out-of-box for all platforms.

I'm working on this project in my spare time and this feature is not high on my priority list. So I'm afraid it won't be fixed by me any time soon.

[nitriques]

@favna

have you done any of your "seeing what can be done" yet?

Yes and all I see are native ad-dons

The good thing is that I need it, so I might have the time to poke around it soon.

[danielberndt]

How do you feel about allowing to use sharp instead of jimp via an optional flag? Since v0.20 sharp doesn't require any additional installation steps besides npm install on most systems. It also comes with webp and svg support and promises to be a faster solution since it's based on native modules.

[danielberndt]

I'm just trying to create a sharp-based ImageClass but it's not gonna be straight forward as resize/scaleDown needs to be synchronous, whereas sharp's resize operation is async.

[danielberndt]

fyi: Here's a ImageClass implementation using sharp. It supports both webp and svg in addition to all the default formats. Due to the resizing issue described in the comment above, the code resizes the image in the load method, effectively ignoring all resize commands that come from node-vibrant:

import * as sharp from "sharp";
import {ImageBase, ImageSource} from "@vibrant/image";

class SharpImage extends ImageBase {
  private _image: ImageData = undefined as any;

  async load(image: ImageSource): Promise<ImageBase> {
    if (typeof image === "string" || image instanceof Buffer) {
      const {data, info} = await sharp(image)
        .resize(200, 200, {fit: "inside", withoutEnlargement: true})
        .ensureAlpha()
        .raw()
        .toBuffer({resolveWithObject: true});
      this._image = {
        width: info.width,
        height: info.height,
        data: (data as unknown) as Uint8ClampedArray,
      };
      return this;
    } else {
      return Promise.reject(
        new Error("Cannot load image from HTMLImageElement in node environment")
      );
    }
  }
  clear(): void {}
  update(): void {}
  getWidth(): number {
    return this._image.width;
  }
  getHeight(): number {
    return this._image.height;
  }
  resize(targetWidth: number, targetHeight: number, ratio: number): void {
    // done in the load step, ignoring any maxDimension or quality options
  }
  getPixelCount(): number {
    const {width, height} = this._image;
    return width * height;
  }
  getImageData(): ImageData {
    return this._image;
  }
  remove(): void {}
}

[Kikobeats]

@danielberndt that code snippet is so good, thanks for it 🙂

is it any reason you are resizing the image first? I think that keeping the image as larger as possible is going to improve the result.

[danielberndt]

@Kikobeats if I remember correctly, I copied the default behaviour of this library which also decreased the size of the image before processing it. Resizing the image before processing leads to more consistent (and much shorter) processing times.

[Kikobeats]

Hum, I tried to find the old code as reference but no success – maybe @akfish @rutchcorn can confirm is resize is a thing done internally these days or not really?

btw, I adopt he code into a NPM high level library: microlinkhq/splashy#15

[crutchcorn]

@Kikobeats, @danielberndt is correct in terms of faster (and lower resource usage) processing when resizing.

However, because the image is now smaller and the image quantizer now inherently has less data, the output colors will be less accurate (altho, FWIW "accurate" + "quantizer" is a... Dicey proposition and idk if we'll ever truly get there so long as I'm involved at any stage lol)

Honestly, I'd go with resizing first. There's very little reason to demand high color accuracy where it would matter, and if it did - this library would already not provide your needs

[danielberndt]

Hum, I tried to find the old code as reference but no success

Here's the place where the resize is triggered:

node-vibrant/src/vibrant.ts

Line 52 in 8843d00

image.scaleDown(opts)

[rendomnet]

is webp support in progress?