Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

composite operation create a gray outline with the latest 0.30.2 #3118

Closed
3 tasks done
rawpixel-vincent opened this issue Mar 3, 2022 · 15 comments
Closed
3 tasks done

Comments

@rawpixel-vincent
Copy link

@rawpixel-vincent rawpixel-vincent commented Mar 3, 2022

Possible bug

Is this a possible bug in a feature of sharp, unrelated to installation?

  • Running npm install sharp completes without error.
  • Running node -e "require('sharp')" completes without error.

Are you using the latest version of sharp?

  • I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?


  System:
    OS: macOS 12.2.1
    CPU: (8) arm64 Apple M1
    Memory: 131.50 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.7.0 - ~/.nvm/versions/node/v16.7.0/bin/node
    Yarn: 1.22.15 - ~/.yarn/bin/yarn
    npm: 7.20.3 - ~/.nvm/versions/node/v16.7.0/bin/npm
  npmPackages:
    sharp: 0.30.2 => 0.30.2 

What are the steps to reproduce?

take a transparent png,
apply a composite with tiles to the background,
output to jpeg or webp,
a gray outline is visble (obvious on white images like attached)

inputFile
input file

inputTile
checker-bg-55

example code 1:

let image = sharp(inputFile);
image.composite([{ input: inputTile, tile: true, blend: 'dest-over', gravity: 'northwest' }]);
image.resize({ width: 400, fit: 'outside' });
image.jpeg({ quality: 100 });
OR
image.webp({ quality: 100 });

example code 2:

let image = sharp(inputFile);
image.resize({ width: 400, fit: 'outside' });
image = sharp(await image.png().toBuffer());
image.composite([{ input: inputTile, tile: true, blend: 'dest-over', gravity: 'northwest' }]);
image.jpeg({ quality: 100 });
OR
image.webp({ quality: 100 });

output with 0.30.2
test

output with 0.30.1
test

@rawpixel-vincent rawpixel-vincent changed the title composite operation create a gray outline line with the latest 0.30.2 composite operation create a gray outline with the latest 0.30.2 Mar 3, 2022
@lovell
Copy link
Owner

@lovell lovell commented Mar 3, 2022

These kind of fringe effects around edges usually relate to premultiplication.

Commit c620025 in v0.30.2 switched from libvips' composite2 to composite as well as passing responsibility for premultiplication to libvips.

Does this only occur for dest-over? Perhaps the problem fixed by libvips/libvips#1301 needs to include that blend also?

@rawpixel-vincent
Copy link
Author

@rawpixel-vincent rawpixel-vincent commented Mar 3, 2022

using image.composite([{ input: inputTile, tile: true, blend: 'dest-in', gravity: 'northwest' }]);

test

doesn't seems to produce that gray line, I can test other options if that's helpful

edit: I only use composite with those options, so didn't spot anything else

@rawpixel-vincent
Copy link
Author

@rawpixel-vincent rawpixel-vincent commented Mar 3, 2022

do you think I should transfer this issue to https://github.com/libvips/libvips and reference the change in libvips/libvips#1301 ?

@lovell
Copy link
Owner

@lovell lovell commented Mar 3, 2022

Thanks for confirming, if you're able to reproduce using vips composite2 in1.png in2.png out.png dest-over at the command line then yes please.

@rawpixel-vincent
Copy link
Author

@rawpixel-vincent rawpixel-vincent commented Mar 3, 2022

thank you, here is my results using vips-8.12.2-Tue Jan 25 09:34:32 UTC 2022 installed with brew install libvips

output of --vips-config

enable debug: no
enable deprecated library components: yes
enable modules: no
use fftw3 for FFT: yes
accelerate loops with orc: yes
ICC profile support with lcms: yes (lcms2)
zlib: yes
text rendering with pangocairo: yes
font file support with fontconfig: yes
RAD load/save: yes
Analyze7 load/save: yes
PPM load/save: yes
GIF load:  yes
GIF save with cgif: yes
EXIF metadata support with libexif: yes
JPEG load/save with libjpeg: yes (pkg-config)
JXL load/save with libjxl: yes (dynamic module: no)
JPEG2000 load/save with libopenjp2: yes
PNG load with libspng: yes
PNG load/save with libpng: yes (pkg-config libpng >= 1.2.9)
quantisation to 8 bit: yes
TIFF load/save with libtiff: yes (pkg-config libtiff-4)
image pyramid save: yes
HEIC/AVIF load/save with libheif: yes (dynamic module: no)
WebP load/save with libwebp: yes
PDF load with PDFium:  no
PDF load with poppler-glib: yes (dynamic module: no)
SVG load with librsvg-2.0: yes
EXR load with OpenEXR: yes
OpenSlide load: yes (dynamic module: no)
Matlab load with matio: yes
NIfTI load/save with niftiio: no
FITS load/save with cfitsio: yes
Magick package: MagickCore (dynamic module: no)
Magick API version: magick7
load with libMagickCore: yes
save with libMagickCore: yes

I don't reproduce the issue using vips composite2 inputFile.png inputTile.jpg output.png dest-over

inputFile.png
inputFile

inputTile.png
inputTile

output.png
output

@rawpixel-vincent
Copy link
Author

@rawpixel-vincent rawpixel-vincent commented Mar 3, 2022

when using vips composite2 inputFile.png inputTile.jpg output.png dest-over --premultiplied
the output give me an white outline border

output with --premultiplied
output

@rawpixel-vincent
Copy link
Author

@rawpixel-vincent rawpixel-vincent commented Mar 3, 2022

fwiw I got the same results when sharp use the globally available libvip instead of the prebuilt binaries
here is the complete test

const { readFileSync, writeFileSync, unlinkSync } = require('fs');
const path = require('path');
const sharp = require('sharp');
let assert = require('assert');

describe('composite', () => {
  it(`create a tiled png composite output from sharp pipeline`, async () => {
    try {
      unlinkSync(path.resolve(`${__dirname}/output/test2.jpg`));
    } catch (error) {}
    let image = sharp(readFileSync(path.resolve(`${__dirname}/inputFile.png`)));
    image.composite([{ input: readFileSync(path.resolve(`${__dirname}/inputTile.jpg`)), tile: true, blend: 'dest-over', gravity: 'northwest' }]);
    image.resize({ width: 400, fit: 'outside' });
    image.jpeg({ quality: 100 });

    writeFileSync(path.resolve(`${__dirname}/output/test2.jpg`), await image.toBuffer());
  });
});

outputs:
test2

@lovell
Copy link
Owner

@lovell lovell commented Mar 3, 2022

Thanks for the all the extra examples, I can reproduce this locally with sharp, but I've yet to try to reproduce with vips at the command line.

If sharp premultiplies both images before compositing and sets premultiplied to true (as it used to prior to this change) then the problem goes away.

I think we're seeing this due to libvips only premultiplying one of the images when compositing, which is usually the "lower" image, but with the dest-* blends I think it might be the "upper" image that needs this instead/as well.

https://github.com/libvips/libvips/blob/330ebf3cd7114afd6bc634b450bea20d73fd864f/libvips/conversion/composite.cpp#L508-L512

Possibly the calculation for the dest-over resulting colour should somehow involve the resulting alpha?

https://github.com/libvips/libvips/blob/330ebf3cd7114afd6bc634b450bea20d73fd864f/libvips/conversion/composite.cpp#L566

@jcupitt When you get a mo, please can you check the current VIPS_BLEND_MODE_DEST_OVER logic to see if we're missing anything obvious, thank you.

@jcupitt
Copy link

@jcupitt jcupitt commented Mar 4, 2022

Hi @lovell, this is an odd one. I tried a small test program

#!/usr/bin/python3

import sys
import pyvips

image = pyvips.Image.new_from_file(sys.argv[1])
tile = pyvips.Image.new_from_file(sys.argv[2])

background = tile \
    .replicate(1 + image.width / tile.width,
               1 + image.height / tile.height) \
    .crop(0, 0, image.width, image.height)

over = background.composite(image, "over")
dest_over = image.composite(background, "dest-over")

diff = (over - dest_over).abs().max()

print(f"maximum difference = {diff}")

With git master libvips and with 8.12 I see:

$ ./composite-tile.py ~/pics/heart.png ~/pics/tile.jpg
maximum difference = 0.0

So over and dest-over seem to work. I saved the images and checked by hand as well, they look OK.

I think I would patch sharp to save the argument images to composite just before calling it, probably as .v format to make sure the saver doesn't change anything. You can then check the argument images to verify that sharp hasn't done any premultiplication (vipsdisp is handy for this, right-click, turn on the info bar and you can see the exact pixel values in the file). Then run composite at the command-line and check the result.

If the input images are wrong I suppose sharp might have a stray premultiply still. If they are OK and the output is wrong, we have a reproducer for a libvips bug.

@lovell
Copy link
Owner

@lovell lovell commented Mar 4, 2022

Thanks John, I think I've found the problem, there was a extra unpremultiply taking place in sharp after composite that affects some of the dest-* blend modes.

Rather annoyingly the unit tests did pick this up at the time but I incorrectly assumed the "new" behaviour was more accurate. c620025#diff-7a6322e019754e2e5f843a65aa8cfb3635d77a757b88ef4c6a9450a022e5678f

@lovell
Copy link
Owner

@lovell lovell commented Mar 4, 2022

I've reverted the test fixtures to their v0.30.1 state, as well as removing the unpremultiply operation when compositing, in commit 23033e2

This will be in v0.30.3, thanks for reporting.

@lovell lovell added this to the v0.30.3 milestone Mar 4, 2022
@lovell
Copy link
Owner

@lovell lovell commented Mar 14, 2022

v0.30.3 now available.

@lovell lovell closed this as completed Mar 14, 2022
@rawpixel-vincent
Copy link
Author

@rawpixel-vincent rawpixel-vincent commented Mar 14, 2022

Hi,
I've tried with sharp 0.30.3

> npm view sharp dist-tags.latest
0.30.3
>npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp

  System:
    OS: macOS 12.2.1
    CPU: (8) arm64 Apple M1
    Memory: 78.98 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.7.0 - ~/.nvm/versions/node/v16.7.0/bin/node
    Yarn: 1.22.15 - ~/.yarn/bin/yarn
    npm: 7.20.3 - ~/.nvm/versions/node/v16.7.0/bin/npm
  npmPackages:
    sharp: 0.30.3 => 0.30.3 

for the following code, with the same input files

describe('bg mosaic', () => {
  it(`create a tiled png output from sharp pipeline`, async () => {
    try {
      unlinkSync(path.resolve(`${__dirname}/output/test2.jpg`));
    } catch (error) {}
    let image = sharp(readFileSync(path.resolve(`${__dirname}/inputFile.png`)));
    image.composite([{ input: readFileSync(path.resolve(`${__dirname}/inputTile.jpg`)), tile: true, blend: 'dest-over', gravity: 'northwest' }]);
    image.resize({ width: 400, fit: 'outside' });
    image.jpeg({ quality: 100 });

    writeFileSync(path.resolve(`${__dirname}/output/test2.jpg`), await image.toBuffer());
  });
});

I still have the same output
test2

edit: I don't have libvips installed, it's using sharp prebuilt binaries
edit2: I don't seem to be able to re-open the issue, I let it rest for a while and open a new one in case you're not notified, fwiw it's possible I'm doing something stupid... but I've tried multiple time, cleaning everything, deleting node_modules, checking my working path..
edit3: re-attaching the input files I'm testing with in case
inputFile
inputTile

@lovell
Copy link
Owner

@lovell lovell commented Mar 14, 2022

v0.30.3 works on my machine:

node -p "require('sharp')('heart.png').composite([{ input: 'checks.jpg', blend: 'dest-over' }]).toFile('out.png')"

@rawpixel-vincent
Copy link
Author

@rawpixel-vincent rawpixel-vincent commented Mar 14, 2022

yes I messed up my test files... and was testing with a wrong input, it works fine, thank you again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants