IconLayer (png) render artifacts

[jarekpelczynski]

I use IconLayer with png sprite.
Icons render with a small glitch, something like a gray border or shadow but originally they have only a white border.

[jianhuang01]

can you add your original png and the final rendering effects here?

[wooly]

hey @jianhuang01,

We've come across this issue as well, so hopefully this will help:

const iconMapping = {
  default: {
    x: 0,
    y: 0,
    width: 80,
    height: 80,
  },
}

const iconLayer = new IconLayer({
  id: `icon-layer`,
  pickable: true,
  iconMapping,
  iconAtlas: 'locations-spritesheet-white.png',
  getIcon: () => 'default',
  getSize: 80,
  sizeScale: 5,
  data: layersData,
  opacity: 1,
});

White sprite-sheet, only using the first sprite:

Rendered output in chrome:

When we add in the second sprite for onHover, the result looks like this:

Any ideas?

[wooly]

You can also see it happening on the IconLayer deck.gl example here: https://deck.gl/#/examples/core-layers/icon-layer

It's noticeable when the markers overlap:

Maybe something to do with alpha blending?

[tsherif]

@wooly Are those sprites using the alpha channel to fade off at the edges?
@jianhuang01 Does the icon layer use the alpha channel from the image it's sampling?

[wooly]

@tsherif they're using the alpha channel to anti-alias the circle edges, yep.

[tsherif]

It looks like the icon layer does sample the texture alpha. The issue might be that deck.gl doesn't premultiply its output alphas (some discussion here of why this is an issue).

[tsherif]

@wooly Can you try adding the following to your icon layer constructor props:

parameters: {
    depthMask: false
}

[wooly]

@tsherif didn't make any discernible difference, unfortunately.

[leeran88]

Happens also to me.
@tsherif , any workarounds?

[BowlingX]

Anything that can be done here? Happens for me as well with the latest version:

    "@deck.gl/react": "^8.2.2",
    "@deck.gl/layers": "^8.2.2",
    "react-map-gl": "^5.2.7",

My original sprite: (looks like Github removes the transparency)

[BowlingX]

If I set premultipliedAlpha: false, the border get's a bit nicer

[BowlingX]

I found the issue in my case. I upgraded to the latest deck.gl version (from 8.1 to 8.2). After reading the changelog I adjusted my IconLayer loadOptions back to the previous default. (see https://deck.gl/docs/upgrade-guide#upgrading-from-deckgl-v81-to-v82).

Example:

new IconLayer({ ...yourProps, loadOptions: {
          image: {
            type: 'image'
          }
        }})

[ibgreen]

@BowlingX Good that you were able to pinpoint this. It is unexpected. ImageBitmap is the recommended format for loading images on modern browsers, but based on your findings it looks like there is potential issue with transparency when converted to texture.

I googled a bit but could not find more information on this.

For completeness, can you provide browser information?

If you have time, you could also try the following and see if it makes a difference:

new IconLayer({ ...yourProps, loadOptions: {
          imagebitmap: {
            premultiplyAlpha: 'premultiply'
          }
        }})

For information on that, see createImageBitmap

[BowlingX]

@ibgreen I tried this option already and it did not fix the issue, unfortunately. The only thing that helped was to change the type to image. Browser is Chrome 83.0.4103.116. But the same issue was also present on Firefox 77.0.1 and 78.0.2. Operating system is Windows 10 with NVIDIA 2080 TI.

[Pessimistress]

@ibgreen I think we want premultiplyAlpha: 'none' to remove the border. I created a demo here: https://codepen.io/Pessimistress/pen/xxZQGMM

The option doesn't seem to have any effect either way in Chrome, but works in FireFox.

[ibgreen]

That sounds good. We could perhaps set it as a default option in the image loader?

https://github.com/visgl/loaders.gl/blob/master/modules/images/src/image-loader.js#L36

[Pessimistress]

@ibgreen ImageBitmap's options do not work in Chrome. Setting a default won't solve this issue.

[cneumann]

I've just run into this as well and I can confirm what @Pessimistress says about loadOptions for ImageBitmap not working on Chrome :(

When looking at what Chrome renders with renderdoc I can see that the texture holding the icons is indeed premultiplied (e.g. fully transparent pixels have value RGBA: 0x00000000).
However, the blend function set when drawing is: GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA; I believe it should be GL_ONE, GL_ONE_MINUS_SRC_ALPHA for premultiplied texture data?

I think icon-layer.js#L208 should call draw with custom parameters, e.g. something like:

    this.state.model
        .setUniforms(
          Object.assign({}, uniforms, {
            iconsTexture,
            iconsTextureDim: [iconsTexture.width, iconsTexture.height],
            sizeScale: sizeScale * (sizeUnits === 'pixels' ? viewport.metersPerPixel : 1),
            sizeMinPixels,
            sizeMaxPixels,
            billboard,
            alphaCutoff
          })
        )
        .draw({
            parameters: {
                blendFunc: [GL.ONE, GL.ONE_MINUS_SRC_ALPHA]
          }
        });

[Pessimistress]

Thanks @cneumann for the insight!

Can confirm that this solves the problem:

  new IconLayer({
    ...
    parameters: {
        blendFunc: [1, 771] // GL.ONE, GL.ONE_MINUS_SRC_ALPHA
    },
    loadOptions: {
        imagebitmap: {
          premultiplyAlpha: 'none'
        }
    }
})

We can change IconLayer's default blending parameters, though that could be a breaking change for users who are customizing global blending. @ibgreen

[cneumann]

Just to be sure: if you change the blend function in this way you do want premultiplied alpha, so the option should be imagebitmap.premultiplyAlpha = 'premultiply' - of course that only matters when/if the options actually get passed to the browser ;)

Also, great to know I can pass parameters this way to the layer, I had actually cloned IconLayer and hacked it's draw call to include the alternate blend function :)

[Pessimistress]

Hmm, you are correct; although changing the option does not seem to make any difference. It looks like Chrome's bitmap color is always premultiplied regardless.

I can confirm the the option is indeed passed to createImageBitmap.

[ibgreen]

Ok, this seems to be a rabbithole. I suppose ImageBitmaps are supposed to be in a format that is particularly efficient for rendering, maybe the implementation is allowed this freedom.

Maybe try loading as Image instead of ImageBitmap then. Less performant but correct results... Something along these lines?

new IconLayer({ ...yourProps, loadOptions: {
          image: {
            type: 'image'
          }
        }})

[Pessimistress]

Edit: no, the option is not passed to createImageBitmap. This is a bug in loaders.gl. I'll put up a fix.

[Pessimistress]

Update: an issue is fixed in @loaders.gl/images@2.2.9, and the artifacts can be removed with

new IconLayer({
    ...
    loadOptions: {
        imagebitmap: {
          premultiplyAlpha: 'none'
        }
    }
})

I plan to make premultiplyAlpha: 'none' the default in the upcoming 8.3.0 release so that this workaround will no longer be needed.

[Pessimistress]

Fixed in 8.3.0-beta.2