LabelmapOutline test fails on Chrome headless

[floryst]

High-level description

Pinging @sedghi.

The labelmap outline test fails on Chrome headless. You can see this when you run npm run test:headless. This didn't show up prior, because of a disparity in our workflows which is being resolved in #2308. Note how the labelmap boundary seems to be affected. I'm not sure how the headless mode affects things in this manner, so I don't have a strong idea of how to debug this currently.

Steps to reproduce

• Run npm run test:headless

[sedghi]

Looking into this I confirm this is not related to the #2298 since turning off the outline completely also generates an inconsistent labelmaps between chrome and headless chrome, but the tests says pass....

I'm looking into changing the dataset to my sample data to narrow down if the problem is the data.
Also checked the recent HalfFloat addition (wondering if WebGL extensions are not in headless), but was not the problem either.

Other possible bugs:

• renderWindow with many renderers has bugs other than labelmapOutline
• camera has some error between headless and chrome

CC @swederik

[floryst]

Thanks for looking into this. As a testing note, I noticed that Chrome non-headless renders correctly with respect to the test image (npm run test). So far, it's just Chrome Headless that fails.

Just checking: did you mean to paste the output/expected/diff images? The test outputs three separate images, so you actually need to screenshot the screen region, or copy and paste all three columns.

[sedghi]

@floryst I was checking the rendered image before, now I updated the comment with the output (headless) and baseline (chrome), which shows the tests pass but the render is different, which is weird. Maybe colorTransferFunction is the problem?

[floryst]

The color transfer function could be an issue. If so, then we should see higher image diff deltas for the color transfer function tests on headless. I think one thing to try is to use a linear grayscale (black->white) colormap to see if the issue still persists.

FYI I am planning on disabling the recently committed labelmap outline test to let new releases be released, but I will still closely follow this issue so that we can turn it back on.

[sedghi]

So black and white also passes the test, I'm wondering if the compareImages is wrong too

labelMap.cfun.addRGBPoint(1, 1, 1, 1); // label "1" will be white
labelMap.ofun.addPoint(0, 0);
labelMap.ofun.addPoint(1, 0.9, 0.9, 1.0); // Red will have an opacity of 0.2.

[floryst]

So compareImages uses pixelmatch, and there's a pixel tolerance applied to it. Here's the relevant bit of code: https://github.com/Kitware/vtk-js/blob/master/Sources/Testing/testUtils.js#L70-L81. It's some color-based metric I am not familiar with. I think trying to tighten the threshold will tell us a bit about how much tolerance there is for the pixel-to-pixel comparison.