HLR green artifacts with OpenCL

[groutr]

I just recompiled master this morning (Feb 13) and noticed that HLR inpaint-opposed and segmented are giving me subtle green artifacts on lots of photos that I don't remember having such artifacts before. I compile dt from master pretty frequently so I'm positive this is a fairly recent change (probably within the last week or two). The problem seems to disappear with opencl disabled.

This is a pristine image with only dt's default initial processing (and opencl enabled)

this is darktable 4.3.0+609~g4be1794967
copyright (c) 2009-2023 johannes hanika
https://github.com/darktable-org/darktable/issues/new/choose

compile options:
bit depth is 64 bit
normal build
SSE2 optimized codepath enabled
OpenMP support enabled
OpenCL support enabled
Lua support enabled, API version 9.0.0
Colord support enabled
gPhoto2 support enabled
GraphicsMagick support enabled
ImageMagick support disabled
libavif support enabled
libheif support enabled
libjxl support enabled
OpenJPEG support enabled
OpenEXR support enabled
WebP support enabled

• OS : Archlinux (up-to-date)
• Graphics card : Intel 530
• OpenCL installed : NEO
• OpenCL activated : yes

[jenshannoschwalm]

Could you share the raw file please?
Any chance you updated the CL driver recently?

[groutr]

I can share a similar file that exhibits the same issue (and to much more severe degree). Where would be the best place to share? About 30MB (which exceeds the 25MB file size limit on github).

The last update to intel opencl packages was back on Jan 18, 2023. It is possible that this opencl update could be responsible. However, when I try downgrading opencl packages to test, I keep getting a segfault inside intel-opencl-runtime. However, when downgrading darktable to a previously compiled package made in early Feb, I get the same green artifacts with the newer opencl runtime.

[jenshannoschwalm]

Just any sharing service to pick from is fine.

[groutr]

Here is a google drive link:
https://drive.google.com/file/d/1KeWA0TSWtj5bfpBwcIal0m_ejNHCJpBM/view?usp=share_link

Included in the file are the raw and an exported jpg.

[groutr]

This was the latest update to intel opencl packages

89806:[2023-01-18T10:22:35-0600] [ALPM] upgraded intel-opencl-clang (14.0.0.r3+g5dd5031-1 -> 15.0.0-1)
89807:[2023-01-18T10:22:36-0600] [ALPM] upgraded intel-graphics-compiler (1:1.0.11485-8 -> 1:1.0.12812.4-1)
89808:[2023-01-18T10:22:36-0600] [ALPM] upgraded intel-compute-runtime (22.30.23789-1 -> 22.43.24595.30-1)

[groutr]

@gi-man thank you for trying it with your nvidia card. I tried many other modules with opencl enabled to see if artifacts could be seen. I can only get artifacts with highlight recovery module.

[jenshannoschwalm]

Can't reproduce either here. Could you also send your log (using -d opencl) and clinfo output please?

Looks a lot like a driver issue but ...

[groutr]

I guess I actually a have an HD520, not a 530.

$ darktable-cltest
[dt_codepaths_init] will be using experimental plain OpenMP SIMD codepath.
[dt_get_sysresource_level] switched to 1 as `default'
  total mem:       15730MB
  mipmap cache:    1966MB
  available mem:   7865MB
  singlebuff:      491MB
  OpenCL tune mem: WANTED
  OpenCL pinned:   WANTED
[opencl_init] opencl related configuration options:
[opencl_init] opencl: ON
[opencl_init] opencl_scheduling_profile: 'default'
[opencl_init] opencl_library: 'default path'
[opencl_init] opencl_device_priority: '*/!0,*/*/*'
[opencl_init] opencl_mandatory_timeout: 400
[opencl_init] opencl library 'libOpenCL' found on your system and loaded
[opencl_init] found 1 platform
[opencl_init] found 1 device

[dt_opencl_device_init]
   DEVICE:                   0: 'Intel(R) HD Graphics 520 [0x1916]'
   CANONICAL NAME:           intelrhdgraphics5200x1916
   PLATFORM NAME & VENDOR:   Intel(R) OpenCL HD Graphics, Intel(R) Corporation
   DRIVER VERSION:           22.43.30
   DEVICE VERSION:           OpenCL 3.0 NEO
   DEVICE_TYPE:              GPU
   GLOBAL MEM SIZE:          12584 MB
   MAX MEM ALLOC:            4096 MB
   MAX IMAGE SIZE:           16384 x 16384
   MAX WORK GROUP SIZE:      256
   MAX WORK ITEM DIMENSIONS: 3
   MAX WORK ITEM SIZES:      [ 256 256 256 ]
   ASYNC PIXELPIPE:          NO
   PINNED MEMORY TRANSFER:   WANTED
   MEMORY TUNING:            WANTED
   FORCED HEADROOM:          400
   AVOID ATOMICS:            NO
   MICRO NAP:                250
   ROUNDUP WIDTH:            16
   ROUNDUP HEIGHT:           16
   CHECK EVENT HANDLES:      128
   PERFORMANCE:              2.765
   TILING ADVANTAGE:         0.000
   DEFAULT DEVICE:           NO
   KERNEL DIRECTORY:         /usr/share/darktable/kernels
   CL COMPILER OPTION:       -cl-fast-relaxed-math
   KERNEL LOADING TIME:       0.0322 sec
[opencl_init] OpenCL successfully initialized.
[opencl_init] here are the internal numbers and names of OpenCL devices available to darktable:
[opencl_init]           0       'Intel(R) HD Graphics 520 [0x1916]'
[opencl_init] FINALLY: opencl is AVAILABLE on this system.
[opencl_init] initial status of opencl enabled flag is ON.
[dt_opencl_update_priorities] these are your device priorities:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       -1      0       0       -1
[dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       0       0       0       0
[opencl_synchronization_timeout] synchronization timeout set to 200
[dt_opencl_update_priorities] these are your device priorities:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       -1      0       0       -1
[dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       0       0       0       0
[opencl_synchronization_timeout] synchronization timeout set to 200

Running darktable with -d opencl didn't seem to produce any meaningful output beyond clinfo.

[dt_codepaths_init] will be using experimental plain OpenMP SIMD codepath.
[dt_get_sysresource_level] switched to 1 as `default'
  total mem:       15730MB
  mipmap cache:    1966MB
  available mem:   7865MB
  singlebuff:      491MB
  OpenCL tune mem: WANTED
  OpenCL pinned:   WANTED
[opencl_init] opencl related configuration options:
[opencl_init] opencl: ON
[opencl_init] opencl_scheduling_profile: 'default'
[opencl_init] opencl_library: 'default path'
[opencl_init] opencl_device_priority: '*/!0,*/*/*'
[opencl_init] opencl_mandatory_timeout: 400
[opencl_init] opencl library 'libOpenCL' found on your system and loaded
[opencl_init] found 1 platform
[opencl_init] found 1 device

[dt_opencl_device_init]
   DEVICE:                   0: 'Intel(R) HD Graphics 520 [0x1916]'
   CANONICAL NAME:           intelrhdgraphics5200x1916
   PLATFORM NAME & VENDOR:   Intel(R) OpenCL HD Graphics, Intel(R) Corporation
   DRIVER VERSION:           22.43.30
   DEVICE VERSION:           OpenCL 3.0 NEO
   DEVICE_TYPE:              GPU
   GLOBAL MEM SIZE:          12584 MB
   MAX MEM ALLOC:            4096 MB
   MAX IMAGE SIZE:           16384 x 16384
   MAX WORK GROUP SIZE:      256
   MAX WORK ITEM DIMENSIONS: 3
   MAX WORK ITEM SIZES:      [ 256 256 256 ]
   ASYNC PIXELPIPE:          NO
   PINNED MEMORY TRANSFER:   WANTED
   MEMORY TUNING:            WANTED
   FORCED HEADROOM:          400
   AVOID ATOMICS:            NO
   MICRO NAP:                250
   ROUNDUP WIDTH:            16
   ROUNDUP HEIGHT:           16
   CHECK EVENT HANDLES:      128
   PERFORMANCE:              2.765
   TILING ADVANTAGE:         0.000
   DEFAULT DEVICE:           NO
   KERNEL DIRECTORY:         /usr/share/darktable/kernels
   CL COMPILER OPTION:       -cl-fast-relaxed-math
   KERNEL LOADING TIME:       0.0301 sec
[opencl_init] OpenCL successfully initialized.
[opencl_init] here are the internal numbers and names of OpenCL devices available to darktable:
[opencl_init]           0       'Intel(R) HD Graphics 520 [0x1916]'
[opencl_init] FINALLY: opencl is AVAILABLE on this system.
[opencl_init] initial status of opencl enabled flag is ON.
[dt_opencl_update_priorities] these are your device priorities:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       -1      0       0       -1
[dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       0       0       0       0
[opencl_synchronization_timeout] synchronization timeout set to 200
[dt_opencl_update_priorities] these are your device priorities:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       -1      0       0       -1
[dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe:
[dt_opencl_update_priorities]           image   preview export  thumbs  preview2
[dt_opencl_update_priorities]           0       0       0       0       0
[opencl_synchronization_timeout] synchronization timeout set to 200
     2.8490 [dt_opencl_check_tuning] use 12184MB (tunemem=ON, pinning=ON) on device `Intel(R) HD Graphics 520 [0x1916]' id=0
     5.9395 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
     5.9405 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
     8.4153 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
     8.4164 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    13.7945 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    13.7956 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    13.8372 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    13.8399 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    42.7362 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    42.7374 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    44.3754 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    44.3765 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    51.9546 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    51.9560 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    52.0506 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    52.0519 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    57.8525 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    57.8538 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    57.8962 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
    57.8976 [dt_ioppr_set_pipe_output_profile_info] unsupported output profile (null) /home/grout/.config/darktable/color/out/LP140WF6-SPB1 #1 2022-09-26 16-40 2.2 F-S XYZLUT+MTX.icc, it will be replaced with sRGB
 [opencl_summary_statistics] device 'Intel(R) HD Graphics 520 [0x1916]' (0): 163 out of 163 events were successful and 0 events lost. max event=24

I cleared the cached kernels to force them to be regenerated and tinkering around some more, I noticed that I can no longer reproduce the artifacts in the segmentation mode. Only the inpainting-opposed mode seems to produce the artifacts now. @jenshannoschwalm I'm not denying it could be a driver issue. It's very possible that I just didn't notice the artifacts after updating the driver. I'm going to try downgrading the driver again and see if I can get past the segfault.

[jenshannoschwalm]

The tunings for opencl will likely not help anyway btw.

[groutr]

Bisecting resulted in this commit 322300a as the first bad commit.

Some of the interesting commits during bisect.
Commit 73954177a. I see this as a good commit.

Commit ded7dfd1a. Strange purple artifacts.

Commit 322300ad3. Purple is gone and green artifacts are there.

[groutr]

bisecting the purple issue resulted in this commit: db9cf76

[jenshannoschwalm]

Thanks a lot for this testing !!

On first (short thinking) sight your results don't make much sense :-) After some more thinking the different "color shifts" in highlighted areas make a lot of sense, they demonstrate some sort of rounding error. And - there is something in the CL docs called "Compute Capability" and the 520 hardware seems to be using v1.2 which has significantly lower precision on maths operators.

You could confirm this by testing on your problematic image with -d pipe running with either opencl on/off (please note you have to restart dt with --disable-opencl to do so, don't just toggle opencl on/off in the preferences) and watch out for the lines

[opposed chroma cache] %f %f %f for opphash

they should be significantly different. Could you please do so? If you can confirm i would know how to fix

[groutr]

@jenshannoschwalm My pleasure. Thank you for your guidance on this issue.

I used the duck pond image from my last post and here are the results

# OpenCL
6.5495 [opposed chroma cache] 1.164860 0.624310 1.471562 for opphash  16254282709876457229

# With --disable-opencl
6.3151 [opposed chroma cache] 0.185472 0.017018 0.319225 for opphash  16254282709876457229

If I may ask, what is the difference between toggling OpenCL in in the preferences vs using --disable-opencl?

[jenshannoschwalm]

If I may ask, what is the difference between toggling OpenCL in in the preferences vs using --disable-opencl?

Normally - none :-) Here we needed to force a recalculation of the cached chroma correction data and this way was the most simple way to describe how you could achieve this.

BTW your reported difference is HUGE. I am not sure if this can be explained with my idea about "compute capability".

I will do a pr later for you to test ...