Delay setupLayers to LayerGui being shown

[k-dominik]

Issue: when switching to dataexport one sometimes raises an exception like (against current volumina main):

Exception

ERROR 2024-05-07 16:55:14,802 excepthooks 93120 6118469632 Unhandled exception in thread: 'Worker #0'
ERROR 2024-05-07 16:55:14,803 excepthooks 93120 6118469632 Traceback (most recent call last):
  File ".../ilastik/lazyflow/request/request.py", line 383, in _execute
    self._result = self.fn()
  File ".../ilastik/lazyflow/slot.py", line 868, in __call__
    result_op = self.operator.call_execute(self.slot.top_level_slot, self.slot.subindex, self.roi, destination)
  File ".../ilastik/lazyflow/operator.py", line 592, in call_execute
    return self.execute(slot, subindex, roi, result, **kwargs)
  File ".../ilastik/lazyflow/operators/opReorderAxes.py", line 171, in execute
    self.Input(*in_roi).writeInto(result_input_view).wait()
  File ".../ilastik/lazyflow/slot.py", line 1292, in __call__
    return self.get(roi)
  File ".../ilastik/lazyflow/slot.py", line 817, in get
    raise Slot.SlotNotReadyError(msg)
lazyflow.slot.Slot.SlotNotReadyError: Can't get data from slot <class 'lazyflow.operators.opReorderAxes.OpReorderAxes'>.Input yet. It isn't ready. First upstream problem slot is: reorder_lazyflow_source_to_volumina/reorder_lazyflow_source_to_volumina.Input []:    {_ready : False, shape : None, has_mask : None, _dirty : False}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../ilastik/lazyflow/request/request.py", line 383, in _execute
    self._result = self.fn()
  File ".../volumina/volumina/tiling/tileprovider.py", line 403, in _fetch_layer_tile
    img = ims_req.wait()
  File ".../volumina/volumina/pixelpipeline/imagesources/grayscale.py", line 59, in wait
    return self.toImage()
  File ".../volumina/volumina/pixelpipeline/imagesources/grayscale.py", line 65, in toImage
    a = self._arrayreq.wait()
  File ".../volumina/volumina/pixelpipeline/slicesources.py", line 51, in wait
    return self._sp(self._ar.wait())
  File ".../volumina/volumina/pixelpipeline/datasources/minmaxsource.py", line 17, in wait
    rawData = self._rawRequest.wait()
  File ".../volumina/volumina/pixelpipeline/datasources/cachesource.py", line 28, in wait
    res = self._rq.wait()
  File ".../volumina/volumina/pixelpipeline/datasources/lazyflowsource.py", line 46, in wrapper
    return func(*args, **kwargs)
  File ".../volumina/volumina/pixelpipeline/datasources/lazyflowsource.py", line 68, in wait
    a = self._req.wait()
  File ".../ilastik/lazyflow/request/request.py", line 564, in wait
    return self._wait(timeout)
  File ".../ilastik/lazyflow/request/request.py", line 592, in _wait
    self._wait_within_request(current_request)
  File ".../ilastik/lazyflow/request/request.py", line 721, in _wait_within_request
    raise RequestError(self.fn) from exc_value
lazyflow.request.request.RequestError: Failed to request data from `reorder_lazyflow_source_to_volumina.Output`

I could verify that the involved OpReorderAxis is not connected anymore once this fires.
I am not 100% sure what happens, but it looks like the LayerViewerGui is shown (with a valid layerstack) and data is being requested. However, showEvent in turn triggers another setupLayers call. Volumina might be requesting data during updateAllLayers calls. This could result in layers being deleted during requests - the reorder ops in volumina are cleaned- up and are not connected to anything anymore - the requests are out though.

Not sure the fix here is the fix, but so far I couldn't find any undesirable side effects, and it makes sense to be a bit more lazy and create layers only once the UI is shown.

ref: ilastik/volumina#307 (would not be visible without it)

Checklist

• Format code and imports.
• Add docstrings and comments.
• Add tests.
• Update user documentation.
• Reference relevant issues and other pull requests.
• Rebase commits into a logical sequence.

[codecov]

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 55.75%. Comparing base (1df74b8) to head (7ff81e6).

Files	Patch %	Lines
ilastik/workflows/carving/carvingGui.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2858      +/-   ##
==========================================
- Coverage   55.95%   55.75%   -0.20%     
==========================================
  Files         533      533              
  Lines       62300    62302       +2     
  Branches     8549     8550       +1     
==========================================
- Hits        34859    34738     -121     
- Misses      25684    25828     +144     
+ Partials     1757     1736      -21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[btbest]

Oh man how did you discover this showEvent? Did you suss out under what circumstances that function is called? It has a call to updateAllLayers without the isVisible and call_when_setup_finished safeguard, so looks like an easy source of race conditions. At the same time, the _need_update flag seems to be implemented specifically for this function to know whether setNeedUpdate has been called but updateAllLayers was delayed.

[k-dominik]

Well, when the exception occurred, I noticed that the operator that volumina is pulling from is not connected to anything - so I checked out how this can happen (volumina.pixelpipeline.datasources.lazyflowsource.LazyflowSource.clean_up). From there it was more like finding when this happens, and I found that updateAllLayers could cause that. And from there I traced the places this gets called and found the one on _after_init combined with the one during showEvent suspicious enough...

Does this by any chance help with the scale switching?

[btbest]

Possibly; switching scale instantiates a new layerViewerGui, so probably showEvent is triggered as well as _after_init. At the same time, I don't remember running into calls coming from showEvent while I was messing with break points inside updateAllLayers...

[k-dominik]

Allright - I'll merge this then and if we encouter anything weird with layers is happening, this should be one of our suspects :)