RunningDeepZoomTiler not working properly with multiple workers

[erexhepa]

Hi,

I was experiencing some problems with DeepZoomTiler python wrapper. I have installed the python package of openslide and running it from Ubuntu16.

Not sure if it is an issue related to a digital pathology particular format, in this case SVS. What happens is that when in multithread mode with multiple workers; some tiles are not generated the workers and the workers hang up. There is an IO error exception raised by the tile reader which if catched and ignored you can get all thread finish but you miss a lot of tiles more or less randomy distributed in the zoom levels.

If you run with only one worker you manage to produce all tiles correctly. Just wanted to raise the issue in case it wasn't known but otherwise was wondering if someone has found a solution ?

Elton

[bgilbert]

Hi @erexhepa, could you provide more details? Are you using DeepZoomGenerator in your own code, running deepzoom_tile.py from the command line, or something else? How are you invoking the tiling code? What is the exception message and backtrace?

[erexhepa]

Hi @bgilbert,

I'm indeed using the DeepZoomGenerator in the deepzoom_tile.py code which is then called from the command line.

The tiling code goes through DeepZoomStaticTiler which is first called (run method) and then the tiling process is handled from inside the DeepZoomStaticTiler class. It's the same code as the one in the git repo.

I can't remember the exception message because I'm running some jobs at the moment (calling several tiling processes manually through the command line works and is an alternative to the issue above) and I'm avraid will interfere with the ongoing analysis. It was an IO error raised by the TIFF library used to read the tiff inside the hierarchical structure of TIFF inside the SVS.

Thanks for your help

[erexhepa]

https://github.com/openslide/openslide-python/blob/master/examples/deepzoom/deepzoom_tile.py

this is the version of deepzoom I'm running with a few modifications in the main to be manage the multiple threads called at the command line.

Elton

[bgilbert]

Okay. When you have it, please post the exception message and traceback, as well as the command line you used to run deepzoom_tile.py. It would also be helpful to know whether you can reproduce the problem with an unmodified deepzoom_tile.py.

[erexhepa]

Hi,

here is the error reproduced with the unmodified version of the deepzoom_tile.py

Process TileWorker:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/opt/CellProfiler/cellprofiler/modules/wsideepzoom.py", line 66, in run
    tile = dz.get_tile(level, address)
  File "/usr/lib/python2.7/dist-packages/openslide/deepzoom.py", line 142, in get_tile
    tile = self._osr.read_region(*args)
  File "/usr/lib/python2.7/dist-packages/openslide/__init__.py", line 219, in read_region
    level, size[0], size[1])
  File "/usr/lib/python2.7/dist-packages/openslide/lowlevel.py", line 222, in read_region
    _read_region(slide, buf, x, y, level, w, h)
  File "/usr/lib/python2.7/dist-packages/openslide/lowlevel.py", line 159, in _check_error
    raise OpenSlideError(err)
OpenSlideError: Premature end of JPEG file

Elton

[RandomForest5a]

hey have u figure it out？

[erexhepa]

nope I couldn't find why it does that but I finally decided to handle the parallelization myself by using multiple threads. This way it works fine.

[RandomForest5a]

Me too. Here is what I did. I modify the source code in openslide, shut down the error arising code. It seems fine with only showing warning. Cheers.

…

On Mar 2, 2017 17:15, "Elton Rexhepaj" ***@***.***> wrote: nope I couldn't find why it does that but I finally decided to handle the parallelization myself by using multiple threads. This way it works fine. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#183 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATLVKGBzC_rAIDYOmIg6uMsxPy1ifzv6ks5rhl6PgaJpZM4KJsbE> .

[erexhepa]

Yes but be careful ! My understanding is that the error rises due to parallel processes (TO BE VERIFIED) extracting tiles at different resolution depths but effectively working on the same resolution, so one of the processes goes out of bound with regard to the X,Y coordinates. I didn't dig more because the parallelisation via multithreating of several slides at the same time (rather then multithreading of one slide) seems to work and the decompositions are also correct.

[RandomForest5a]

I gonna think about it carefully cheers！

…

On Mar 3, 2017 19:05, "Elton Rexhepaj" ***@***.***> wrote: Yes but be careful ! My understanding is that the error rises due to parallel processes (TO BE VERIFIED) extracting tiles at different resolution depths but effectively working on the same resolution, so one of the processes goes out of bound with regard to the X,Y coordinates. I didn't dig more because the parallelisation via multithreating of several slides at the same time (rather then multithreading of one slide) seems to work and the decompositions are also correct. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#183 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATLVKKtbvHo7CMODvTU2ZZ-F-TV8r61lks5rh8m2gaJpZM4KJsbE> .