Big Images_From_ROIs

[will-moore]

This allows the Images_From_ROIs.py script to create tiled images from 'Big' ROIs. With contributions from @drmatthews.
See https://www.openmicroscopy.org/community/viewtopic.php?f=6&t=7565

To test:

• create Rectangle ROIs of various sizes on a Big image. Some should be bigger than ~3k x 3k and some smaller
• Run the Images_From_ROIs script on the image. It will take a long time, but should create Big images for Big Rectangles and regular non-tiled images for the smaller Rectangles.

[pwalczysko]

Took 8kx8k.jpeg and drawn 3 ROIs almost as big as 8kx8k and 3 smaller rois, hopefully under 3k.
After I ran the script, it ended up very quickly with

Traceback (most recent call last):
  File "./script", line 522, in <module>
    runAsScript()
  File "./script", line 511, in runAsScript
    robj, message = makeImagesFromRois(conn, parameterMap)
  File "./script", line 424, in makeImagesFromRois
    newImage, newDataset, link = processImage(conn, iId, parameterMap)
  File "./script", line 325, in processImage
    description, r)
UnboundLocalError: local variable 'description' referenced before assignment

Edit: From the other window accessible after a script has been run I get:

{'Make_Image_Stack': False, 'IDs': [21900L], 'Data_Type': 'Image', 'Container_Name': 'From_ROIs'}

Cropping ROI (x, y, w, h) (256, 128, 7776, 8000) to be within image. New ROI: (256, 128, 7744L, 7872L)

Cropping ROI (x, y, w, h) (64, 0, 8032, 8000) to be within image. New ROI: (64, 0, 7936L, 8000)
rois
[(320, 288, 7456, 7520, 0L, 0L, 0L, 0L), (4800, 992, 1472, 1152, 0L, 0L, 0L, 0L), (256, 128, 7744L, 7872L, 0L, 0L, 0L, 0L), (64, 0, 7936L, 8000, 0L, 0L, 0L, 0L), (1344, 2944, 1152, 1024, 0L, 0L, 0L, 0L), (4800, 5408, 704, 672, 0L, 0L, 0L, 0L)]
  ROI x: 320 y: 288 w: 7456 h: 7520 z1: 0 z2: 0 t1: 0 t2: 0
Script timer = 0.258255958557 secs

[will-moore]

Fixed that bug and uploaded new script to eel.

[pwalczysko]

Works now as expected.

[jburel]

@will-moore:

• Any particular reason not to use the same tile size that we use server side i.e. 256, in the script tileWidth = 1024
• create_image_from_tiles should probably be more suitable in as a "utils" method.

[will-moore]

@jburel The choice of tile size 1024 means that we have 16x fewer calls to setTile() than if we use 256 and this was the value chosen by @drmatthews in his original script.

E.g. if we have an ROI of 10k x 10k for RGB image that's 300 calls to setTile() with a 1k x 1k tile but 4800 calls to setTile() with a 256 tile.
I'll try a little to see what speed difference it makes overall...

[will-moore]

Approx 10k * 10k image creation took: 221.9 seconds with 1k tile image-26101 and, and 557.9 seconds with 256 tile image-26104.
The viewing of these images in Insight is a little "nicer" with 256 tiles than with 1k tiles but not so much difference between them in web. Not sure that it's faster overall with 256 tiles but feels more responsive.
However, a doubling of the creation time with 256 tiles may be a blocker, especially for really big images. When I tried this with a 47k * 47k image it never completed, even with 1k tiles: https://cowfish.openmicroscopy.org/webmerge/webclient/?show=image-26012

[jburel]

how about the loading time for viewing?

[will-moore]

I don't think the total time is much slower with 1k tiles, but it maybe feels a bit less responsive because there's a bigger wait for fewer tiles, rather than lots of smaller tiles coming quicker.
Probably notice it more with smaller viewport.

[jburel]

When I tried this with a 47k * 47k image it never completed, even with 1k tiles: https://cowfish.openmicroscopy.org/webmerge/webclient/?show=image-26012
In that case we should offer the option to specify the tile size as a parameter i.e. tilesizeX and tileSizeY

[will-moore]

@jburel I had thought of that already. The only tricky thing is to give users enough info in the script launch dialog for them to make an informed choice of tile size. For non-big images the extra parameter will be confusing. Also, if the image is too big 47k * 47k image it never completed, even with 1k tiles then it makes no difference what tile size they choose.
However, I can add a tile size parameter (I don't think we need to support X and Y - no need for rectangular tiles) if you think this is useful and will be clear enough to users?

[jburel]

I think adding option to set the tile size will be useful
Also indicating I will add in the description that the script will not complete for very large images

[will-moore]

@jburel Added 'Tile Size' parameter and updated description. I didn't say "Script will fail for large images" since we don't know how large is large and it will also depend on your setup. The failure for the image above was on my laptop - possibly because it was closed or went to sleep etc. On a proper server we don't know what the limits are. Some people run scripts for days!

[jburel]

flake8.main.application MainProcess 1783 INFO Reporting errors ./omero/util_scripts/Images_From_ROIs.py:60:80: E501 line too long (82 > 79 characters)

[jburel]

so only square tiles will be supported. Just wondering if we should not support non square tiles

[will-moore]

Unless we have any reason to support rectangular tiles, let's not increase the complexity of the script unnecessarily.

[jburel]

something to check during testing too since this is using RPSTileLoop
ome/openmicroscopy#4846

[jburel]

Could you remove the commented lines?
tile2d = mktile(z, c, t, x, y,tileWidth, tileHeight) tile2d = faketile(tileWidth, tileHeight)

[jburel]

eel user-1
Image ID 1475 (big image)
The image has a rectangular ROI, I used the default tile size. The value is bigger that the size of the roi. The message returns No rectangle ROIs found

[will-moore]

@jburel Hmmm - works for me!

[jburel]

I tried again with the same image and another one. It works.
Unless I was mislead by the measurement tool.

I will do further testing

[jburel]

Everything seems to work, I will keep an eye on that strange behaviour that I noticed.

[jburel]

--rebased-to #127