Export figure without script

[will-moore]

Export Figures without scripting service

Install now doesn't require any server-side script or dependencies.
Figure export to TIFF/PDF/zip or OMERO image happens during a single AJAX POST request (attaching files to images as before) and JSON returns IDs etc to enable download / link.
Instead of writing images and files to disk (as in the script) we now read/write to file objects or use a temp dir to compile contents of zip file.

To test: Needs to fully test figure export:

• Create figure (not in default group) with a few images, ROIs, scalebars & labels.
• Export as PDF, TIFF (exports single file)
• Export as PDF-with Images, TIFF with Images (exports zip)
• Export as OMERO image.
• In each case, should get a link to download file/zip or show OMERO image
• Also test export of multi-page Figure:
  • TIFF export should export as zip, even "without Images"
  • Export to OMERO image should give link to multiple images.

TODO:

• Error handling - show error button with link to show error text

[jburel]

any cap for the version of reportlab and markdown?
I remember us having issue with some version and numpy.

[will-moore]

I just copied these over from requirements-server.txt.

[jburel]

since we were using yum install python-reportlab for example to install the dependency on the server and not pip, the version installed via pip will be more recent so we will need to watch for potential issues
requirements-server was never used.

[jburel]

you will need to remove omego

[jburel]

What about the font for legend?

[will-moore]

Good catch! Fonts added.

[will-moore]

Running this on web-dev-merge now gives me error "ImportError: Need to install https://bitbucket.org/rptlab/reportlab"

[jburel]

requirements are not installed
You will need to modify setup.py and use install_requires parameter to install the dependencies
You can use https://github.com/openmicroscopy/omero-marshal/blob/master/setup.py as an exmple

[will-moore]

https://ci.openmicroscopy.org/view/Failing/job/WEB-DEV-trial-deploy/405/console
Failing with

Cloning git://github.com/snoopycrimecop/omero-figure.git (to master/merge/daily) to /tmp/pip-build-8NORjU/omero-figure
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-8NORjU/omero-figure/setup.py", line 29, in <module>
        import omero_figure.utils as utils
      File "omero_figure/utils.py", line 22, in <module>
        from PIL import Image
    ImportError: No module named PIL

[will-moore]

@chris-allan points out that long-running http requests are going to cause problems tying up workers.
So unless there's some other approach we can take, it looks like this is not going to work.

[chris-allan]

I'm not 100% clear on what the goal was here but if the main reason was to improve dependency management and aid installability I don't see a reason not to continue this PR. Having all the dependencies in a requirements file with setuptools integration is certainly in keeping with all our other work. As is having the export code itself available as a module.

The trend is to have OMERO.web plugins be installable via pip anyway and copying an OMERO.script into an installation is far from difficult. Furthermore, using a virtualenv is part of all of our installation documentation now. Just having a small shim OEMRO.script that can be installed, potentially even by a setuptools entry point, is certainly not a bad thing in my view.

[joshmoore]

A potential next time to examine this is when the MQ is in place. If the mechanism for registering a new job type is also pip installable, then we can free ourselves from the script dependency.

cc: @c0c0n3

[jburel]

Copying the script is one step.
I think we are mixing 2 things: figure dependencies (installed in virtualenv) and the script dependencies (usually not in virtualenv).
The virtualenv installation strategy used to install web is not used in that case. The script dependencies (reportlab and markdown) and font have to be installed server side, usually not in virtualenv (the only instructions page we have for server in virtual env is for C6 py27)
For dependencies server side, we have moved away from installing via pip since it had some major side effect e.g. Numpy/pillow saga (we had to cap to an oldish version for a long time)
There are still some benefits from moving away from the current implementation using another approach.
For example to use all the options of the latest released version of figure, it requires OMERO 5.3.2 due to a font that needs to be installed in OMERO
The font could be added manually (not tested) but that will be another manual step to document

[manics]

Can figure still be run standalone e.g. on a different machine to OMERO.web? If so I think there's value in this, e.g. if we wanted to add figure to the IDR we wouldn't want to run scripts on the OMERO.server backend, but we could install figure on an isolated VM so the impact of long-running requests would be isolated to just figure users.

[chris-allan]

Figure is just an OMERO.web plugin; so yes, of course it can. That is provided you are using Redis backed sessions and have both OMERO.web instances configured to use the same Redis server instance. You would then need to set up the correct redirects and/or transparent proxying in nginx.

This deployment strategy is exactly how the microservice integration is currently documented.

[will-moore]

As discussed with @jburel - We could add the Figure_To_Pdf.py script to the OMERO scripts so it will always be on the server. The export functionality could go in a separate pip-installable package that is used by the script or other python code. The script wouldn't have to updated/released unless the inputs/outputs change.

[manics]

Long term it'd be ideal if we had a scripting/processing service that's independent of OMERO, then you'd be able to run figure on a standalone OMERO.web against a public/read-only OMERO.server. Or make the Figure_To_Pdf.py script into a micro-service.