Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure gmic display works in Jupyter #63

Open
myselfhimself opened this issue Sep 21, 2020 · 16 comments
Open

Ensure gmic display works in Jupyter #63

myselfhimself opened this issue Sep 21, 2020 · 16 comments
Assignees
Labels
bug Something isn't working enhancement New feature or request

Comments

@myselfhimself
Copy link
Owner

Nice to have - result image display

@myselfhimself
Copy link
Owner Author

myselfhimself commented Sep 21, 2020

Possibly:

  1. detect non-interactivity / IPython environment
  2. use unconfined IPython display as in this StackOverflow answer using either a temporary file address or a base-64-encoded image.

2. tested successfully on a Binder Jupyter.

@myselfhimself
Copy link
Owner Author

myselfhimself commented Sep 21, 2020

Jupyter detection techniques:
https://gist.github.com/DIYer22/5cc015dcef53d62c16bdff0f8f345e96

In short, detecting a gui (be it web or Qt) boils down to: 'ipykernel' in sys.modules == True. Tested in jupyter qtconsole (True), jupyter online (True), Google Colab (True) and ipython (False)

@myselfhimself
Copy link
Owner Author

We could, depending on availability on the client's Jupyter notepad:

  1. use the matplotlib interactive viewer if available... without using the to_numpy_array() converter for now, but just an image output (b64 or temp file)
  2. use IPython's image display

@myselfhimself
Copy link
Owner Author

"DISPLAY" key existence-testing in sys.environ helps to know if gmic.run("display") will fail display anything for sure (tested) https://stackoverflow.com/a/8258144/420684

@myselfhimself
Copy link
Owner Author

The pure-Python implementation could be like:

def gmic_run(command, images):
    # image processing .. changing 'images' in place
    if "DISPLAY" not in sys.environ:
        for pos, image in enumerate(images):
            #get image base64 or have it saved to temp files in the same gmic.run() run or another one
            try:
                import matplotlib.pyplot
                if image_names and pos < len(image_names):
                    pyplot.title(image_names[pos]) #use gmic image names if available
                #save image or make it b64
                pyplot.show(image_saved_or_b64)
            except:
                try:
                    from IPython.core.display import Image, display
                    display(Image('image_path...', unconfined=True))

Will implement something not today

@myselfhimself
Copy link
Owner Author

myselfhimself commented Sep 22, 2020

the "display" command within a jupyter / ipython / X-less environment will (now just with a nodisplay=True flag in gmic.run.. because my computer has display) dump images to local temporary directory under unique id png filenames and display with:

  1. matplotlib - if available
  2. ipython.core.display() - if available

This implementation with temporary files instead of base64 has yet to be bulletproved for web interfaces... I shall try with jupyter in a local web browser.

calling display multiple times in a same command works OK and all generated images are used for display
G'MIC provides display without parameters (supported), with a prepended dash -display (unsupported) but also with parameters (unsupported, would error), display3d (unsupported for ipython/jupyter and will result in a parsing error for now)..

My goal here is not to do a full fledge support of all ways to reinterpret the display command... I might introduce a flag for auto-display as well, so that any final image gets displayed without having to use a display command as well...

gmic-display-jupyter-ipython-support

@myselfhimself
Copy link
Owner Author

Coming soon: matplotlib support - inline or with the qt5 widget..
image

@myselfhimself
Copy link
Owner Author

Matplotlib support is now there with multiple images working:
image

@myselfhimself
Copy link
Owner Author

myselfhimself commented Sep 23, 2020

  • test in python console
  • test in ipython console
    • prevent wurlitzer macro non-enabling message on import gmic
  • test in jupyter qtconsole
  • test in jupyter notebook (local)
  • test in jupyter notebook (online)
  • test in google colab (online)

@myselfhimself
Copy link
Owner Author

Now works well without any command to write for UNIX users. See #64 (comment)

myselfhimself added a commit that referenced this issue Sep 25, 2020
…module start, fix display-less output images order, better messages
myselfhimself added a commit that referenced this issue Sep 28, 2020
@myselfhimself
Copy link
Owner Author

In Google Colab, after the 2.9.1-alpha1 release:
image

@myselfhimself
Copy link
Owner Author

In Jupyter Lab online:
gmic_jupyterlab

@myselfhimself
Copy link
Owner Author

A pure ipython console, in a non-web setup, with DISPLAY environment variable unset, does not need to use wurlitzer to show console output, still there is a warning on G'MIC import, which could be omitted or clarified if we detected a pure IPython console.

The G'MIC display is used instead of any matplotlib with DISPLAY set, and there is not display at all, if DISPLAY is unset.
image

Adding TODO checkbox above for fine-tuning this import-time message regarding wurlitzer in IPython console.

@myselfhimself
Copy link
Owner Author

Wurlitzer's code shows that when IPython has no kernel available, we are in within a terminal Python context, and wurlitzer itself is useless.

@myselfhimself
Copy link
Owner Author

Good enough, will not be automatically tested, in order to give more time for other tasks

@myselfhimself
Copy link
Owner Author

myselfhimself commented Oct 6, 2020

@myselfhimself myselfhimself reopened this Oct 6, 2020
@myselfhimself myselfhimself added bug Something isn't working enhancement New feature or request labels Apr 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant