`matplotlib` defaults to non-interactive backend, causing `UserWarning` for `plt.show()` #3392

joshuacwnewton · 2021-05-17T17:06:06Z

The easiest way to demonstrate is using sct_dmri_display_bvecs:

--
Spinal Cord Toolbox (git-master-a62fb672c602b9863a8c03521eb272bfd7bad9e1)

sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt
--

Saving figure: bvecs.png

/home/joshua/repos/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_dmri_display_bvecs.py:137: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()

This warning occurs and no figure is shown.

We explicitly set the backend type using the MPLBACKEND environment variable. (Option 2. in this list.)

spinalcordtoolbox/install_sct

Lines 266 to 272 in a62fb67

    
           if [[ "$THE_RC" == "bash" ]]; then 
        
             echo "export SCT_DIR=$SCT_DIR" 
        
             echo "export MPLBACKEND=Agg" 
        
           elif [[ "$THE_RC" == "csh" ]]; then 
        
             echo "setenv SCT_DIR $SCT_DIR" 
        
             echo "setenv MPLBACKEND Agg" 
        
           fi

We started using the Agg backend in #2157 (cd521db) to avoid crashes on headless systems (see #2138).

Moving forward, I can think of two options, depending on whether we need interactive matplotlib windows in SCT.

If we want interactive windows, we should figure out how to set MPLBACKEND=Agg (a non-interactive backend) for headless systems only.
If we don't care about interactive windows, and instead only want to generate images to disk, then we should remove any instances of plt.show().

The text was updated successfully, but these errors were encountered:

jcohenadad · 2021-05-17T18:31:50Z

Thank you for opening this @joshuacwnewton

In the case of sct_dmri_display_bvecs I we can go without interactive figure, and having it work would be like the cherry on the cake. Maybe we can check if user is on headless system, and if that's the case interactive mode is turned off?

joshuacwnewton · 2021-05-17T20:56:16Z

Maybe we can check if user is on headless system, and if that's the case interactive mode is turned off?

The interesting thing is, I think we already do check if the user is on a headless system... but in the launcher:

spinalcordtoolbox/spinalcordtoolbox/compat/launcher.py

Lines 20 to 22 in a62fb67

    
           if "DISPLAY" not in os.environ: 
        
               # No DISPLAY, set suitable default matplotlib backend as pyplot is used 
        
               env["MPLBACKEND"] = "Agg"

So we're setting the same environment variable twice: once in the launcher, and once in the user's RC file (via the installer), which seems redundant. It's possible that this was accidental, and that just the check in the launcher will be sufficient (as you've suggested).

valosekj · 2021-05-18T05:34:09Z

We explicitly set the backend type using the MPLBACKEND environment variable. (Option 2. in this list.)

spinalcordtoolbox/install_sct

Lines 266 to 272 in a62fb67

if [[ "$THE_RC" == "bash" ]]; then

echo "export SCT_DIR=$SCT_DIR"

echo "export MPLBACKEND=Agg"

elif [[ "$THE_RC" == "csh" ]]; then

echo "setenv SCT_DIR $SCT_DIR"

echo "setenv MPLBACKEND Agg"

fi

~~Interesting, there is missing elif statement for zsh. Is that the intention?~~

EDIT - Sorry, I have not noticed, that it is due to:

spinalcordtoolbox/install_sct

Lines 220 to 221 in a62fb67

    
           elif [[ "$SHELL" == *"zsh"* ]]; then 
        
             THE_RC="bash"

BTW.
I thought that export MPLBACKEND=TkAgg run in my zsh CLI could solve the issue with plt.show(), so I tried:

git checkout master
export MPLBACKEND=TkAgg
sct_dmri_display_bvecs -bvec ~/sub-001/dwi/sub-001_dwi.bvec

But it didn't and I still get the same warning:

UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure. plt.show()

joshuacwnewton · 2021-05-20T19:04:33Z

BTW.
I thought that export MPLBACKEND=TkAgg run in my zsh CLI could solve the issue with plt.show(), so I tried:
git checkout master
export MPLBACKEND=TkAgg
sct_dmri_display_bvecs -bvec ~/sub-001/dwi/sub-001_dwi.bvec
But it didn't and I still get the same warning:

UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure. plt.show()

Interesting. I tried this just now, and your steps work for me:

> echo $MPLBACKEND
Agg
> sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt

--
Spinal Cord Toolbox (git-master-9a4b45715240d6c08b8d71587ffbe99a41326cc6*)

sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt
--

Saving figure: bvecs.png

/home/joshua/repos/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_dmri_display_bvecs.py:137: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()
> export MPLBACKEND=TkAgg
> echo $MPLBACKEND
TkAgg
> sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt

--
Spinal Cord Toolbox (git-master-9a4b45715240d6c08b8d71587ffbe99a41326cc6)

sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt
--

Saving figure: bvecs.png

I'm also using zsh (in PyCharm, on Ubuntu 20.04).

valosekj · 2021-05-21T06:17:27Z

Interesting. I tried this just now, and your steps work for me:

Hm, I have just replicated your steps and still get backend warning, although backend looks changed to TkAgg in my CLI:

$ git checkout master
$ echo $MPLBACKEND
Agg
$ sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec

--
Spinal Cord Toolbox (git-master-a62fb672c602b9863a8c03521eb272bfd7bad9e1)

sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec
--

Saving figure: bvecs.png

/usr/local/sct/spinalcordtoolbox/scripts/sct_dmri_display_bvecs.py:137: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()


$ export MPLBACKEND=TkAgg 
$ echo $MPLBACKEND
TkAgg
$ sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec

--
Spinal Cord Toolbox (git-master-a62fb672c602b9863a8c03521eb272bfd7bad9e1)

sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec
--

Saving figure: bvecs.png

/usr/local/sct/spinalcordtoolbox/scripts/sct_dmri_display_bvecs.py:137: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()

I tried this in my Terminal using zsh with oh-my-zsh extension on MacOS Catalina (10.15.7).

But, interesting is that for another script which uses plt.show() (mrs_vis from FSL package) the backend change works:

see here

$ conda activate fsl_mrs
$ echo $MPLBACKEND
Agg
$ mrs_vis svs.nii.gz
Performing coil combination
Averaging DIM_DYN
/Users/valosek/miniconda/bin/mrs_vis:100 : UserWarning:

Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.        


$ export MPLBACKEND=TkAgg
$ echo $MPLBACKEND       
TkAgg       
$  mrs_vis svs.nii.gz
Performing coil combination
Averaging DIM_DYN

valosekj · 2021-05-21T08:22:08Z

Another interesting thing -- I tried to add following two lines into master sct_dmri_display_bvecs.py:

import matplotlib
print(f'Backend in python: {matplotlib.get_backend()}')

And I have found out that even if I run in my Terminal export MPLBACKEND=TkAgg, the backend in python is still agg:

$ export MPLBACKEND=TkAgg                                                                                                                                              /0,2s
$ echo $MPLBACKEND       
TkAgg                                                                                                                                                                                                  /0,1s
$ sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec

--
Spinal Cord Toolbox (git-master-a62fb672c602b9863a8c03521eb272bfd7bad9e1*)

sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec
--

Saving figure: bvecs.png

Backend in python: agg
/usr/local/sct/spinalcordtoolbox/scripts/sct_dmri_display_bvecs.py:139: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()

joshuacwnewton · 2021-05-21T12:49:41Z

That's very curious, hrm. Trying what you suggest, I still see it working for me:

Output of print(f'Backend in python: {matplotlib.get_backend()}')

> echo $MPLBACKEND
Agg
> sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt

--
Spinal Cord Toolbox (git-master-77d4b7ddc716e69b1dc223536a7c026ff12abad8*)

sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt
--

Saving figure: bvecs.png

Backend in python: agg
/home/joshua/repos/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_dmri_display_bvecs.py:139: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()
> export MPLBACKEND=TkAgg
> echo $MPLBACKEND
TkAgg
> sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt

--
Spinal Cord Toolbox (git-master-77d4b7ddc716e69b1dc223536a7c026ff12abad8*)

sct_dmri_display_bvecs -bvec sct_example_data/dmri/bvecs.txt
--

Saving figure: bvecs.png

Backend in python: TkAgg

But, interesting is that for another script which uses plt.show() (mrs_vis from FSL package) the backend change works:

If it's SCT-specific, then perhaps it has to do with our check for DISPLAY that overrides whatever the current MPLBACKEND value is:

spinalcordtoolbox/spinalcordtoolbox/compat/launcher.py

Lines 20 to 22 in 77d4b7d

    
           if "DISPLAY" not in os.environ: 
        
               # No DISPLAY, set suitable default matplotlib backend as pyplot is used 
        
               env["MPLBACKEND"] = "Agg"

Could you run sct_check_dependencies to double-check? On my machine, DISPLAY is set:

Output of sct_check_dependencies

> sct_check_dependencies

--
Spinal Cord Toolbox (git-master-77d4b7ddc716e69b1dc223536a7c026ff12abad8*)

sct_check_dependencies 
--

SCT info:
- version: git-master-77d4b7ddc716e69b1dc223536a7c026ff12abad8*
- path: /home/joshua/repos/spinalcordtoolbox
OS: linux (Linux-5.8.0-53-generic-x86_64-with-debian-bullseye-sid)
CPU cores: Available: 8, Used by ITK functions: 8
RAM: Total: 15863MB, Used: 4336MB, Available: 10626MB
Check Python executable.............................[OK]
  Using bundled python 3.6.13 |Anaconda, Inc.| (default, Feb 23 2021, 21:15:04) 
[GCC 7.3.0] at /home/joshua/repos/spinalcordtoolbox/python/envs/venv_sct/bin/python
Check if data are installed.........................[OK]
Check if colored is installed.......................[OK] (1.4.2)
Check if dipy is installed..........................[OK] (1.4.0)
Check if futures is installed.......................[OK]
Check if h5py is installed..........................[OK] (2.10.0)
Check if Keras (2.1.5) is installed.................[OK] (2.1.5)
Check if ivadomed is installed......................[OK] (2.7.4)
Check if matplotlib is installed....................[OK] (3.3.4)
Check if nibabel is installed.......................[OK] (3.2.1)
Check if numpy is installed.........................[OK] (1.19.5)
Check if onnxruntime (1.4.0) is installed...........[OK] (1.4.0)
Check if pandas is installed........................[OK] (1.1.5)
Check if psutil is installed........................[OK] (5.8.0)
Check if pyqt5 (5.11.3) is installed................[OK] (5.11.3)
Check if pytest is installed........................[OK] (6.2.3)
Check if pytest-cov is installed....................[OK] (__version__ = '2.11.1')
Check if raven is installed.........................[OK]
Check if requests is installed......................[OK] (2.25.1)
Check if requirements-parser is installed...........[OK] (0.2.0)
Check if scipy is installed.........................[OK] (1.5.4)
Check if scikit-image is installed..................[OK] (0.17.2)
Check if scikit-learn is installed..................[OK] (0.24.1)
Check if tensorflow (1.5.0) is installed............[OK] (1.5.0)
Check if torch (1.5.0+cpu) is installed.............[WARNING] (1.5.0 != 1.5.0+cpu mandated version))
Check if torchvision (0.6.0+cpu) is installed.......[WARNING] (0.6.0 != 0.6.0+cpu mandated version))
Check if xlwt is installed..........................[OK] (1.3.0)
Check if tqdm is installed..........................[OK] (4.60.0)
Check if transforms3d is installed..................[OK] (0.3.1)
Check if urllib3 is installed.......................[OK] (1.26.4)
Check if pytest_console_scripts is installed........[OK]
Check if wquantiles is installed....................[OK] (0.4)
Check if spinalcordtoolbox is installed.............[OK]
Check ANTs compatibility with OS ...................[OK]
Check PropSeg compatibility with OS ................[OK]
Check if DISPLAY variable is set....................[OK]
Check if figure can be opened with PyQt.............[OK]

valosekj · 2021-05-22T06:10:18Z

Could you run sct_check_dependencies to double-check? On my machine, DISPLAY is set:

Sure, below is the output of sct_check_dependencies. DISPLAY variable is not set on my machine:

Output of `sct_check_dependencies`

$ sct_check_dependencies

--
Spinal Cord Toolbox (git-master-a62fb672c602b9863a8c03521eb272bfd7bad9e1*)

sct_check_dependencies 
--

SCT info:
- version: git-master-a62fb672c602b9863a8c03521eb272bfd7bad9e1*
- path: /usr/local/sct
OS: osx (Darwin-19.6.0-x86_64-i386-64bit)
CPU cores: Available: 4, Used by ITK functions: 4
RAM: Total: 8192MB, Used: 5039MB, Available: 2630MB
Check Python executable.............................[OK]
  Using bundled python 3.6.12 |Anaconda, Inc.| (default, Sep  8 2020, 17:50:39) 
[GCC Clang 10.0.0 ] at /usr/local/sct/python/envs/venv_sct/bin/python
Check if data are installed.........................[OK]
Check if colored is installed.......................[OK] (1.4.2)
Check if dipy is installed..........................[OK] (1.2.0)
Check if futures is installed.......................[OK]
Check if h5py is installed..........................[OK] (2.10.0)
Check if Keras (2.1.5) is installed.................[OK] (2.1.5)
Check if ivadomed is installed......................[OK] (2.0.2)
Check if matplotlib is installed....................[OK] (3.3.0)
Check if nibabel is installed.......................[OK] (3.1.1)
Check if numpy is installed.........................[OK] (1.19.1)
Check if onnxruntime (1.4.0) is installed...........[OK] (1.4.0)
Check if pandas is installed........................[OK] (1.0.5)
Check if psutil is installed........................[OK] (5.7.2)
Check if pyqt5 (5.11.3) is installed................[OK] (5.11.3)
Check if pytest is installed........................[OK] (6.1.0)
Check if pytest-cov is installed....................[OK] (__version__ = '2.10.1')
Check if raven is installed.........................[OK]
Check if requests is installed......................[OK] (2.24.0)
Check if requirements-parser is installed...........[OK] (0.2.0)
Check if scipy is installed.........................[OK] (1.5.1)
Check if scikit-image is installed..................[OK] (0.17.2)
Check if scikit-learn is installed..................[OK] (0.23.1)
Check if tensorflow (1.5.0) is installed............[OK] (1.5.0)
Check if torch (1.5.0) is installed.................[OK] (1.5.0)
Check if torchvision (0.6.0) is installed...........[OK] (0.6.0)
Check if xlwt is installed..........................[OK] (1.3.0)
Check if tqdm is installed..........................[OK] (4.48.0)
Check if transforms3d is installed..................[OK] (0.3.1)
Check if urllib3 is installed.......................[OK] (1.25.10)
Check if pytest_console_scripts is installed........[FAIL]
No module named 'pytest_console_scripts'
Check if wquantiles is installed....................[FAIL]
No module named 'wquantiles'
Check if spinalcordtoolbox is installed.............[OK]
Check ANTs compatibility with OS ...................[OK]
Check PropSeg compatibility with OS ................[OK]
Check if DISPLAY variable is set....................[FAIL]

If it's SCT-specific, then perhaps it has to do with our check for DISPLAY that overrides whatever the current MPLBACKEND value is:

Yeah, it looks like that! I tried to comment out following lines from launcher.py:

spinalcordtoolbox/spinalcordtoolbox/compat/launcher.py

Lines 20 to 22 in 77d4b7d

    
           if "DISPLAY" not in os.environ: 
        
               # No DISPLAY, set suitable default matplotlib backend as pyplot is used 
        
               env["MPLBACKEND"] = "Agg"

And then I run again following code and figure was finally displayed for me!

Output of print(f'Backend in python: {matplotlib.get_backend()}')

$ echo $MPLBACKEND
Agg
$ export MPLBACKEND=TkAgg                                                                                                                                                         
$ echo $MPLBACKEND       
TkAgg
$ sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec

--
Spinal Cord Toolbox (git-master-a62fb672c602b9863a8c03521eb272bfd7bad9e1*)

sct_dmri_display_bvecs -bvec ~/sub-01/dwi/sub-01_dwi.bvec
--

Saving figure: bvecs.png

Backend in python: TkAgg

joshuacwnewton · 2021-05-22T11:48:51Z

And then I run again following code and figure was finally displayed for me!

Hoorah! At least we know that MPLBACKEND does what we hope for, then. 😊

DISPLAY variable is not set on my machine:

Hrm, that's a bit odd. Assuming you are using X Window System (i.e. a UNIX-based system with a display), then DISPLAY should be set automatically when you log in.

Is there anything unique about your environment (e.g. Remote/SSH session)? If so, this SE post might be relevant to you.

valosekj · 2021-05-23T07:04:48Z

Hrm, that's a bit odd. Assuming you are using X Window System (i.e. a UNIX-based system with a display), then DISPLAY should be set automatically when you log in.
Is there anything unique about your environment (e.g. Remote/SSH session)? If so, this SE post might be relevant to you.

I have pretty standard MacBook Pro and I use local terminal (no ssh/remote sessions). Problem can be that MacOS doesn't run X11 by default and X11 is available using XQuartz.

joshuacwnewton · 2021-05-23T12:34:47Z

Problem can be that MacOS doesn't run X11 by default and X11 is available using XQuartz.

Ah, I see. That makes detecting headless macOS machines much more difficult.

We could show all interactive plots on macOS by default, but then SCT may appear to hang if no display is available.

joshuacwnewton added bug category: fixes an error in the code sct_dmri_display_bvecs context: labels May 17, 2021

joshuacwnewton mentioned this issue May 17, 2021

Allow sct_dmri_display_bvecs.py to display multi-shell acquisition #3390

Merged

8 tasks

joshuacwnewton self-assigned this May 17, 2021

joshuacwnewton mentioned this issue May 22, 2021

Improve how we check $DISPLAY and set $MPLBACKEND on headless systems #3401

Closed

8 tasks

joshuacwnewton mentioned this issue Jun 16, 2021

Stop trying to detect headless systems using DISPLAY, and no longer set MPLBACKEND #3420

Merged

8 tasks

joshuacwnewton closed this as completed in #3420 Jun 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`matplotlib` defaults to non-interactive backend, causing `UserWarning` for `plt.show()` #3392

`matplotlib` defaults to non-interactive backend, causing `UserWarning` for `plt.show()` #3392

joshuacwnewton commented May 17, 2021 •

edited

jcohenadad commented May 17, 2021

joshuacwnewton commented May 17, 2021 •

edited

valosekj commented May 18, 2021 •

edited

joshuacwnewton commented May 20, 2021 •

edited

valosekj commented May 21, 2021 •

edited

valosekj commented May 21, 2021

joshuacwnewton commented May 21, 2021 •

edited

valosekj commented May 22, 2021 •

edited

joshuacwnewton commented May 22, 2021 •

edited

valosekj commented May 23, 2021

joshuacwnewton commented May 23, 2021

matplotlib defaults to non-interactive backend, causing UserWarning for plt.show() #3392

matplotlib defaults to non-interactive backend, causing UserWarning for plt.show() #3392

Comments

joshuacwnewton commented May 17, 2021 • edited

jcohenadad commented May 17, 2021

joshuacwnewton commented May 17, 2021 • edited

valosekj commented May 18, 2021 • edited

joshuacwnewton commented May 20, 2021 • edited

valosekj commented May 21, 2021 • edited

valosekj commented May 21, 2021

joshuacwnewton commented May 21, 2021 • edited

valosekj commented May 22, 2021 • edited

joshuacwnewton commented May 22, 2021 • edited

valosekj commented May 23, 2021

joshuacwnewton commented May 23, 2021

`matplotlib` defaults to non-interactive backend, causing `UserWarning` for `plt.show()` #3392

`matplotlib` defaults to non-interactive backend, causing `UserWarning` for `plt.show()` #3392

joshuacwnewton commented May 17, 2021 •

edited

joshuacwnewton commented May 17, 2021 •

edited

valosekj commented May 18, 2021 •

edited

joshuacwnewton commented May 20, 2021 •

edited

valosekj commented May 21, 2021 •

edited

joshuacwnewton commented May 21, 2021 •

edited

valosekj commented May 22, 2021 •

edited

joshuacwnewton commented May 22, 2021 •

edited