How to run Python scripts in the code folder non-interactively

[Robinlovelace]

For minimal dependency testing. Am thinking of a CI workflow that does not depend on Quarto which is a bit niche and which still has no dedicated quarto package on pypi.

[Robinlovelace]

Two main options:

• With Python script: https://stackoverflow.com/questions/56378285/run-all-python-scripts-in-a-folder-from-python
• With shell script: https://stackoverflow.com/questions/5015316/run-all-python-files-in-a-directory

Plan to go with latter for now as more concise.

[Robinlovelace]

This works fine but seems to get blocked when the plot below is generated.

[Robinlovelace]

This is really bugging me: simply running

python code/chapters/02-spatial-data.py

fails because matplotlib seems to block execution of the script until the plot popup has been manually closed. That seems weird, bad design, or I'm missing something. I have tried to debug this: https://www.google.com/search?channel=fs&client=ubuntu&q=run+python+code+non+interactive+plots+bash

Any ideas?

[michaeldorman]

This is really bugging me: simply running

python code/chapters/02-spatial-data.py

fails because matplotlib seems to block execution of the script until the plot popup has been manually closed. That seems weird, bad design, or I'm missing something. I have tried to debug this: https://www.google.com/search?channel=fs&client=ubuntu&q=run+python+code+non+interactive+plots+bash

Any ideas?

Perhaps we can test the .ipynb files, using jupyter nbconvert --execute --to html 02-spatial-data.ipynb? This should fail if there is an error in any code cell

[Robinlovelace]

I tried this but it has the same problem: figures pop up and it seems execution is paused until they are manually closed. V. annoying feature, unless I have missed something.

[Robinlovelace]

See here my attempt at getting in working. Interested to know if others can reproduce the issue: https://github.com/geocompr/py/blob/main/run-code.sh

[michaeldorman]

I tried this but it has the same problem: figures pop up and it seems execution is paused until they are manually closed. V. annoying feature, unless I have missed something.

I don't see figures popup with jupyter nbconvert locally. The command just executes, creating an HTML file if successful, or prints error otherwise

[Robinlovelace]

Aha, yes that is a good solution. I was conflating ipython with jupyter convert 👍 will add a command that does that now...

[Robinlovelace]

Can you reproduce the issue with python /code/chapters/02*.py etc? Keen to find a way to a solution to that issue but more due to curiosity than necessity: it should be possible!

[Robinlovelace]

Good news, after the commit above this is fixed! Will leave this open for a bit in case anyone knows how to get the .py files running non interactively but basically this one is fixed.

[anisotropi4]

It's good if you have found a fix for this but there it is a known feature in which matplotlib blocks waiting for keyboard input when you generate a plot GUI. There are other things you can do, for example you can pause for a given period of time rather than block , but this is a matplotlib foible based on its use of another-application-to-render-image...

[Robinlovelace]

Any ideas how to do this Will? I tried a number of suggested solutions I found online but none of them worked! Reproducible code examples welcome here. In the meantime I will open a new issue to run the code in an action to pick up on issues like that in #66