Issues with python env. when running a python chunk inside the qmd

[bkostadi]

Bug description

I've been playing on the RStudio Cloud with the matplotlib demo from the Quarto documentation on Using Python - https://quarto.org/docs/computations/python.html. I followed the instructions and I installed jupyter and matplotlib, python 3.8 and numpy are already installed. I rendered the matplotlib demo qmd successfully, however when I tried running the python chunk inside the qmd, I was prompted to update jsonlite and reticulate, then I got the errors:

• there is no package called 'reticulate',
• No non-system installation of Python could be found. Would you like to download and install Miniconda?

I said yes to installing miniconda, but then I got the error message Error creating conda environment 'r-reticulate' [exit code 137]. Despite this error, a new python environment must have been created because I was able to run a python chunk with basic math computations with import math, but I still could not run the chunk based on matplotlib since now a different python environment was being used rather than the jupyter python kernel where I had installed matplotlib.

Finally, I tried to render the qmd file again, but now the rendering failed with the error message:

Starting Jupyter kernel.../home/r157428/.local/lib/python3.8/site-packages/jupyter_client/__init__.py:23: UserWarning: Could not import submodules
  warnings.warn("Could not import submodules")
Traceback (most recent call last):
  File "/opt/quarto/0.9.293/share/jupyter/jupyter.py", line 21, in <module>
    from notebook import notebook_execute, RestartKernel
  File "/opt/quarto/0.9.293/share/jupyter/notebook.py", line 14, in <module>
    from nbclient import NotebookClient
  File "/home/r157428/.local/lib/python3.8/site-packages/nbclient/__init__.py", line 6, in <module>
    from .client import NotebookClient, execute  # noqa: F401
  File "/home/r157428/.local/lib/python3.8/site-packages/nbclient/client.py", line 13, in <module>
    from jupyter_client import KernelManager
ImportError: cannot import name 'KernelManager' from 'jupyter_client' (/home/r157428/.local/lib/python3.8/site-packages/jupyter_client/__init__.py)

The initial quarto check was okay, but the final quarto check was not okay since the initial python environment was replaced by the freshly created conda environment without jupyter.

One quick solution to this issue with the python environments is to select the system python in Tools > Global Options > Python. That way one forces RStudio to use the system python to render the qmd file as well as run the python chunks inside the qmd file. If that was the original design plan then it should really be spelled out in the Quarto documentation. It's also very important to emphasize the role the reticulate R package is playing when running chunks of python code inside the Quarto document.

However, I came across another issue when running two consecutive python code chunks inside the qmd notebook. I added at the end of the matplotlib demo qmd from the Quarto documentation, the following python chunk:

```{python}
x = np.linspace(-2,2,41)
y = np.exp(-x**2) * np.cos(2*np.pi*x)
plt.plot(x,y,alpha=0.4,label='Decaying Cosine',
         color='red',linestyle='dashed',linewidth=2,
         marker='o',markersize=5,markerfacecolor='blue',
         markeredgecolor='blue')
plt.ylim([-2,2])
plt.legend()
plt.show()
```

The output of this python chunk was the following plot:

Here, the plot from the last python chunk, given above, was plotted on the top of the polar plot created by the previous python chunk. There is no issue when I render the entire qmd notebook.

Below is the complete qmd file I was playing with:

---
title: "matplotlib demo"
format:
  html:
    code-fold: true
jupyter: python3
---

For a demonstration of a line plot on a polar axis, see @fig-polar.

```{python}
#| label: fig-polar
#| fig-cap: "A line plot on a polar axis"

import numpy as np
import matplotlib.pyplot as plt

r = np.arange(0, 2, 0.01)
theta = 2 * np.pi * r
fig, ax = plt.subplots(
  subplot_kw = {'projection': 'polar'} 
)
ax.plot(theta, r)
ax.set_rticks([0.5, 1, 1.5, 2])
ax.grid(True)
plt.show()
```

```{python}
x = np.linspace(-2,2,41)
y = np.exp(-x**2) * np.cos(2*np.pi*x)
plt.plot(x,y,alpha=0.4,label='Decaying Cosine',
         color='red',linestyle='dashed',linewidth=2,
         marker='o',markersize=5,markerfacecolor='blue',
         markeredgecolor='blue')
plt.ylim([-2,2])
plt.legend()
plt.show()
```

The initial quarto check Output

/cloud/project$ quarto check jupyter

[✓] Checking Python 3 installation....OK
      Version: 3.8.10
      Path: /usr/bin/python3
      Jupyter: 4.11.1
      Kernels: python3

[✓] Checking Jupyter engine render....OK

The final quarto check Output

/cloud/project$ quarto check jupyter

[✓] Checking Python 3 installation....OK
      Version: 3.8.13 (Conda)
      Path: /cloud/home/r157428/.local/share/r-miniconda/envs/r-reticulate/bin/python
      Jupyter: 4.11.1
      Kernels: python3

(\) Checking Jupyter engine render..../home/r157428/.local/lib/python3.8/site-packages/jupyter_client/__init__.py:23: UserWarning: Could not import submodules
  warnings.warn("Could not import submodules")
Traceback (most recent call last):
  File "/opt/quarto/0.9.293/share/jupyter/jupyter.py", line 21, in <module>
    from notebook import notebook_execute, RestartKernel
  File "/opt/quarto/0.9.293/share/jupyter/notebook.py", line 14, in <module>
    from nbclient import NotebookClient
  File "/home/r157428/.local/lib/python3.8/site-packages/nbclient/__init__.py", line 6, in <module>

quarto tools check Output

/cloud/project$ quarto tools check
[✓] Reading Tool Data
Tool                Status                          Installed     Latest        
tinytex             Not installed                   ----          v2022.07   

Checklist

• formatted your issue so it is easier for us to read?
• included a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
• documented the quarto version you're running, by pasting the output from running quarto check in the "Quarto Check Output" text area?
• documented the version of the quarto tools you're running, by providing the output from running quarto tools check in the "Quarto Tools Check Output" text area?
• documented the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
• documented which operating system you're running? If on Linux, please provide the specific distribution as well.
• upgraded to the latest version, including your versions of R, the RStudio IDE, and relevant R packages?