Not running jupyter from a virtual environment?

[gregnordin]

I’m very interested in hearing your thoughts on the todo item:

Document rationale for one notebook server per user vs. running jupyter notebook in project's virtual environment

[bhrutledge]

Thanks for asking, @gregnordin! It's my understanding that Jupyter Notebook and Lab are intended to be standalone applications, running independently of a specific project. I think of it similarly to other applications like word processors, IDEs, and web browsers. Take Microsoft Word, for example: that will only be installed once per user, but it will be used to edit multiple documents. Similarly, I have one installation of Visual Studio Code, but I use it to work on multiple projects, each with a distinct development environment. With Jupyter Notebook/Lab, the document/project is the .ipynb file, and the development environment is the kernel.

It's certainly tempting to pip install jupyter into a project's virtual environment, because that's usually pretty easy. But, jupyter comes with lots of dependencies, which make your project's requirements larger than necessary, and could lead to conflicts. Furthermore, pip install is specific to Python projects, but one of the beauties of Jupyter is that it can run notebooks using kernels from all sorts of languages.

This has me thinking that it would be better to recommend installing Jupyter with a separate package manager (such as brew on macOS or apt on Ubuntu), or with pipx.

What are your thoughts?

[gregnordin]

Thanks @bhrutledge--you make some very good points comparing jupyter to an application like MS Word. I hadn't thought of it that way. I have also found that trying to run two instances of jupyter at the same time, each from a different virtual environment, can mess things up in small ways.

My approach at this point is to create a virtual environment from which I only run jupyter and nothing else, and never jupyter from any other virtual environment. My approach is documented in this repo (note: I no longer follow the part about running jupyter from within other virtual environments).

That's an interesting idea about installing jupyter with a package manager, which I assume would take care of setting up an appropriate virtual environment for it like pipx does.

[bhrutledge]

Thanks for sharing!

My approach at this point is to create a virtual environment from which I only run jupyter and nothing else

I think that's the same approach suggested by this repo, except that I include installation of some frequently-used packages in the default kernel. I then leave it up to other projects' virtual environments to install kernel specs with their specific dependencies.

I'm going to leave this issue open as a supplement to the README, until I get around to updating it.

[gregnordin]

@bhrutledge: sounds good!

I think our approaches are nearly the same, with the exception that I don't use the jupyter virtual environment as a jupyter kernel since I only have jupyter installed and want to keep it simple and clean. Instead, I have a separate anaconda virtual environment added as one of the available jupyter kernels that I access from within jupyter for generic situations when I don't need to go to the trouble of setting up a specific virtual environment for small projects. If I do need to set up a specific virtual environment for a project, I do the same thing as suggested in this repo.

The reason I have separated out anaconda in a separate virtual environment is that makes it easy to replace it with an updated version in the future. My main problem has been setting up python on my mac laptop and have it degrade and/or fall behind current versions as the years progress. My solution has been to make everything easily swappable and replaceable without affecting other virtual environments. This is documented in the link in my previous comment.

In any case, I appreciate you sharing your thoughts--I've found it to be quite helpful in clarifying some of my thinking!