Julia kernel not respecting environment selection

[deoradh]

Environment data

• VS Code version: 1.56.0-insider
• Jupyter Extension version (available under the Extensions sidebar): 2021.6.790799743
• Python Extension version (available under the Extensions sidebar): XXX
• OS (Windows | Mac | Linux distro) and version: Win10 21H1 (19043.962)
• Python and/or Anaconda version: (Conda pkg from Julia)
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A?
• Jupyter server running: Local

Expected behaviour

When using a Julia notebook, having selected a Julia environment, available packages should reflect that environment, and adding/removing packages should affect only that environment.

Actual behaviour

Only the default (eg, "v1.6") environment is used. The selected environment is indexed, but commands are not run against it.

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

1. Install Julia
2. In Julia: import Pkg
3. Pkg.add("IJulia")
4. Pkg.add("Conda")
5. import Conda; Conda.add("ipython")
6. ] to get to Pkg mode, prompt should reflect v1.6 environment
7. build
8. (At this point, I believe Jupyter kernel should now be available for connection)
9. still in Pkg mode: activate cas
10. (At this point a new environment folder has been created)
11. add SymPy
12. (At this point, cas environment, not the v1.6 environment, should have SymPy installed. Verify with st command.)
13. In VSCode, add new notebook, set for Julia.
14. 1+1 (should return 2)
15. using SymPy (returns LoadError -- this is expected)
16. select cas environment via UI in status bar .. probably located under user folder
17. using SymPy (returns LoadError -- this is the problem)
18. Open new Julia shell. Type ] for Pkg mode. Prompt should show v1.6 environment.
19. st to confirm SymPy not present
20. add SymPy; verify with st
21. Repeat starting at (11). Lines 13 and 15 should both succeed. @vars x y should return (x, y)
22. while in the cas environment in VSCode, import Pkg; Pkg.remove("SymPy")
23. In Julia shell's Pkg mode, st should now show SymPy removed from v1.6 environment, yet remains in cas environement. This is the problem, again.

Logs

> ~\AppData\Local\Programs\Julia-1.6.1\bin\julia.exe -i --color=yes --project=@. ~\.julia\packages\IJulia\e8kqU\src\kernel.jl ~\AppData\Local\Temp\tmp-26076Hwwked5aynhv.json
cwd: ~\AppData\Local\Programs\Microsoft VS Code Insiders

[rchiodo]

Thanks for the bug.

It's not clear to me what this step is:

• select cas environment via UI in status bar .. probably located under user folder

I'm not sure how to create extra Julia kernels. Are you saying that the steps above create another Julia jupyter kernel? And that you're able to select this 'kernel' from within VS code?

If that's true, can you print the contents of that kernelspec.json? It should have the necessary arguments for us to start the julia kernel with the cas environment. If it doesn't, then Julia isn't writing this environment data for us to make this work.

[deoradh]

If that's true, can you print the contents of that kernelspec.json? It should have the necessary arguments for us to start the julia kernel with the cas environment. If it doesn't, then Julia isn't writing this environment data for us to make this work.

Julia environments are basically folders that allow different sets of package dependencies to be loaded. You could say they map to projects. VSCode's Julia support already knows about them and allows the user to select different ones, which the Language Service dutifully indexes. What isn't happening is code being executed within any environment other than the default.

See https://towardsdatascience.com/how-to-setup-project-environments-in-julia-ec8ae73afe9c

Based on the above, the issue is that either the --project or -e flag isn't being passed with the correct value (ie, with the environment).

[rchiodo]

How are you selecting the 'cas' environment?

You specified this in your repro steps:

• select cas environment via UI in status bar .. probably located under user folder

Did you mean this UI?

If so, that would be coming from a kernelspec.json on disk that we read. Julia wrote it out. For example, my Julia kernel is here:

D:\Tools\julia-1.6.1>jupyter kernelspec list
Available kernels:
  python3      C:\Users\aku91\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\ipykernel\resources
  julia-1.6    C:\Users\aku91\AppData\Roaming\jupyter\kernels\julia-1.6 <-- This one

Do you have multiple kernels in your AppData\Roaming\Jupyter\Kernels?

[deoradh]

• select cas environment via UI in status bar .. probably located under user folder

Did you mean this UI?

Yes

If so, that would be coming from a kernelspec.json on disk that we read. Julia wrote it out. For example, my Julia kernel is here:

Incorrect. The UI just shows the contents of the [user]\.julia\environments folder, which is a well-known folder, but not required for an environment (the cas environment I created above was actually created in [user]). I suspect the environment list is being populated by the Language Service. It isn't anything to do with the kernelspec list (as I just confirmed by simply copying the existing 1.6 environment folder in explorer, and bringing up the UI again. These folders only contain project manifests -- no kernel files.

Either way, VSCode knows about the environment, and allows an arbitrary environement to be selected by picking a folder.

Do you have multiple kernels in your AppData\Roaming\Jupyter\Kernels?

I do. I have (among other non-Julia kernels) the base-level Julia kernels (1.5, 1.6..).

Environments cannot be expected to be known a priori -- that is, when either VSCode or IJulia is installed -- so VSCode's going to need to either (1) adjust its use of the base kernelspec to derive the appropriate values the use environment the user specified, or (2) disable the environment selector when in a notebook (ie, only allow the base). (2) is adjusting VS UI to match the current capability, but (1) is more useful to the end-user.

[deoradh]

(NB: I may have mixed Insiders vs non-Insiders versions here .. I've been bouncing between both, but environment is not being treated properly in either)

[rchiodo]

Incorrect. The UI just shows the contents of the [user].julia\environments folder, which is a well-known folder, but not required for an environment (the cas environment I created above was actually created in [user]). I suspect the environment list is being populated by the Language Service. It isn't anything to do with the kernelspec list (as I just confirmed by simply copying the existing 1.6 environment folder in explorer, and bringing up the UI again. These folders only contain project manifests -- no kernel files.

If this is true, then the Jupyter extension did not put that item there. Some other extension must have. We only list kernelspecs in that UI. VS code allows other extensions to contribute execution for notebooks (in insiders) . Your bug would be on that other extension (perhaps this one? : https://github.com/julia-vscode/julia-vscode)

[deoradh]

Pardon, I was too hasty -- I looked again at the image. That isn't the selector I'm intending. This is:

[rchiodo]

That's not controlled by our extension. It's controlled by the julia extension AFAIK. We'll have to sync up with them to get it to work with notebooks.

[deoradh]

I'll open an issue on their end and xref.