Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple notebooks installing packages in parallel can break #485

Closed
fonsp opened this issue Sep 24, 2020 · 7 comments
Closed

Multiple notebooks installing packages in parallel can break #485

fonsp opened this issue Sep 24, 2020 · 7 comments
Labels
backend Concerning the julia server and runtime bug Something isn't working

Comments

@fonsp
Copy link
Owner

fonsp commented Sep 24, 2020

I've seen it happen once 馃憖

@shashankp
Copy link
Contributor

I guess notebook on the right is worker 3
image

@fonsp fonsp added bug Something isn't working backend Concerning the julia server and runtime labels Sep 24, 2020
@Moelf
Copy link
Contributor

Moelf commented Oct 8, 2020

https://github.com/simonbyrne/PkgLock.jl
https://discourse.julialang.org/t/distributed-packages-and-race-condition-in-pre-compiling/43340
I guess this is what it is with Distributed.jl, not something we can fix here in Pluto.

Maybe advise user to use PkgLock at the mean time if they have multi-user situation. (much like HPC use-case, where this error is reported the most)

@fonsp
Copy link
Owner Author

fonsp commented Oct 29, 2020

It caused this error on windows:

SystemError: opening file "C:\Users\fonsv\.julia\registries\General\Registry.toml": No such file or directory

@fonsp
Copy link
Owner Author

fonsp commented Nov 1, 2020

not something we can fix here in Pluto.

I disagree, we can and should fix it inside Pluto

@fonsp
Copy link
Owner Author

fonsp commented Jan 26, 2021

JuliaLang/Pkg.jl#2219

@fonsp fonsp closed this as completed Jan 26, 2021
@fonsp fonsp reopened this Jan 26, 2021
@fonsp
Copy link
Owner Author

fonsp commented Nov 3, 2021

Solved if you use #844

@fonsp fonsp closed this as completed Nov 3, 2021
@fonsp
Copy link
Owner Author

fonsp commented Dec 10, 2021

This has been solved by Pluto's built-in package manager, which uses a global lock to ensure that only one notebook's package environment is being processed at the same time, all other notebook evaluation happens in parallel.

This can still be an issue when you run multiple notebooks in parallel which set up their own package environment using Pkg.activate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Concerning the julia server and runtime bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants