Redundant resolving of packages

[yha]

Sometimes Pluto re-resolves packages and says "Restart notebook (recommended)" when I just moved a using to a different place.

[icweaver]

I think the problem is that you are not just moving an already imported package to a different place. When you hit backspace on the second cell to delete it, you are also removing StatsBase from the notebook environment, and so Pluto rightly asks you to restart the notebook to sync things up. I think the act of the notebook updating after a cell is deleted can been seen a bit more clearly here:

example-2022-02-24_21.29.22.mp4

So to get the expected behavior when moving packages around, I would just do all my moving first, and then run all the cells at once by pressing Ctrl+S on the keyboard after I am finished:

example-2022-02-24_21.27.24.mp4

btw, thanks for including a video. It made it a lot easier to see what you were trying to do!

[icweaver]

Ok, just to make sure we are on the same page here, I am assuming that by "editing code" you mean before hitting save/running the cell.

I see deleting a cell as different from this case because you are actually removing information from the dependency graph, which needs to be accounted for in some way. Just using the same video I shared above again as an example for convenience:

155642367-beb4f270-2204-4bb5-9ec4-a5b505532e75.mp4

Before removing the empty cell, no changes have been committed (the cell is highlighted in yellow as a visual cue that once you run this cell, the current state will be altered), and so the dependency graph is unchanged. So far everything is the same between deleting a line of code and "deleting a cell but not committing that change just yet". Once you go the extra step of actually deleting the cell, we have fully committed to altering the dependency graph. That is the difference.

Now whether you agree with this behavior or not is totally up to you, but as it stands, I believe this is fully in line with the reactive, reproducible, and non-hidden state that Pluto notebooks promise.

[yha]

So far everything is the same between deleting a line of code and "deleting a cell but not committing that change just yet".

I think we are still using different terminology here. When I refer to "deleting a cell", I do not mean deleting its contents. I mean pressing backspace in an empty cell or selecting "Delete cell" from the cell's "..." menu. So currently there is no such thing in Pluto as "deleting a cell but not committing that change just yet", and I think there should be.

Once you go the extra step of actually deleting the cell, we have fully committed to altering the dependency graph.

This is the behavior of Pluto I'm trying to understand the rationale for, and you seem to be using it as an assumption.

[icweaver]

I think we are still using different terminology here. When I refer to "deleting a cell", I do not mean deleting its contents. I mean pressing backspace in an empty cell or selecting "Delete cell" from the cell's "..." menu.

I think we are on the same page about this.

So currently there is no such thing in Pluto as "deleting a cell but not committing that change just yet"

Yes, that's why I used quotations, but I could have made that more clear

and I think there should be

This is where we diverge

This is the behavior of Pluto I'm trying to understand the rationale for, and you seem to be using it as an assumption.

I think it would be unreasonable to expect folks to be immediately comfortable with the quirks of Pluto, especially when coming from editors that follow a more traditional paradigm, so I appreciate you taking the time to try and approach things from a different perspective =]

[yha]

I think it would be unreasonable to expect folks to be immediately comfortable with the quirks of Pluto, especially when coming from editors that follow a more traditional paradigm

Perhaps I should clarify that I am not a newcomer to Pluto. For the last half year or so, I've been using it almost every workday, and this still tripped me up.
I still don't see any downside to my proposed change (apart from the work involved in implementing it, obviously), but I'm willing to let it go at this point, and just get used to being careful about deleting cells when there are unsaved changes. Thank you very much for your time.

[icweaver]

I'm sorry, I never meant to imply that you were. I think different aspects of Pluto can trip up users of any experience level, and that there can always be room for improvement in this regard, so thank you for sharing your experience. I have been discussing how to handle this moving forward with Fons, and share some potential next steps below:

Fons van der Plas: anyways! their suggestion does make sense: allow you to delete a cell without committing, just like you can currently delete a line within a cell without committing. This strategy of "making multiple edits, and running them later with a single Ctrl+S" is actually really useful, so that would be cool.

Fons van der Plas: But! I think it would be confusing without new UI, we would clearly need to show "a cell was deleted here, you still need to save it". And this might be perceived as annoying ("why doesn't Pluto just save it?")

Fons van der Plas: We could also go at the underlying issue (moving imports around can lead to "restart required") in another way, by making pluto keep track of which imports at which versions existed before. Perhaps this could be done by Julia itself, there is some ongoing work in this direction: JuliaLang/Pkg.jl#3002 (read "sysimage" as "already loaded in the process") and JuliaLang/julia#44329

It might be better to move this to a separate thread or the Zulip, but will leave this up to you!

[fonsp]

Hey all! Very interesting question! I have seen other people confused/annoyed by this behaviour.

Like pointed out in the discussion above (I only read parts of it): the behaviour is "intentional": our current model is that deleting a cell should immediately "commit" that change, hence causing this issue.

---

The feature suggestion by @yha makes a lot of sense: allow you to delete a cell without committing, just like you can currently delete a line within a cell without committing. This strategy of "making multiple edits, and running them later with a single Ctrl+S" is actually really useful, so that would be cool.

But! It would need some good new UI to pull this off, and I imagine that this new behaviour would then become a source of confusion/annoyance for its own reasons. (Especially for new users.)

---

We could also go at the underlying issue (moving imports around can lead to "restart required") in another way, by keeping track of which imports at which versions existed before. That way, removing an import, committing, and then adding it, committing, would become a no-op as a whole. This "restart recommended" message would appear, and then disappear again.

But! I think this would be too difficult to implement correctly, especially because we try to only expose public Pkg API through GUI, without touching internals. (And the gain might be too small.) Now, if this were done by Julia itself, then that might be the way to go! There is some ongoing work in this direction: JuliaLang/Pkg.jl#3002 (read "sysimage" as "already loaded in the process") and JuliaLang/julia#44329 .

---

All in all, I would say that right now, this is something we have to live with, because a fix would be too difficult and time-consuming to implement. But I can imagine a future where we have more API to work with from Pkg and Julia.

Specifically:

• API like which_packages_are_loaded_with_versions() would make it much easier to check whether a restart is recommended or required, by simply comparing this list with the Manifest.toml. Right now we have to jump through lots of heuristic hoops to get it.
• API like Pkg.add(name; only_allow_versions_already_loaded=true) would make sure that removing, and then re-adding a package will not change its version in the manifest.