Allow multiple namespaces inside one notebook

[cocoa1231]

This issue was raised on Discourse, here's the original link -
https://discourse.julialang.org/t/notebooks-need-modules-i-e-multiple-separate-global-namespaces/68541

My main issue is that I may have multiple variables with the same name that I want to redefine as I go along using the notebook. For instance, I'm reading a CSV with a column of current values, and I have two or three such CSVs all analysed in different ways in the same project (notebook), so I would like something like a chapter/module option so that I can import data from the CSV into a variable named current multiple times. A more detailed case is given in the link by Discourse user Luapulu, here's it copy pasted for convenience -

---

For context, a typical workflow for me, using a Pluto or Jupyter notebook, looks like this

1. Read in some data / create some initial data
2. Do some processingPlot some intermediate results
3. Do some more processing
4. Plot final results
5. Do processing again in a different way (different data, different method, etc.)
6. Plot intermediate results again
7. Do some more processing
8. Plot final results for second version
9. Compare both versions (often by plotting)

So what’s the problem?

First, I have multiple plots and because plots invariably require variables like an ax = Axis(...) or fig = Figure(...). I either have to give each axis or figure variable a new name like ax_for_intermediate_results_version1 or I have to wrap the whole plotting expression in a let ... end block, which is my current solution. This solution, however, doesn’t allow for any kind of interactive plotting. The whole plot has to be defined in one block. Adding elements piece by piece is not possible. That’s sad because, for me, the whole point of a notebook is to have a more interactive coding experience, which includes the plotting.

Second, since notebooks are great for experimentation, I often implement things in a few different ways and compare the results. Or I process different data using similar methods. In any case, I often have repeated code. And again, I run into issues with namespaces. When I write the second implementation I have to make sure I change every foo and bar to foo2 and bar2. Pluto helps quite a bit here because it won’t let you assign to the same variable twice. But still, this process of appending some kind of number or letter sequence to every variable is the cause of a great number of bugs in my notebooks and is quite tedious. You might argue that I should just write more reusable code. Put everything in functions and you’ve solve the problem, right? Yes, you have, but if you can do that, why even use a notebook? The thing that makes notebooks work, is that you have a lot of stuff in global scope that you can freely play with and inspect. Reusable code is necessarily less interactive.

What’s my proposed solution?

Implement a chapter/module feature using modules. Allow me to put multiple cells into one module. This way, both versions of my implementation get their own global scope and everything remains interactive because everything is still global and I can easily use code from other modules (or chapters) via the import/using syntax.

Every plot can live in its own little module where I can freely create the plot across multiple cells. I avoid naming conflicts and I have even better control of what I do and don’t import from the surrounding scope, than I would with a let ... end block.

[rikhuijzer]

My main issue is that I may have multiple variables with the same name that I want to redefine as I go along using the notebook.

Generally, I quite like that Pluto doesn't allow this. Since code can become very complex very quickly, some things are generally advised against. Re-using variable names is one such a thing. The reason is that once variables can contain different things at different points, it becomes very hard for a reader of the code to track down what is inside the variable at a given point in time.

The most capable solution, in my opinion, is to write proper functions.

Put everything in functions and you’ve solve the problem, right? Yes, you have, but if you can do that, why even use a notebook? The thing that makes notebooks work, is that you have a lot of stuff in global scope that you can freely play with and inspect. Reusable code is necessarily less interactive.

It is a fair point that code inside functions is sort of locked away and that you cannot easily access the insides. However, a better way to think about functions is as building blocks. When writing code, when a proper function is created which hits the right abstraction, the function is created once and only when bugs occur the inside needs to be inspected. Then, functions are not about "locking away" things, but functions are about "hiding unnecessary details".

[Luapulu]

With the way I use notebooks this just isn't feasible. Yes, once you've figured out the right level of abstraction and where all the pieces need to go, it is much cleaner to write more reusable code. The thing is: if I'm using a notebook it's because I have yet to figure out where all the pieces go. I don't know yet what level of abstraction I need or how I can split my code into nice reusable building blocks. The way I figure that out is by playing around interactively in a notebook. If I want to play interactively and avoid namespace conflicts from duplicate code, I need some kind of system to separate global namespaces. My proposal above would be one way of potentially implementing this.

[cocoa1231]

I would also like to add to the above reply by saying that allowing multiple namespaces per notebook doesn't take away from the reactivity. If one wants to keep Pluto as is, then one can simply use a single global namespace and nothing would change. But if one wants multiple namespaces per notebook, this feature would just allow that. As of now for analysing multiple datasets I am either using multiple notebooks (which isn't convenient as I have to wait for first compilation of plots, DataFrames, etc.) or just use IJulia. I am doing the latter lately.

[rashidrafeek]

I have to wrap the whole plotting expression in a let ... end block, which is my current solution. This solution, however, doesn’t allow for any kind of interactive plotting. The whole plot has to be defined in one block. Adding elements piece by piece is not possible.

I didn't understand this. What do you mean by "interactive" here? Do you mean reactivity? Also, isn't the only difference between IJulia plotting with this that we need to add let and end in the beginning and end of the cell respectively. How is the IJulia workflow better here. Asking since I used to use IJulia and I have found plotting in Pluto a lot better since it forces me to use let ... end blocks and so doesn't pollute the global namespace.

As of now for analysing multiple datasets I am either using multiple notebooks (which isn't convenient as I have to wait for first compilation of plots, DataFrames, etc.) or just use IJulia.

Why is it difficult to analyse multiple datasets in one notebook? Can't plotting for different datasets be done in different cells. And isn't this the same in IJulia.