support sage + python3 + Sage worksheets?

[williamstein]

There's thousands (?) of lines of code to make Cocalc work with Sage worksheets, and it is -- of course -- all written as Python2 scripts, since Sage is Python2. It's this code:

https://github.com/sagemathinc/cocalc/tree/master/src/smc_sagews/smc_sagews

We haven't even begun thinking about rewriting this in Python3. I don't know if we ever will, exactly, though today Hal Snyder mentioned he wants to look into it. I would probably rather spend them time writing a "cocalc" kernel that basically does the same thing (i.e., would evaluate code using the exact same semantics as the current Sage worksheets). Combining such a Jupyter kernel with a new Jupyter "single document mode" would provide the same functionality and UI as Sage worksheets.

[williamstein]

Sage-9.0 -- which comes out soon -- will use python3 only, so now is the time to think this through.

For about a decade I've wondered what the switch of Sage to Python3 would really be like. Here's how it is for the sage cell server, which I guess a ton of external content depends on:

"Public servers at https://sagecell.sagemath.org/ have been switched to SageMath 9.0.beta9 running Python 3. You have a few days to adjust before the next term starts!"

I'm guessing this will completely break a massive amount of content, e.g., anything that uses a print statement is broken.

I wonder what we should do with Sage worksheets on CoCalc? Can we do better, given our more complicated (than sage cell server) user interface?

Jupyter? With Jupyter kernels, we'll just have sage-8.x (which is all python2) and then sage-9.x (which is all python3), and people with sage-8.x notebooks just continue to use that no problem.

With Sage worksheets it's less obvious what to do... We have a LOT of these worksheets on cacalc, and, e.g., we want a solution that doesn't break any of UCLA's materials, or anybody else's.

Also, there are major subtle changes where python2 --> python3 can introduce subtle mathematical bugs, e.g., because the semantics of int division changed, and it would be hard to automatically address these in general. (I don't want to do it.)

The ideal UI would probably be that:

• If you click the +New -> Sage worksheet, you get a sage-9.x python3 worksheet by default
• One a Sage worksheet is opened, we have a new menu at the top where you can select the version of Sage it uses, but there are only two options: sage-9.0 (or whatever the current is), and sage-8.9. That's it.
• Any sage worksheet that doesn't have some extra metadata about the version of sage gets sage-8.9 by default when opened. This will of course apply to all the UCLA course materials.
• Any new worksheet gets sage-9.0 (or whatever is current).

So that's the UI, but says nothing about the implementation. The implementation would involve:

• Add some metadata to the sage worksheet to specify the version. Metadata is something we don't really have in any way yet and is tricky because the sage worksheet is basically the text that you see, with some uuid lines converted to horizontal html widgets... But I'm sure there is a way to do this that isn't too horrible, and of course whatever is done will just "melt away" when we eventually switch to something that is just Jupyter notebooks under the hood (someday!).
• Have up to two sage servers that run, one for sage-8.9 (written in python2), and the other for sage-.9 (rewrite in python3 of sage_server.py and friends), each on a different port.
• And implement the menu to choose between these two.

The big question is to what extend is all of the above necessary. Could we just say "screw it" and do what Andrey did with the sage cell server? We have paying customers and he doesn't, so we are in a different position.

[haraldschilly]

Yes, keeping old worksheets and material work is a good idea. At least, it should not be hard to make them work.

I don't understand why we have to "rewrite" the sage server in python 3. Why not make it work with Python 3 like we did for the pyutils? If there are any Sage specifics we can put them behind a version flag.

Besides that, IMHO selecting the version of Sage behind a Sage Worksheets should be possible, regardless of the python3 change or not.

[williamstein]

Why not make it work with Python 3 like we did for the pyutils?

That would be ideal!

[williamstein]

We discussed this in video chat today and decided the shortterm plan, which doesn't require changing the format of sagews files, is the following:

1. Get this PR in so that Sage worksheets fully work with both python2 and python3 versions of Sage: sagews: sage 9.0 / python3 compatibility #4304

2. For now keep sage-8.9 as the default sage in all projects.

3. BUT, if the user does sage_select 9.0, then Sage worksheets will use the new python3 version of Sage.

4. Add some GUI support for sage_select.

5. Eventually sage will default to sage-9.x, but we're not sure when.

It might be that we do a sagews --> ipynb transition soon-ish, at which point this problem dissolves.

Related discussion: https://groups.google.com/forum/#!msg/sage-devel/vYlbnAwKATM/1BiqGNeVBwAJ

[dimpase]

you don't have to sit on 8.9, you can ./configure --with-python=2 to build it Python 2-based.

It's not correct that 9.0 supports python3 only, it supports both 2 and 3, it's the default Python version that changed.

By the way, it indicates that a good Sage worksheet format should not only store the Sage version, but also Python version (and probably versions of various crucial packages, too).

[haraldschilly]

well, yes, but I think it's even more confusing to offer a python 2 variant of 9.0 … definitely long-term.

…

[dimpase]

It's only confusing if you think of Sage as a monolitic distribution, and this is something that is not really a good angle any more.

[vbraun]

If you want to transition sagews --> ipynb sometime soon then you could just offer Python 2 + sagews + sage-9.0 now and switch to Python 3 + ipynb + sage-10.0 in one step.

[williamstein]

If you want to transition sagews --> ipynb sometime soon then you could just offer Python 2 + sagews + sage-9.0 now and switch to Python 3 + ipynb + sage-10.0 in one step.

Thanks for your comment!

We already have a button that exports sagews files to ipynb (the "Jupyter" button in sagews).

We are going to offer python2 + sagews + sage-8.9 indefinitely, but will somehow have a way to also offer python3 + sagews + sage-x.y (for x > 8). We are definitely not ever planning to deprecate sagews support, but will hopefully provide a way to 100% use sagews in a way that uses the ipynb format internally (it will still have nearly identical functionality and look and feel to sagews). IUsers tells us that Sage worksheets have significant UI advantages over Jupyter notebooks, similar to how something like VScode is preferable to Vim for some developers...

[williamstein]

This was fixed years ago...