upgrade sagenb and build sagenb in sage/python3

[fchapoton]

sagenb is not compatible with python3, at least in the current version 1.0.1

The latest release 1.0.2 is certainly slightly better in this respect, but maybe still not fully py3 compliant.

Let us do the upgrade to 1.0.2. We also re-activate the build of sagenb in sagePy3, to help building the doc of sagePy3.

Side remark: to cut the ties between sagenb and sage, we need to get rid:

• of the cluster interact convert cluster interact to jupyter notebook #24760 and upgrade to ipywidgets 7.2.0 #25074
• of the mandelbrot interact convert mandelbrot/julia interact to jupyter notebook #24994
• of the graph editor #?????
• of the graph database navigator #?????
• of the interactive debugger #?????
• of calls to some tab completion function py3: incorporate a tab completion function from sagenb #24998

tarball with sagenb update: https://github.com/sagemath/sagenb/releases/download/1.0.2/sagenb-1.0.2.tar.bz2

CC: @dimpase @vbraun @jdemeyer @kcrisman @embray

Component: python3

Author: Dima Pasechnik

Branch: ef854c1

Reviewer: Frédéric Chapoton, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/22431

[jdemeyer]

comment:3

Given that sagenb is essentially a dead project, I don't think that it will ever be Python3 compliant.

[fchapoton]

comment:4

I did a lot of work on python3 compliance of sagenb since the last release. At least this should prevent the build to stop on syntax errors.

[dimpase]

comment:5

IMHO one does not need supernatural resurrection powers to port a dead python2 project of sagenb size to python3---given that all of its python components are python3 compatible (which was not true for Twisted until a couple of years back).

[fchapoton]

comment:6

The update of sagenb is probably dependent of #20922.

[fchapoton]

comment:8

can we hope to make a (maybe last) release of the legacy notebook ?

[kcrisman]

comment:9

At the least sagemath/sagenb#416 and your own sagemath/sagenb#423 would need to be merged first, right?

I don't really see any reason to not keep the legacy notebook around for quite some time, granting that it will not receive much attention. We just got a support request from someone using Sage 4.6 (!) who really didn't want to upgrade; this is positively new compared to that.

But I definitely agree that whatever the absolute minimum needed to get sagenb to work minimally with python3 is fine.

[fchapoton]

comment:10

​sagemath/sagenb#423 should now be ready

[fchapoton]

comment:11

Would it be possible to make a new release of sagenb, please ?

[dimpase]

comment:12

Replying to @fchapoton:

Would it be possible to make a new release of sagenb, please ?

I'll try.

[dimpase]

comment:13

Please test https://github.com/sagemath/sagenb/tree/1.0.rc0
(copy of the current master)
before I go ahead with making a new Sage package.
It works for me following the instructions in HACKING (updated, to take care of changed names and of the need to deal with mathjax).

Report issues on github, please

[dimpase]

comment:14

the new sagenb is on #23066 and needs review.

[kcrisman]

comment:15

the new sagenb is on #23066 and needs review.

Reading the comments there and at #22787 make it unclear to me whether this one is, in fact, py3 compatible.

What about comment:6 about #20922 - still valid?

[dimpase]

comment:16

Replying to @kcrisman:

the new sagenb is on #23066 and needs review.

Reading the comments there and at #22787 make it unclear to me whether this one is, in fact, py3 compatible.

Given that Sage does not run with Python 3 yet, it's a bit early to say, no?

What about comment:6 about #20922 - still valid?

Probably not really, for it does work without #20922 - I don't get doctest failures mentoined in comment 4 on #20922.

$ sage -t --long local/lib/python2.7/site-packages/sagenb/notebook/challenge.py
Running doctests with ID 2017-05-25-05-57-19-d9e71fa3.
Git branch: sagenb10
Using --optional=ccache,database_gap,gap_packages,mpir,python2,sage
Doctesting 1 file.
sage -t --long --warn-long 117.7 local/lib/python2.7/site-packages/sagenb/notebook/challenge.py
    [113 tests, 0.38 s]
----------------------------------------------------------------------
All tests passed!

I also do not see any mention of click package in sagenb's installation log.

[kcrisman]

comment:17

the new sagenb is on #23066 and needs review.

Reading the comments there and at #22787 make it unclear to me whether this one is, in fact, py3 compatible.

Given that Sage does not run with Python 3 yet, it's a bit early to say, no?

Haha, I thought we were closer to that now. No worries, just checking in.

[dimpase]

comment:19

see also sagemath/sagenb#440

• some of these dependencies should go, indeed.

[fchapoton]

comment:20

We should also get rid of the sagenb documentation building, which does not work in the python3 sage..

See

https://patchbot.sagemath.org/log/0/Ubuntu/18.04/x86_64/4.15.0-20-generic/petitbonum/2018-05-15%2012:37:45?plugin=docbuild

[fchapoton]

comment:21

see #25382

[kcrisman]

comment:22

What is the current status of this ticket (as opposed to the documentation build)? I thought at least some of these issues were dealt with. If it's just the graph functionality those can certainly be disabled, as I don't think they have worked properly for some time. If it's certain dependencies that is required for py3 that is somewhat different.

[fchapoton]

comment:23

trying to install sagenb, one gets

[sagenb-1.0.1]   Removing source in /tmp/pip-pzy8td0f-build
[sagenb-1.0.1] Successfully installed sagenb-1.0.1
[sagenb-1.0.1] Cleaning up...
[sagenb-1.0.1] ./spkg-install: line 56: cd: /home/chapoton/sage3/local/lib/python2.7/site-packages/sagenb/data: No such file or directory
[sagenb-1.0.1] Error: Cannot find SageNB data directory.
[sagenb-1.0.1] 

because it is looking for sageNB data in the python2 library..

[fchapoton]

comment:24

Dima, could you please make a new release of sagenb ?

Currently (apart from the mathjax issue above in spkg-install) sagenb 1.0.1 installs with sage3, but fails to run because of

Traceback (most recent call last):
  File "/home/chapoton/sage3/src/bin/sage-notebook", line 268, in <module>
    launcher(unknown)
  File "/home/chapoton/sage3/src/bin/sage-notebook", line 69, in __init__
    from sagenb.notebook.notebook_object import notebook
  File "/home/chapoton/sage3/local/lib/python3.6/site-packages/sagenb/notebook/notebook_object.py", line 23, in <module>
    from . import run_notebook
  File "/home/chapoton/sage3/local/lib/python3.6/site-packages/sagenb/notebook/run_notebook.py", line 21, in <module>
    from exceptions import SystemExit
ModuleNotFoundError: No module named 'exceptions'

which has been fixed after the release of 1.0.1

[fchapoton]

comment:43

ping ?

[dimpase]

comment:44

I can try building the latest beta with python3,what should I pull in, besides the branch on this ticket?

[fchapoton]

comment:45

Nothing else to pull, the develop branch itself build fine with python3.

[dimpase]

comment:46

and how do I start sagenb so that the underlying python is python 3?

[fchapoton]

comment:47

Well, just sage -n=sagenb. But it will fail to start, because of twisted or some of the latest commited changes, not yet in the release here.

We only need to make sure that sage3 builds and starts with the branch here.

We also need to make sure that for sage2, the upgrade of sagenb does not break something inside the notebook.

[fchapoton]

comment:48

I propose to rename this ticket to "upgrade sagenb and build sagenb in sage/python3"

and keep for another ticket the goal of being able to run sagenb in sage/python3

[kcrisman]

comment:49

I was going to try this today on py2 to test the notebook but had some troubles upgrading Sage, see sage-devel.

[dimpase]

comment:50

With python3 sage, I get

$ ./sage -n sagenb
SageMath version 8.3.beta3, Release Date: 2018-05-27
Please wait while the old SageNB Notebook server starts...
/home/dima/sagepy3/local/lib/python3.6/site-packages/sage/misc/inline_fortran.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Traceback (most recent call last):
  File "/home/dima/sagepy3/src/bin/sage-notebook", line 268, in <module>
    launcher(unknown)
  File "/home/dima/sagepy3/src/bin/sage-notebook", line 74, in __init__
    notebook(*self.args, **self.kwds)
  File "/home/dima/sagepy3/local/lib/python3.6/site-packages/sagenb/notebook/notebook_object.py", line 243, in __call__
    return self.notebook(*args, **kwds)
  File "/home/dima/sagepy3/local/lib/python3.6/site-packages/sagenb/notebook/run_notebook.py", line 535, in notebook_run
    nb = notebook.load_notebook(directory)
  File "/home/dima/sagepy3/local/lib/python3.6/site-packages/sagenb/notebook/notebook.py", line 1839, in load_notebook
    nb = Notebook(dir)
  File "/home/dima/sagepy3/local/lib/python3.6/site-packages/sagenb/notebook/notebook.py", line 151, in __init__
    self.__conf = S.load_server_conf()
  File "/home/dima/sagepy3/local/lib/python3.6/site-packages/sagenb/storage/filesystem_storage.py", line 265, in load_server_conf
    return self._basic_to_server_conf(self._load('conf.pickle'))
  File "/home/dima/sagepy3/local/lib/python3.6/site-packages/sagenb/storage/filesystem_storage.py", line 187, in _load
    result = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

[fchapoton]

comment:51

Thanks Dima. It seems that you got a hard time doing that..

Failing is as expected. Maybe this specific issue is already fixed by the pull request done after you made the release. But this is not so important : let us keep that for another ticket.

Do you confirm that sage3 builds and starts with the branch here ? I think I did check that also. So it seems to be good to go from this point of view.

So now, there only remains to see if the branch here is not going to break anything in the sage2 sagenb.

[fchapoton]

comment:52

I have just checked again with sage3 release 8.3.b4 that sage3 builds and starts.

[dimpase]

comment:53

also, for me with beta4. You can give it a positive review, I think.

[fchapoton]

Reviewer: Frédéric Chapoton, Dima Pasechnik

[fchapoton]

comment:54

ok, let it be. I change the title and description to be more adequate to what is done here.

[vbraun]

Changed branch from public/ticket/22431 to ef854c1

[kcrisman]

comment:56

As a followup, in 8.3.beta6 I am having some troubles - see this post for details. I am pretty sure this isn't due to some stale process.

[kcrisman]

Changed commit from ef854c1 to none

[dimpase]

comment:57

Replying to @kcrisman:

As a followup, in 8.3.beta6 I am having some troubles - see this post for details. I am pretty sure this isn't due to some stale process.

Does sagenb still open for you directly, via sage -n sagenb ?

[kcrisman]

comment:58

As a followup, in 8.3.beta6 I am having some troubles - see this post for details. I am pretty sure this isn't due to some stale process.

Does sagenb still open for you directly, via sage -n sagenb ?

Yes, or at least with sage -notebook=sagenb it does. I think there is something with the browser + token not opening.

[fchapoton]

comment:59

could you try when undoing #25548 ?

[kcrisman]

comment:60

could you try when undoing #25548 ?

You are oracular.

It would be interesting to trace where exactly quit_sage is even called that it would be necessary ...

[kcrisman]

comment:61

could you try when undoing #25548 ?

You are oracular.

Maybe I wasn't clear; that fixes it. Shouldn't we have a blocker for 8.3 that includes that?

[dimpase]

comment:62

I can only say that the description of the issue you give it too vague.
Please feel free to open a ticket with details on how to reproduce this.

[kcrisman]

comment:63

Ok; see #25667.