Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

D4.4: Basic Jupyter interface for GAP, PARI/GP, SageMath, Singular #93

Closed
minrk opened this issue Sep 8, 2015 · 100 comments
Closed

D4.4: Basic Jupyter interface for GAP, PARI/GP, SageMath, Singular #93

minrk opened this issue Sep 8, 2015 · 100 comments

Comments

@minrk
Copy link
Contributor

@minrk minrk commented Sep 8, 2015

The Jupyter Notebook is a web application that enables the creation and sharing of executable documents which contain live code, equations, visualizations and explanatory text. Thanks to a modular design, Jupyter can be used with any computational system that provides a so-called Jupyter kernel implementing the Jupyter messaging protocol to communicate with the notebook. OpenDreamKit therefore promotes the Jupyter notebook as user interface of choice, in particular since it is particularly suitable for building modular web based Virtual Research Environments.

This deliverable aims at ODK's main computational components all having basic Jupyter kernels implemented and distributed. Full featured Jupyter interfaces will follow up in deliverable D4.7. The SageMath Jupyter kernel is a foundation for D4.5: Sage notebook / Jupyter notebook convergence.

We have implemented basic kernels for:

Meanwhile, the following Jupyter kernel has been implemented by the community, with some help of the ODK participants:

  • SageMath (built on top of Jupyter's Python kernel)
    • see this series of trac tickets
    • Part of the SageMath distribution
    • SageMathCloud integration: native
    • Features: TAB-completion, syntax highlighting, pretty printing, 2D/3D plots, documentation lookup, live manual, code introspection, output history, interruption, timing, debugger.
    • Interactive widgets (IPyWidgets): see D4.5

Finally, the following kernel has been made available within SageMath:

@minrk minrk added this to the D4.4 milestone Sep 8, 2015
@jdemeyer jdemeyer changed the title D4.4: Basic Jupyter interface for GAP, PARI/GP , Sage , Singular D4.4: Basic Jupyter interface for GAP, PARI/GP , SageMath, Singular Sep 9, 2015
@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Sep 9, 2015

For SageMath, a basic implementation is already part of SageMath itself.
Some improvements need review at:

Update: these three tickets were given positive review and were closed
as part of developer release "Sage 6.9.beta7" in 2015-09.
The corresponding functionality exists in public releases of SageMath
since version 6.9, which was released on 2015-10-09.
See changelog at http://www.sagemath.org/changelogs/sage-6.9.txt

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Sep 9, 2015

Basic PARI/GP Jupyter kernel: https://github.com/jdemeyer/pari_jupyter

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 23, 2016

@sebasguts
Copy link
Member

@sebasguts sebasguts commented Feb 23, 2016

I have started a basic interface to Singular here: https://github.com/sebasguts/jupyter-singular
Currently maybe an early beta. Also, one needs the following python module for the Singular bindings: https://github.com/sebasguts/SingularPython

@embray
Copy link
Collaborator

@embray embray commented Feb 23, 2016

👍

@jpflori
Copy link
Contributor

@jpflori jpflori commented Feb 24, 2016

Hi Jeroen,

Can you report on what you plan to bind here?
Is it mainly meant to give an interface similar to Singular interpreter?
Or is it more intricate like Sage library-like (broken) interface?

Best,

2016-02-24 0:16 GMT+09:00 Erik Bray notifications@github.com:

[image: 👍]


Reply to this email directly or view it on GitHub
#93 (comment)
.

Jean-Pierre Flori

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 24, 2016

Hi Jeroen,

Jeroen? I guess you mean Sebastian .

@jpflori
Copy link
Contributor

@jpflori jpflori commented Feb 24, 2016

Oh yes!
Sorry about that.
For some mysterious reason I assumed that the same person posted the two
first mails in this thread.

2016-02-24 16:15 GMT+09:00 jdemeyer notifications@github.com:

Hi Jeroen,

Jeroen? I guess you mean Sebastian .


Reply to this email directly or view it on GitHub
#93 (comment)
.

Jean-Pierre Flori

@sebasguts
Copy link
Member

@sebasguts sebasguts commented Feb 24, 2016

No worries :).
Current plan is to have an interpreter-like interface, but with some additional features like displaying pictures in the notebook and supporting ipywidgets. What kind of features would you like to see?

@nthiery nthiery modified the milestones: Month 12: 2016-08-31, D4.4 Mar 22, 2016
@jdemeyer jdemeyer changed the title D4.4: Basic Jupyter interface for GAP, PARI/GP , SageMath, Singular D4.4: Basic Jupyter interface for GAP, PARI/GP, SageMath, Singular May 11, 2016
@markuspf
Copy link
Member

@markuspf markuspf commented Jul 26, 2016

If the GAP Jupyter notebook is supposed to go into sagemath (cloud?), someone will have to tell me how to get it there. It's not obvious to me how that works.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Jul 26, 2016

On Tue, Jul 26, 2016 at 08:41:24AM -0700, Markus Pfeiffer wrote:

If the GAP Jupyter notebook is supposed to go into sagemath (cloud?),
someone will have to tell me how to get it there. It's not obvious to
me how that works.

I guess this could boil down to something like:

  • Preparing a gap package with everything needed to run the gap kernel
    (if needed)

  • Preparing a Sage package. That is essentially an install shell
    script in /build/pkg/jupyter_kernel_gap/ + some metadata
    files; see:
    http://www.sagemath.org/git-developer-guide/packaging.html.

    The install script would:

    • install the gap package in the gap tree in the Sage distribution:
      /local/gap/latest
    • copy the kernel file in:
      /local/share/jupyter/kernels
  • Creating a ticket to get this as optional and then standard Sage
    package.

We certainly can help for the latter parts!

Cheers,
Nicolas

@dimpase
Copy link
Contributor

@dimpase dimpase commented Jul 27, 2016

My understanding that this should happen simultaneously (ideally, at least) with libGAP (a Sage package, a C-library interface to GAP kernel) being replaced with what Markus is going to provide.

And GAP jupyter interface would become another way to launch GAP as a part of "Sage distro". Currently we have "sage --gap" launching GAP in terminal, then we would have as well something like "sage --gap-jupyter". I don't know how this will be packaged in GAP. Is there a (half)working prototype already?

@markuspf
Copy link
Member

@markuspf markuspf commented Aug 1, 2016

Thanks @nthiery.

I don't think the Jupyter Kernel is an appropriate GAP Package, since it's a program that wraps around GAP (and can't be loaded in any sensible way).
For the initial implementation as it is available now we only need to have the kernel installed for Jupyter, and make sure that the correct configuration (i.e. the GAP executable) is available.

There might be some merit in distributing this wrapper as part of GAP 4.9.

Is there any way for me to get access to a sage development system without having to install it myself? My operating system is not supported by sage (and it seems a lot of effort to get any development system to run at all).

With the prototype kernel, of course some of the requirements for contributions might be debated by the sage community.

At this point it is important that we provide a deliverable which is not just a link, and I am not sure what that means for a "Basic Jupyter Interface for GAP". The kernel has been available since the kick-off meeting in September last year, and I have worked on improving it since.

The C library is a different beast entirely and in my mind has little to do with this deliverable at this point.

@dimpase
Copy link
Contributor

@dimpase dimpase commented Aug 1, 2016

What kind of OS is this? Do you mean to say there is no binary available, or that it's really unsupported, like it's something like OpenBSD or Hurd?

@dimpase
Copy link
Contributor

@dimpase dimpase commented Aug 1, 2016

If your box supports VMWare or VirtualBox, you can install a supported OS into it and build Sage there.

@videlec
Copy link
Contributor

@videlec videlec commented Aug 1, 2016

... or qemu, docker. You can also open an account on https://cloud.sagemath.com where you can do Sage development online (not very handy). - for development you'd most probably need a paid-for upgrade.

@slel
Copy link
Contributor

@slel slel commented Aug 2, 2016

To use SageMathCloud at https://cloud.sagemath.com for Sage development,

  • open an account,
  • create a project,
  • email help@sagemath.com with the project id
    and ask them to upgrade the project for Sage development
  • you can ssh into your project (after starting it from the web interface)
  • a good tool is tmux (or screen) in combination with ssh
  • once your project is upgraded you have network access from it,
    so you can git clone etc.
@dimpase
Copy link
Contributor

@dimpase dimpase commented Aug 2, 2016

By the way, it is interesting to hear that it has nothing to do with libGAP or its replacement. I believe that back in Bremen we heard something different, although I might have misunderstood this.

@markuspf
Copy link
Member

@markuspf markuspf commented Aug 4, 2016

We just published a Docker container to deploy the basic GAP/Jupyter interface here: https://github.com/gap-system/gap-docker-jupyter, I also wrote a blogpost about it there: http://opendreamkit.org/activities/2016-08-03-gap-docker-jupyter/ (btw, these activities are not very discoverable or visible on the website).

The roadmap for the next iteration of the Jupyter interface for GAP is to export an API from GAP for the purposes of using it as a library in the Jupyter kernel, and to get rid of the REPL wrapping, as its really fragile. Currently work on this is scheduled for September. We also have two project students contributing GUI elements for method selection/availability, subgroup explorer, and hopefully also an interactive Todd-Coxeter, as currently available in xgap.

I hope the CodeMirror GAP syntax highlighting and formatting helper will be ready for merge sooner rather than later, too.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Jan 6, 2017

Hi @sebasguts, @markuspf, @jdemeyer,

We will all be attending the Jupyter workshop in two weeks from now. Shall we use the occasion to have a quick sprint together and finalize this ticket? All that remains to do is the pip/sage packaging. Sitting side by side, this should be a breeze.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Jan 6, 2017

+1

Note that I will arrive Tuesday evening in Edinburgh.

@sebasguts
Copy link
Member

@sebasguts sebasguts commented Jan 6, 2017

Sounds good to me. I will be there the whole week, so plenty of time.

@dimpase
Copy link
Contributor

@dimpase dimpase commented Jan 6, 2017

I will be there only for Thursday and Friday, as I must attend to visitors in Oxford.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Jan 19, 2017

I just volunteered to lead this deliverable.

@markuspf
Copy link
Member

@markuspf markuspf commented Jan 20, 2017

The GAP jupyter kernel is now available as a pip package: https://pypi.python.org/pypi/jupyter_gap_wrapper/0.8 (though I am still fiddling with the release process, in particular I'd like to synchronise the sources available from github and from pypi). Also note that I am more actively working on a standalone version using GAP itself (no python in between).

@markuspf
Copy link
Member

@markuspf markuspf commented Jan 20, 2017

Right, I renamed the package to jupyter-kernel-gap to be in line with jupyter-kernel-singular: https://pypi.python.org/pypi/jupyter-kernel-gap and I polished it a little bit.

I will get @mcmartins to integrate the syntax highlighting we have into it too (today).

@sebasguts
Copy link
Member

@sebasguts sebasguts commented Jan 26, 2017

So, even if everything is already checked in the boxes above, I will give a quick summary on the jupyter kernel for Singular:

  • Kernel is available on PyPI as jupyter_kernel_singular
  • Kernel is integrated into SageMathCloud
  • Fully functional interface to Singular
  • Either interacts directly with the Singular interpreter from Python via the PySingular Python module, available on PyPI, or
    with the REPL via pexpect.
  • IPyWidgets can be used for interactions
  • Images can be created using surf, an external program, and Singular's plot_jupyter.lib
  • Example Notebook
  • Detailed installation instructions
  • Direct link to screenshot
@nthiery
Copy link
Contributor

@nthiery nthiery commented Feb 6, 2017

Dear M12 deliverable leaders,

Just a reminder that reports are due for mid-february, to buy us some time for proofreading, feedback, and final submission before February 28th. See our README for details on the process.

In practice, I'll be offline February 12-19, and the week right after will be pretty busy. Therefore, it would be helpful if a first draft could be available sometime this week, so that I can have a head start reviewing it.

Thanks in advance!

@minrk
Copy link
Contributor Author

@minrk minrk commented Feb 9, 2017

@nthiery my understanding is that this report is all complete as of November. Is there anything more to do?

@nthiery
Copy link
Contributor

@nthiery nthiery commented Feb 10, 2017

Hi @minrk,
For the report itself, all that remains to do is fetching the latest github issue description and adding a screenshot for the GAP interface (it currently is a copy of the screenshot for Singular).
Other than this, the more of the remaining items we can tick in the issue description, the better. @jdemeyer?

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 11, 2017

I'll have a look at this on monday.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 17, 2017

@markuspf : could you provide an example notebook which shows a simple GAP computation? I know essentially nothing of GAP, so it's best if somebody else does that.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 21, 2017

What exactly does "SageMathCloud integration" mean?

@nthiery
Copy link
Contributor

@nthiery nthiery commented Feb 21, 2017

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 22, 2017

I just sent a request to help@sagemath.com for the GAP and PARI/GP Jupyter kernels.

The report is updated with examples of all 4 kernels (thanks to @dimpase for the GAP example).

Note that some Sage tickets are still in needs_review, I hope that is not a problem.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Feb 22, 2017

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 23, 2017

They are non controversial

Well, the PARI one might be because I'm adding lots of patches to PARI which are not (yet?) accepted by upstream.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Feb 23, 2017

I just proofread the issue description and report. I made some minor edits (larger screenshots, markdown syntax fixes to not confuse pandoc, fixed our Makefile so that trac references would not get messed up). I believe it's good to go, up to the tick boxes for the SMC integration. Please ping me if there is progress there, and I'll then submit. Otherwise, I'll wait until Monday and submit as is.

Cheers,

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Feb 27, 2017

I made some minor edits to the GitHub issue description. If you regenerate the corresponding TeX, you can submit.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Feb 27, 2017

Done and submitted!
Thank you @jdemeyer, @markuspf, @sebasguts, @minrk, and all those that contributed to this deliverable. It's great for our users to have Jupyter as unified interface for all our systems.
Now on to the next stage with #96 to get the interfaces even better!

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Apr 25, 2017

I just noticed now that the Sage ticket for the GAP kernel still needs review: https://trac.sagemath.org/ticket/21416

If you didn't know what to do on thursday, please review this ticket!

@slel
Copy link
Contributor

@slel slel commented Jul 30, 2017

Regarding CoCalc (formerly SageMathCloud) integration of Jupyter kernels:

  • Singular: tested at ODK workshop at ICMS, live in CoCalc since late January 2017
  • GAP: tested at Sage Days 87, live in CoCalc since late July 2017
  • PARI: tested at Sage Days 87, live in CoCalc since late July 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.