Export to .ml with a jupyter-nbconvert command ?

[Naereen]

See this issue on iocaml for reference.

[Naereen]

Note that if a --to ocaml option on the jupyter-nbconvert script is unlikely, a tweak on the --to script should be possible...
Worst case, write a small script ipynb2ml and ship it when opam-installing ocaml-jupyter.

[akabe]

Thank you for often telling me bugs. I reproduced the problem you said. In order to register the ocaml exporter to jupyter-nbconvert, I think I need to write the custom exporter. I will implement it.

$ jupyter nbconvert --to ocaml notebooks/introduction.ipynb 
Traceback (most recent call last):
  File "/home/aabe/.local/bin/jupyter-nbconvert", line 11, in <module>
    sys.exit(main())
  File "/home/aabe/.local/lib/python3.5/site-packages/jupyter_core/application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/aabe/.local/lib/python3.5/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/home/aabe/.local/lib/python3.5/site-packages/nbconvert/nbconvertapp.py", line 313, in start
    self.convert_notebooks()
  File "/home/aabe/.local/lib/python3.5/site-packages/nbconvert/nbconvertapp.py", line 470, in convert_notebooks
    cls = get_exporter(self.export_format)
  File "/home/aabe/.local/lib/python3.5/site-packages/nbconvert/exporters/base.py", line 110, in get_exporter
    % (name, ', '.join(get_export_names())))
ValueError: Unknown exporter "ocaml", did you mean one of: asciidoc, custom, html, html_ch, html_embed, html_toc, html_with_lenvs, html_with_toclenvs, latex, latex_with_lenvs, markdown, notebook, pdf, python, rst, script, selectLanguage, slides?

In addition, an output file from --to script option seems wrong because some lines require ;; at the end of them. Do you know how to overwrite the script exporter?

$ jupyter nbconvert --to script notebooks/introduction.ipynb 
[NbConvertApp] Converting notebook notebooks/introduction.ipynb to script
[NbConvertApp] Writing 867 bytes to notebooks/introduction.ml
$ cat notebooks/introduction.ml
let x = List.map (fun x -> x * 2) [1; 3; 5; 7; 9]

#thread ;;
#require "jupyter.notebook" ;;

JupyterNotebook.display "text/html" "<b>Hello, World!</b>"

let ic = open_in_bin "datasets/lenna.png" in
let n = in_channel_length ic in
let data = really_input_string ic n in
close_in ic ;
JupyterNotebook.display ~base64:true "image/png" data

#require "jupyter.archimedes"

let vp = A.init ~w:560. ~h:260. ["jupyter"] in
A.Axes.box vp ;
A.set_color vp A.Color.red ;
A.fx vp sin 0.0 10.0 ;
A.close vp

let rec walk acc p i =
  if i = 0 then List.rev acc else begin
    let q = if Random.bool () then succ p else pred p in
    walk (q :: acc) q (pred i)
  end in
let vp = A.init ~w:560. ~h:260. ["jupyter"] in
A.Axes.box vp ;
A.set_color vp A.Color.red ;
A.List.y vp ~style:`Lines (List.map float @@ walk [] 0 1000) ; (* Plot a random walk of 1000 points. *)
A.close vp

[Naereen]

Actually, because you handled correctly the metadata tags, the --to script already works better than it did with iocaml, because the output file has the correct extension.

The obvious advantage of using a --to ocaml or a separate script would be to convert the Markdown and non-code cells to OCaml comments.

[akabe]

OK. I think the custom exporter can be released in 1-2 weeks maybe.

[Naereen]

Do you know how to overwrite the script exporter?

• This paragraph of the doc shows that it is possible to ship a Python library/package/module that will register --to ocaml as a valid jupyter nbconvert format.

[Naereen]

I have started to do it, can you let me try please?

[akabe]

Thank you! Of course.

[Naereen]

Should be OK 🚀 ! See https://github.com/Naereen/Jupyter-NBConvert-OCaml
It works perfectly fine on my machine, I have not (and will not) test it on other machines.

See this example to see the difference between --to script and --to ocaml. Any suggestion on how to improve the --to ocaml step? (the template and filter are easy to modify to basically do whatever we want).

I think I will let you take it from where... I don't know how you could also for a pip install or python setup.py install step in the installation phase of ocaml-jupyter...
If you want this to be published on Pypi, it would be make more sense if it were by you, right?
(but if you don't know how or don't want to do it, I already published a few packages on Pypi, I could do it again, as you wish)

[Naereen]

And for less trivial examples of outputs, I just re-generated all the .ml files from my OCaml notebooks present in this directory.

[Naereen]

The code (__init__.py and template) are here : https://github.com/Naereen/Jupyter-NBConvert-OCaml/tree/master/jupyter_nbconvert_ocaml

[akabe]

@Naereen Great! I tried your exporter and got a file including code with markdown comments. However the output .ml file might have syntax errors due to lacks of ;; at the end of cells (which I suggested above). For example, a notebook like

In[1]> 1 + 3 (Shift+Enter)
Out[1]> - : int = 4
In[2]> let x = 42 (Shift+Enter)
Out[2]> val x : int = 42

is exported to the following .ml file

(* In[1]: *)
1 + 3
(* In[2] *)
let x = 42

but the above should be

(* In[1]: *)
1 + 3;;
(* In[2] *)
let x = 42;;

The simplest solution is naively adding ;; at the end of each cell.

[akabe]

I think I will let you take it from where... I don't know how you could also for a pip install or python setup.py install step in the installation phase of ocaml-jupyter...
If you want this to be published on Pypi, it would be make more sense if it were by you, right?

The current installation step seems simple enough. I leave to you whether it is published on PyPI since it is your product. I will add the link to your repository in README.

[Naereen]

OK. But it's already weird to not use ;; in your notebook/repl ocaml session.
It works but it shouldn't really work... The documentation clearly says "Input to the toplevel can span several lines. It is terminated by ;; (a double-semicolon)".
See http://caml.inria.fr/pub/docs/manual-ocaml/toplevel.html

[Naereen]

I can change the filter to check if there is a ;; at the end of each cell, and add it if there is not?

[akabe]

I see. You are right. ocaml jupyter kernel shouldn't execute top-level phrases without ;; at the end according to the specification of ocaml. It is a minor bug I should fix. Thanks.

[Naereen]

Yes but I can tweak the filter and force to add a ;; at the end of each cell... I am trying.

[Naereen]

Oh, I just saw that the GUI allows to save the notebook to a OCaml file, but it uses --to script in the backend.
Just tried something, and to compare, here is first the --to script output:

1 + 3

let x = 42

and then the new --to ocaml output:

(*
This OCaml script was exported from a Jupyter notebook
using an open-source software (under the MIT License) written by @Naereen
from https://github.com/Naereen/Jupyter-Notebook-OCaml
This software is still in development, please notify me of a bug at
https://github.com/Naereen/Jupyter-Notebook-OCaml/issues/new if you find one
*)

(* # Title Markdown cell! *)

(* In[1]: *)


1 + 3;;

(* In[2]: *)


let x = 42;;

[Naereen]

Let's publish it on Pypi like this, I let you add the links on your README, and we will see, if anyone complains about this filter adding ';;' (when needed), I will remove it... Or add options to keep or remove it.
(I will publish on Pypi next week, don't want to rush too much, if I think of ways to improve all this)

[akabe]

Thank you!

[Naereen]

(Naereen/Jupyter-NBConvert-OCaml#1 as soon as I can)

[akabe]

To be honest, I cannot be sure that I should reject cells without ;;. I think this change will not make ocaml-jupyter kernel more useful...

[Naereen]

No I don't see the point in rejecting cells without ';;', don't change anything.

[akabe]

OK. Thanks.