[electron] Jupyter Integration

[anirrudh]

After discussion with @israelvicars, I started this issue for tracking the progress on Jupyter Integration.

Workflow

The current user workflow can be broken down like so:

Before Application Launch

• Launch Application [User] -> Start OmniSciDB
• Launch Application [User] -> Start Immerse
• Launch Application [User] -> Start JupyterLab

implementation details:

• @israelvicars noted here that we need to wait for JupyterLab to finish launching until the frontend renders. This will avoid us having to mess with the Jupyter Button in the future.

• Jupyter will be launched using a bash script. This path will be in the config file as attribute JupyterPath. The output will be piped back into node, parsed, url extracted and then written to a temporary local file. The file will simply contain the jupyterlab url.

• The code will only make two assumptions to remain as generic as possible.

  1. @xmnlab is developing the conda-datascience-installer. I will assume that on macOS, the conda install path will be at ~/conda/, and that the 'omnisci' environment will exist at ~/conda/envs/omnici which will also contain the jupyter executable.
  2. The base environment will be untouched, and an omnisci kernel will be created to launch in the environment.

After Application Launch:

Short-Term Delivarable

• Click Jupyter Button -> Open JupyterLab in WebBrowser
• New Notebook -> Open with connection configured

Long Term

• Click Jupyter Button -> New Window with JupyterLab
• Window Launch -> Notebook + Kernel Pre-loaded
• ???

cc: @dharhas @niviksha @andrewseidl

[dharhas]

We are using conda constructor not miniconda. Your path assumption may be incorrect.

[anirrudh]

We are using conda constructor not miniconda. Your path assumption may be incorrect.

fixed to conda in the issue above.

[anirrudh]

@israelvicars as we were speaking today, it occurred to me all you may need are the template files here. This is going to be the general template:

JupyterServiceStart.sh

#!/bin/bash

J_PORT=9999
J_PATH=~/conda/envs/omnisci/bin/jupyter
J_TOKEN="ea436a31623ee0c6ef25f8f340fcf3bdd17f914260dc3e17620ae04f93dfbb3c"

${J_PATH} lab --no-browser --port ${J_PORT} --LabApp.token=${J_TOKEN}

This would be the shell file we run on start to start the jupyter server. @xmnlab I am assuming that the environment will contain jupyterlab.

There is also an option for adding the path for the notebooks to be saved to. Do we want to set a default path?

Furthermore, @israelvicars we can predict the URL for jupyterlab! I was mistaken; the URL for the jupyterlab for the above example would be:

http://127.0.0.1:<J_Port>/lab?token=<J_TOKEN> or localhost instead of 127.0.0.1, as per usual.

This means that to 'open' jupyterlab in a browser all one would need is the following attached to the onClick function in the electron application:
(where config.jupyterPath.url = http://127.0.0.1:9999/token=...)

const { shell } = require('electron');
const url = config.jupyterPath.url;
function openJupyter() {
      shell.openExternal(url);
}
...
document.querySelector('#btnJupyter').addEventListener('click', () => {
    openJupyter()
})

I know we are using typescript for the project, but these are just my thoughts.

What do you think @israelvicars? Does this sound feasible?

cc: @andrewseidl @xmnlab

[xmnlab]

@anirrudh the current default path (v0.0.9) is ${HOME}/omnisci/bin/jupyter-lab ... I am trying to change to the new PATH .. ${HOME}/Library/... but I am having some problems with PATH with space, I am investigating that ...

[xmnlab]

Currently, conda constructor doesn't support target directory with spaces, so we cannot use
"${HOME}/Library/Application\ Support/OmniSci/" for the default target (default_prefix), see: conda/constructor#430

@israelvicars @andrewseidl, can we keep using the folder ${HOME}/omnisci for now?

[israelvicars]

Noting that we can ping localhost:<port>/api until its successful to verify the service has launched on startup, like we do for the web server before opening up a window with Immerse. Thanks for finding that @anirrudh!

[andrewseidl]

@xmnlab yeah, that's probably fine for now. Another option would be under ${HOME}/Documents/omnisci / app.getPath('documents') in electron. My personal preference is for the latter, but either works.

[xmnlab]

@andrewseidl that sounds great! I will change to Documents, so I don't need to create a link for the notebooks to the Documents folder (as we discussed before).

[xmnlab]

@andrewseidl I was talking to @israelvicars and he commented that probably would be better to have the target parent directory to $HOME (instead of $HOME/Documents), and instead of omnisci store that as omnisci-datascience. And we can have a symbolic symbol from omnisci-examples to $HOME/Documents

let me know your thoughts about that.

[andrewseidl]

Yeah, directly in $HOME is fine. What's the use case for the omnisci-examples symlink?

[xmnlab]

there was a discussion about saving the notebook into the Documents folder .. but as omnisci-examples is a conda package, we can create a symlink to Documents. I tried to find the history about this but I didn't find anything.
I have no problem not having the link for now.

[anirrudh]

cc: @israelvicars @nikitaved @andrewseidl

Setting up OmnisciDB connection on Notebook Start

In order to setup the omnisci db connection, we need to run a python command in the ipython process that connects to the db. The command should also be hidden from the user notebook, and the variables instantiated properly.

Install

As a part of the data science installer, we should add a shell command to ensure that a default ipython profile exists for the user.

ipython profile create <custom_name>

Note that this command is unnecessary if using default profile.
NOTE The default profile name is profile_default.

After this point, the directory ~/.ipython/<profile_name>/startup should be created.

• This would be ~/.ipython/profile_default if not using a custom name.

`~/ipython/<profile_name>/startup/README

In this folder, there is a readme that looks like this:

This is the IPython startup directory

.py and .ipy files in this directory will be run *prior* to any code or files specified
via the exec_lines or exec_files configurables whenever you load this profile.

Files will be run in lexicographical order, so you can control the execution order of files
with a prefix, e.g.::

    00-first.py
    50-middle.py
    99-last.ipy

These dictate the order of the files. I will be using the default 00-first.py file.

Writing the startup command

~/ipython/<profile_name/startup/00-first.py
NOTE This is mocked up code. Need some information regarding username/password detail passing and connection syntax for a more complete solution.

import pyomnisci 
import json

f = json.load(conf.json)

connection = pyomnisci.connect(f.user, f.pass)