Open a notebook from JupyterLab terminal

[dclong]

Is it possible to open a Jupyter(Lab) notebook from command? The file navigation of JupyterLab is not convenient when I want to work with notebooks in different folders. It definite will makes things much easier if I open a notebook from command line.

[blink1073]

If you mean open JupyterLab itself with a certain file open, you can run jupyter lab foo.ipynb. To open by path within JupyterLab you can select Open From Path from the commands side bar.

[dclong]

I mean open a notebook by path within JupyterLab. Open From Path in commands side bar helps but it's certainly more convenient if we are able to open a notebook from command line in terminal.

[blink1073]

As in, open from a Terminal that you've opened from within JupyterLab?

[dclong]

That's right. Can we open a notebook from a terminal in JupyterLab?

[jasongrout]

That's right. Can we open a notebook from a terminal in JupyterLab?

No, not currently. I'm having difficulty seeing how we would be able to make opening from the jlab terminal work.

Perhaps assigning a keyboard shortcut to the "Open From Path" command makes it convenient enough?

[dclong]

That helps a little bit but not much. A user still have to type in the whole path, which is not convenient. This is related to issues #2160 and #2043. They are probably better ways.

[jpcaram]

This feature makes a lot of sense to me, and not just limited to notebooks, but opening text files in the text editor as well. For example, in a terminal (running within Jupyterlab):
jlabopen somenb.ipynb
or
jlabopen myscript.py
Each would open up a tab, respectively for the specific type of file.

This shouldn't be hard to implement: Since Jupyterlab launches the terminal, it can inject environment variables with the port where Jupyterlab is running. Then, the jlabopen script can talk to the server and tell it to do this.

A possible client-side approach shouldn't be too complicated. The jlabopen call must be caught before it is sent to the "real" terminal on the server.

I don't know about how specifically to implement this, or the difficulties we might encounter, but we could discuss them here.

[dclong]

Gitpod (based on the Theia IDE) supports opening a file from terminal using the command open.

[ktaletsk]

Inside of Cocalc, you can type open in Terminal to open both text documents and Jupiter notebooks, so this would be useful here as well

[jasongrout]

@williamstein - just curious, any hints or general direction on how you implement opening a document in cocalc from the terminal?

[williamstein]

@williamstein - just curious, any hints or general direction on how you implement opening a document in cocalc from the terminal?

Short answer: invent a new fake xterm style control code, write a script called open that emits such a code, and parse it on the client before xterm.js sees it. This is very doable in jlab, and the functionality is EXTREMELY useful and nice to have.

Longer answer:

Step 1: We wrote an open command in Python that outputs a JSON message surrounded by certain escape codes, and has a workaround for tmux users. Here's the code:
https://github.com/sagemathinc/cocalc/blob/master/src/smc_pyutil/smc_pyutil/smc_open.py

It's AGPL licensed, but I'm happy to relicense it BSD in the interest of compatibility, since it would be nice if we solve this problem in a compatible way.

Step 2: The backend terminal server watches for such escape codes in the output stream, and parses them. This is a little tricky, since they might span multiple bursts of output data (it's watching a stream). When it finds such a message, it parses it, then sends it via a separate communications channel to our frontend (the code in the browser client that is managing the terminal):

channel.write({ cmd: "message", payload });

This 'channel' is managed via primus-multiplex, which is a half-broken primus plugin for multiplexing channels over a single websocket (but you can use whatever you use). There's a bunch of other messages that get sent over this same channel, which you'll also perhaps eventually want to implement. Most are relevant to collaboration, e.g., to deal with multiple clients looking at the same terminal with different size screens.

We used to send these messages over the exact same stream that term.js receives, but that kept resulting in subtle bugs and corruption (you'll probably be tempted to take that approach though). By just using two completely different channels of communcation (one for term.js and one for all the other messaging that term.js shouldn't know about) you avoid a lot of pain. That said, due to the minimization of a backend component of jupyter lab (and not worrying about collaboration yet), you'll probably end up taking the path of parsing out the message on the frontend end, rather than the backend.

Alternatively, maybe this functionality could be added to xterm.js itself, and xterm.js would just emit some event when it gets a "non xterm control message", which would be arbitrary JSON. For all I know, xterm.js already has this by now; it would be very natural. I bet @Tyriar has thought about this.

Once you have such communication in place, you could probably do a lot more, e.g., a whole

jlab ...

command that lets you control the client in all kinds of ways.

One subtlety (for us) is: what happens when two people have the same terminal open, and one person types open foo? Our solution is that if they both have the terminal in the foreground, then the file opens. If the terminal is not in the foreground for one user, then nothing happens for that user.

[jasongrout]

Thanks - that sounds like a very interesting approach. If we are collaborating on a terminal in cocalc, so that we see each other's output, and if I use this open command, will it open the document for both of us? It sounds like it will.

[williamstein]

On Thu, May 2, 2019, 12:36 PM Jason Grout ***@***.***> wrote: Thanks - that sounds like a very interesting approach. If we are collaborating on a terminal in cocalc, so that we see each other's output, and if I use this open command, will it open the document for both of us? It sounds like it will.

Yes, it will. But only if you both have the terminal opened. At least that's what it is supposed to do (unless there is a bug). —

…

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5056 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJXS5UO27TQ6QTUEG6BSHLPTM7FHANCNFSM4FN62MDA> .

[Tyriar]

Whenever I've thought about doing this sort of thing the approach of trying to prevent the text going to the backing process seems like an unreliable mess as you can't reliably tell the state of the terminal. For example maybe it's running vim and you type "open blah" and it opens the doc, ok for that example we can avoid the alt buffer but it could still happen when you're in less/git diff/etc. Also line editing keybindings are different for different shells, lots of things can go wrong.

The only approach I would recommend at least currently is adding something to PATH and communicating back to your app, but that can be tricky and maybe even a security risk.

[williamstein]

@Tyriar (lead dev of xterm.js) -- thanks for your comment! You wouldn't watch the stream for open blah, but instead for [control codes]{some json message}[control codes]. This is something that is just as likely to happen by accident in vim as any other xterm control codes.

[Tyriar]

@williamstein oh I missed that part, interesting idea. You could experiment with adding a custom CSI/OSC handler via the recent API:

https://github.com/xtermjs/xterm.js/blob/ebc0083531ff119973998800c1e83fc7e41dadd0/typings/xterm.d.ts#L574-L596

Just make sure you don't conflict with existing ones. I'm interested to hear how this works out if you pursue 🙂

[kwlzn]

+1, this would be awesome. at Twitter we're more often than not navigating a large monorepo in jupyterlab's terminal, so being able to e.g. open ./SomeNotebook.ipynb would be a huge productivity win for loading notebooks from disk into a running JL notebook.

[Quetzalcohuatl]

I have one terminal running Jupyter Lab, and i can open it on my localhost:8888. Would love to just be able to type jupyter lab newnotebook.ipynb or jlabopen newnotebook.ipynb and it opens on localhost:8888 again instead of creating it on localhost:8889. :-)

[dclong]

I really think this is a critical feature for heavy users of JupyterLab. Is there anyway to make it into the 3.0 plan?

[jasongrout]

The best way for something to make it into the 3.0 plan is for someone to volunteer to submit a pr for it.

[EricCousineau-TRI]

EDIT: Er, got confused between this and a related issue.
Moved the content of this comment there: #8959 (comment)

[fperez]

I was just thinking that a jlabopen <fname>-style command that could be a) run from the JLab terminal b) would be equivalent to the user double-clicking on <fname> in the file browser would also be (in addition to many other benefits) a perfect workaround for #2049. If we had this, we can at least open files in a text editor (potentially with a --with flag to add tool options such as markdown preview vs edit, but that can come later) even if hidden, and without the need for further UI support...

I can see two routes for this - one is the client-side logic with terminal codes that @williamstein describes above (thx btw!!). The other would be perhaps having the server listen on a local socket we can communicate with, and pass commands to. The problem with that one is, of course, how to manage the added complexity of terminals/user kernels being run in say a container that's isolated from the server. I don't have enough of the wiring diagram in my head right now to know how complex this would be, curious if folks have any further input that could help clarify which route will be best in the long run?

[jasongrout]

FYI, there is relevant work going on in #9687.

[StefanBrand]

The file navigation of JupyterLab is not convenient when I want to work with notebooks in different folders.

This extension might be relevant:https://github.com/youngthejames/jupyterlab_filetree

[josephko91]

Is there a current working solution for this? I see this issue is quite old and wondering if there is any update. It would be a really beneficial feature.

[minrk]

I just ran into a situation where I want this. It would also be amazing if there could be a way to support the $EDITOR standard, specifically the ability to wait-until-close, so Lab could be used as the editor for e.g. commit messages. That sounds super complicated to communicate through from web -> terminal process.

[krassowski]

wait-until-close [...] That sounds super complicated to communicate through from web -> terminal process.

This is not that complicated - nbdime already does it (closes process when browser tab with the diff closes) - we could reuse the code from there.