Bring Jupyter to Zed Editing #12062

rgbkrk · 2024-05-20T17:01:15Z

Run any Jupyter kernel in Zed on any buffer (editor):

TODO

Lifecycle

Launch kernels on demand
Wait for kernel to be started
Request Kernel info on start
Show in progress indicator

Media Outputs

Render text and tracebacks with ANSI color handling
Render markdown as text
Render PNG and JPEG images using an explicit height based on line-height
Process page data from payloads as outputs

Document

Select code and run
Run current line
Clear previous overlapping runs

Other missing features

The following is a list of missing functionality or expectations that are out of scope for this PR.

Python Environments

Detecting python environments should probably be done in a separate PR in tandem with how they're used with LSP. Users likely want to pick an environment for their project, whether a virtualenv, conda env, pyenv, poetry backed virtualenv, or the system. Related issues:

LSP Integration

Submit complete_request messages for completions to interleave interactive variables with LSP
LSP for IPython semantics (%%timeit, !ls, get_ipython, etc.)

Future release notes

Run code in any editor, whether it's a script or a markdown document

Release Notes:

N/A

toiletsandpaper · 2024-05-25T07:16:48Z

Hi! Great job, man. When this will be merged and released I will drop VSCode forever.

Some reminders that you probably missed in your TODOs:

magic line support %pip, %sql, %%timeit, etc.
! shell support !pip, !pwd, !rm -rf /, etc.
plugin system https://github.com/jupyter/jvm-magics

And again, great job!

rgbkrk · 2024-05-25T21:24:50Z

Hi! Great job, man. When this will be merged and released I will drop VSCode forever.

That rocks. There are more improvements to make here that are going to need help in GPUI land, mostly since this isn't a browser. I'll have to review what else is in #9778 as I flesh this first version out.

Some reminders that you probably missed in your TODOs:

magic line support %pip, %sql, %%timeit, etc.

! shell support !pip, !pwd, !rm -rf /, etc.

plugin system https://github.com/jupyter/jvm-magics

Magics and shell commands work out of the box with IPython (and other kernels that have them):

However, completions are not hooked up. I'll leave that as an exercise for a future contributor so we can have the basics soon.

And again, great job!

Thank you!

Bonus as of now -- plots (and images) are working:

plots.mp4

bicharhamid · 2024-05-26T22:57:32Z

Amazing! Thank you for creating this. By any chance could you add support to Ruby?

rgbkrk · 2024-05-27T12:22:24Z

Amazing! Thank you for creating this. By any chance could you add support to Ruby?

Support should be built in but there's some issues on the ruby jupyter kernel. Would you be interested in helping them bring the kernel up to date? SciRuby/iruby#340

rgbkrk · 2024-05-28T16:25:25Z

Yikes, looks like I need some way to handle when more than 256 lines come in. We need a scrollable output area when that happens or we have to trim it.

jansol · 2024-05-29T19:26:39Z

crates/runtimes/src/outputs.rs

+    pub status: ExecutionStatus,
+}
+
+pub fn svg_to_vec(text: &str, scale: f32) -> Result<OutputType> {


I had the impression that the GPUI image view can already render SVGs and SVG files are just forced to use the text editor instead of the image preview because people want to edit them. Could that functionality be reused here instead of reimplementing it?

The SVG renderer is based on reading a path. I should probably drop SVG support entirely from this PR because I've disabled it above due to not having full SVG features enabled, like text:

Yeah, shipping the initial version without SVG and later improving the already existing image view to handle the needed SVG features sounds like a preferrable approach to me.

rgbkrk · 2024-05-31T22:07:46Z

I'm going on vacation this coming week so I'll just post a few in-progress details here:

With Introduce async-dispatcher runtime feature zeromq/zmq.rs#191 landed, I'll be updating runtimelib to allow for swapping out the runtime so that Zed can use it's own spawn implementation instead of having Tokio in the mix. Once I'm sure of that I'll be shipping a new Rust ZeroMQ release. PR for runtimelib Introduce async dispatcher runtimed/runtimed#104
Each workspace should have its own spawned task for tracking runtimes, so that we have a WindowContext we can use with ViewContext. I was running into issues trying to bring it all together with the global RuntimeManager being a ModelContext.

as-cii · 2024-06-05T14:19:48Z

Heads-up: I needed the block resize feature, so I pulled those changes into main. It was pretty additive so I think you shouldn't see too many merge conflicts.

rgbkrk · 2024-06-07T19:09:06Z

Heads-up: I needed the block resize feature, so I pulled those changes into main. It was pretty additive so I think you shouldn't see too many merge conflicts.

Rad, glad to see that landed.

rgbkrk · 2024-06-10T16:28:30Z

Alright I'm back from vacation and ready for the next phase of this PR.

I want to get all the tasks that are spawned to listen for events for each execution view to instead get turned into one listener that will route messages to the right execution view. In order to do that, I'll need the runtime manager (model) to push execution requests over to a model that will emit events targeting a view for the session.

rgbkrk · 2024-06-17T15:34:12Z

assets/keymaps/default-macos.json

+      // Bindings for running code in the editor
+      "cmd-enter": "repl::Run"
+    }
+  },


I'm going to take these out for now to make this feature opt-in for now.

rgbkrk · 2024-06-17T17:03:38Z

I've opted to bring this in behind a feature flag. It's not ready for primetime just yet. Follow on PRs will be coming.

Run any Jupyter kernel in Zed on any buffer (editor): <img width="1074" alt="image" src="https://github.com/zed-industries/zed/assets/836375/eac8ed69-d02b-4d46-b379-6186d8f59470"> ## TODO ### Lifecycle * [x] Launch kernels on demand * [x] Wait for kernel to be started * [x] Request Kernel info on start * [x] Show in progress indicator * [ ] Allow picking kernel (it defaults to first matching language name) * [ ] Menu for interrupting and shutting down the kernel * [ ] Drop running kernels once editor is dropped ### Media Outputs * [x] Render text and tracebacks with ANSI color handling * [x] Render markdown as text * [x] Render PNG and JPEG images using an explicit height based on line-height * ~~Render SVG~~ -- not happening for this PR due to lack of text in SVG support * [ ] Process `update_display_data` message and related `display_id` * [x] Process `page` data from payloads as outputs * [ ] Render markdown as, well, rendered markdown -- Note: unsure if we can get line heights here ### Document * [x] Select code and run * [x] Run current line * [x] Clear previous overlapping runs * [ ] Support running markdown code blocks * [ ] Action to export session as notebook or output files * [ ] Action to clear all outputs * [ ] Delete outputs when lines are deleted ## Other missing features The following is a list of missing functionality or expectations that are out of scope for this PR. ### Python Environments Detecting python environments should probably be done in a separate PR in tandem with how they're used with LSP. Users likely want to pick an environment for their project, whether a virtualenv, conda env, pyenv, poetry backed virtualenv, or the system. Related issues: * zed-industries#7646 * zed-industries#7808 * zed-industries#7296 ### LSP Integration * Submit `complete_request` messages for completions to interleave interactive variables with LSP * LSP for IPython semantics (`%%timeit`, `!ls`, `get_ipython`, etc.) ## Future release notes - Run code in any editor, whether it's a script or a markdown document Release Notes: - N/A

altaic · 2024-06-19T08:41:32Z

This is really great, thank you! I wonder, would one be able to use/adapt this for marimo support as well?

bicharhamid · 2024-07-01T20:07:10Z

Hey man, any updates on how we can get this? Even the test version so far looks amazing to use

rgbkrk · 2024-07-04T00:13:40Z

Looking to ship this in the next couple weeks. Refining the UI for active kernels at the moment and introducing some ways to configure default kernels in #13625.

Initial runtimes UI panel. The main draw here is that all message subscription occurs with two background tasks that run for the life of the kernel. Follow on to #12062 * [x] Disable previous cmd-enter behavior only if runtimes are enabled in settings * [x] Only show the runtimes panel if it is enabled via settings * [x] Create clean UI for the current sessions ### Running Kernels UI <img width="205" alt="image" src="https://github.com/zed-industries/zed/assets/836375/814ae79b-0807-4e23-bc95-77ce64f9d732"> * [x] List running kernels * [x] Implement shutdown * [x] Delete connection file on `drop` of `RunningKernel` * [x] Implement interrupt #### Project-specific Kernel Settings - [x] Modify JupyterSettings to include a `kernel_selections` field (`HashMap<String, String>`). - [x] Implement saving and loading of kernel selections to/from `.zed/settings.json` (by default, rather than global settings?) #### Kernel Selection Persistence - [x] Save the selected kernel for each language when the user makes a choice. - [x] Load these selections when the RuntimePanel is initialized. #### Use Selected Kernels - [x] Modify kernel launch to use the selected kernel for the detected language. - [x] Fallback to default behavior if no selection is made. ### Empty states - [x] Create helpful UI for when the user has 0 kernels they can launch and/or 0 kernels running <img width="694" alt="image" src="https://github.com/zed-industries/zed/assets/836375/d6a75939-e4e4-40fb-80fe-014da041cc3c"> ## Future work ### Kernel Discovery - Improve the kernel discovery process to handle various installation methods (system, virtualenv, poetry, etc.). - Create a way to refresh the available kernels on demand ### Documentation: - Update documentation to explain how users can configure kernels for their projects. - Provide examples of .zed/settings.json configurations for kernel selection. ### Kernel Selection UI - Implement a new section in the RuntimePanel to display available kernels. - Group on the language name from the kernel specification - Create a dropdown for each language group to select the default kernel. Release Notes: - N/A --------- Co-authored-by: Kirill <kirill@zed.dev>

cla-bot bot added the cla-signed The user has signed the Contributor License Agreement label May 20, 2024

rgbkrk mentioned this pull request May 20, 2024

Jupyter Notebook #5273

Open

1 task

rgbkrk force-pushed the runtimes branch from 56bc36a to 7e0cb95 Compare May 21, 2024 13:41

rgbkrk mentioned this pull request May 21, 2024

Stick data types on MimeType enums runtimed/runtimed#98

Closed

SomeoneToIgnore assigned rgbkrk May 21, 2024

rgbkrk force-pushed the runtimes branch 4 times, most recently from c0a8895 to e4524e5 Compare May 22, 2024 18:46

rgbkrk changed the title ~~[WIP] Bring Hydrogen to Zed~~ [WIP] Bring Jupyter to Zed Editing May 22, 2024

JosephTLyons mentioned this pull request May 22, 2024

Console for print/debugging #12065

Open

1 task

rgbkrk force-pushed the runtimes branch 3 times, most recently from e8eb9d4 to 9398745 Compare May 24, 2024 02:32

This was referenced May 24, 2024

Include date in headers SciRuby/iruby#340

Open

Resize blocks by block_id #12261

Closed

rgbkrk force-pushed the runtimes branch from 9334687 to cc78e4b Compare May 25, 2024 18:44

rgbkrk force-pushed the runtimes branch from dde6c41 to e23086b Compare May 25, 2024 23:54

rgbkrk force-pushed the runtimes branch from e803817 to 9a00e93 Compare May 29, 2024 19:22

jansol reviewed May 29, 2024

View reviewed changes

rgbkrk force-pushed the runtimes branch 2 times, most recently from d4b081e to 4ac882c Compare May 29, 2024 22:02

This was referenced May 30, 2024

Is it possible to compile packages and use them on the web? tectonic-typesetting/tectonic#677

Open

MathJax subset to PNG tectonic-typesetting/tectonic#1199

Open

hook up async dispatcher in repl setup

4052c05

rgbkrk force-pushed the runtimes branch from eb28c3e to 4052c05 Compare June 2, 2024 20:16

fully remove tokio from repl crate

e5e5a4a

rgbkrk added 4 commits June 7, 2024 14:13

Merge branch 'main' into runtimes

0400f10

bump runtimelib, alacritty

58f04af

💄

4c5a9d0

remove unused dep

5b61a9c

rgbkrk added 2 commits June 12, 2024 12:58

Merge branch 'main' into runtimes

9a4e775

initial runtime settings

ad499e1

rgbkrk commented Jun 17, 2024

View reviewed changes

rgbkrk added 2 commits June 17, 2024 08:39

read settings to enable/disable jupyter repl

14d53e8

observe settings to enable and disable runtime manager

260d95c

rgbkrk force-pushed the runtimes branch from 56c0a10 to 260d95c Compare June 17, 2024 16:45

rgbkrk mentioned this pull request Jun 17, 2024

New desktop binaries? nteract/nteract#5753

Open

rgbkrk changed the title ~~[WIP] Bring Jupyter to Zed Editing~~ Bring Jupyter to Zed Editing Jun 17, 2024

rgbkrk marked this pull request as ready for review June 17, 2024 16:54

rgbkrk merged commit 221edfc into main Jun 17, 2024
11 checks passed

rgbkrk deleted the runtimes branch June 17, 2024 17:02

rgbkrk mentioned this pull request Jul 1, 2024

Runtimes UI Starter #13625

Merged

14 tasks

rgbkrk added the jupyter label Jul 8, 2024

rgbkrk mentioned this pull request Jul 8, 2024

Tracking Issue for Zed REPL #13942

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bring Jupyter to Zed Editing #12062

Bring Jupyter to Zed Editing #12062

rgbkrk commented May 20, 2024 •

edited

Loading

toiletsandpaper commented May 25, 2024 •

edited

Loading

rgbkrk commented May 25, 2024

bicharhamid commented May 26, 2024

rgbkrk commented May 27, 2024

rgbkrk commented May 28, 2024 •

edited

Loading

jansol May 29, 2024

rgbkrk May 29, 2024

jansol May 29, 2024

rgbkrk commented May 31, 2024 •

edited

Loading

as-cii commented Jun 5, 2024

rgbkrk commented Jun 7, 2024

rgbkrk commented Jun 10, 2024 •

edited

Loading

rgbkrk Jun 17, 2024

rgbkrk commented Jun 17, 2024

altaic commented Jun 19, 2024 •

edited

Loading

bicharhamid commented Jul 1, 2024

rgbkrk commented Jul 4, 2024

Bring Jupyter to Zed Editing #12062

Bring Jupyter to Zed Editing #12062

Conversation

rgbkrk commented May 20, 2024 • edited Loading

TODO

Lifecycle

Media Outputs

Document

Other missing features

Python Environments

LSP Integration

Future release notes

toiletsandpaper commented May 25, 2024 • edited Loading

rgbkrk commented May 25, 2024

bicharhamid commented May 26, 2024

rgbkrk commented May 27, 2024

rgbkrk commented May 28, 2024 • edited Loading

jansol May 29, 2024

Choose a reason for hiding this comment

rgbkrk May 29, 2024

Choose a reason for hiding this comment

jansol May 29, 2024

Choose a reason for hiding this comment

rgbkrk commented May 31, 2024 • edited Loading

as-cii commented Jun 5, 2024

rgbkrk commented Jun 7, 2024

rgbkrk commented Jun 10, 2024 • edited Loading

rgbkrk Jun 17, 2024

Choose a reason for hiding this comment

rgbkrk commented Jun 17, 2024

altaic commented Jun 19, 2024 • edited Loading

bicharhamid commented Jul 1, 2024

rgbkrk commented Jul 4, 2024

rgbkrk commented May 20, 2024 •

edited

Loading

toiletsandpaper commented May 25, 2024 •

edited

Loading

rgbkrk commented May 28, 2024 •

edited

Loading

rgbkrk commented May 31, 2024 •

edited

Loading

rgbkrk commented Jun 10, 2024 •

edited

Loading

altaic commented Jun 19, 2024 •

edited

Loading