Api for editor insets

[jrieken]

This issue track the API proposal for editor insets. The current proposal is this

vscode/src/vs/vscode.proposed.d.ts

Lines 135 to 147 in ffe3749

	export interface WebviewEditorInset {
	readonly editor: TextEditor;
	readonly line: number;
	readonly height: number;
	readonly webview: Webview;
	readonly onDidDispose: Event<void>;
	dispose(): void;
	}
	export namespace window {
	export function createWebviewTextEditorInset(editor: TextEditor, line: number, height: number, options?: WebviewOptions): WebviewEditorInset;
	}

[davidanthoff]

Could there be a feature that such an inset can change its height dynamically depending on the content of the webview?

I would like to use this in the Julia extension in such a way that one can run individual blocks of code from the editor, and then the result of the expression that is run would be shown in an inset below that code. For that result I would like to use a HTML tree view that one can use to explore the structure of a composite type that can be returned from an expression, but that would require that an inset can change its height, for example in reaction to a user expanding a node in the tree.

[jrieken]

Yeah - that's something we can consider but we are also afraid that this makes the editing very nervous. Image you are typing and an inset above you keeps changing height as you type; we will likely fail to keep in the view port stable in those cases.

[davidanthoff]

So the way I would use this is that I would only ever change the height dynamically if the user interacts with some UI element inside the webview, i.e. never in reaction to some event outside of the webview. I think that should work, right?

I've been thinking a bit more about this, and even for many use cases where I don't need the ability to dynamically adjust the height I would often not be able to specify a height when I call createWebviewTextEditorInset. For example, say I'm running a piece of code that generates a static plot figure. What I have is some HTML for that that I want to show, but I wouldn't know what height that would have. So I think I would really need a version of createWebviewTextEditorInset that doesn't take a height parameter at all.

[wadethestealth]

@jrieken I am a developer from nteract/hydrogen, an Atom package that provide inline Jupyter support, and some of the developers and users have wanted us to create an extension in vscode for quite some time as they enjoyed our workflow in Atom. Unfortunely without this api, it is not possible to create our workflow, but now it should be feasible.

In Atom's api, we use a point inside the editor's text buffer to inject an html (technically react) element into the page. Atom allows us to style the element however we wish, as if it was a normal element in a browser, which allows for really unique customization.

I am unsure of the underlying code that allows for this api, but only allowing for a set height is extremely limiting in cases where we don't know the height of our content. You can see this particularly in our case because as we send code for jupyter kernels to execute, we use a tiny loading icon, but then the result that is sent back (which we have no control over and dont know the size of) replaces this icon. From my understanding it seems like once we got the result we would have to render it ourselves to be able to calculate the height and choose a proper inset height (which also is not a multiple of a line number).

Point being, I would love to be able to add in inline html elements, but this webview inset seems restricting because of its height requirements.

[davidanthoff]

@waderyan Not relevant to this discussion, but I would love to see nteract in VS Code!!! Maybe another option would be to just use webviews and the new custom editor option? And then just use the standard nteract UI? Also, there seems to be something going on at #86632. And sorry for the noise here, we should probably open a separate issue if we want to discuss the nteract story further.

[tamuratak]

@jrieken, How about exposing the API of updating the height of insets and delegating the UX of inset height entirely to extension authors? It would reduce the number of complaints about the UX to the VS Code dev team. The authors might know better what is good UX for each situation.

	export interface WebviewEditorInset {
		readonly editor: TextEditor;
		readonly line: number;
		readonly height: number;
		readonly webview: Webview;
		readonly onDidDispose: Event<void>;
		dispose(): void;
		updateHeight(height: number): void;
	}

From the technical point of view, because the insets are implemented with iframe, I think the correct height must be sent from the inside of the iframe. The height can not be known outside the frame.

This is an experiment of mine.

• https://github.com/tamuratak/vscode/tree/inset_update_height
• https://github.com/tamuratak/vscode-extension-samples/tree/inset_update_height

[jrieken]

Yeah, something along those lines would make sense. It could also be an API function inside the iframe/webview so that the code inside of it can announce its new height

[tamuratak]

FYI: A related issue #88243 is going on.

[shaman-apprentice]

Out of curiosity: Why not providing the monaco-editor instance? Then we could use all native monaco-editor API directly, without having to wait for a vscode wrapper API (I personally would love that). Is that a security concern?

[tamuratak]

@jrieken I am glad to hear that you are not negative to add API. I have another suggestion. How about separating creating EditorWebviewZone and creating Webview in the zone? Exposing createWebview to extension authors allows us to implement the lazy loading of EditorInset with onDidChangeTextEditorVisibleRanges, which behaves like the lazy loading of img on HTML.

	export interface WebviewEditorInset {
		readonly editor: TextEditor;
		readonly line: number;
		readonly height: number;
		readonly webview: Webview | undefined;
		readonly onDidDispose: Event<void>;
		dispose(): void;
		createWebview(): Promise<Webview | undefined>;
		disposeWebview(): void;
	}

An experiment of mine.

• https://github.com/tamuratak/vscode/tree/expose_create_webview_2
• https://github.com/tamuratak/vscode-extension-samples/tree/example_expose_create_webview/helloworld-sample

[shaman-apprentice]

May I kindly ask for a very rough estimation if / when something like this is to come? :) I am looking forward into this a lot and it would increase my already great joy for vscode ;)

[usernamehw]

It was sort of already asked before, but still not answered: #66418 (comment) , #66418 (comment)

Would it be possible to have an alternative rendering mode of webview floating on top of the code? (work as an absolutely/fixed positioned element)

[xixixao]

Yeah - that's something we can consider but we are also afraid that this makes the editing very nervous. Image you are typing and an inset above you keeps changing height as you type; we will likely fail to keep in the view port stable in those cases.

@jrieken It might be useful to have a general approach for maintaining current scroll position of a cursor across changing content. I'm not sure whether this is technically feasible with Monaco or not. Here's a scenario which could benefit from this (regardless of the inset API):

Consider last selected cursor to be the primary cursor. If I place several cursors in a file, and then start typing new lines, the primary cursor (which I was looking at) quickly disappears from the view.

[davidanthoff]

@jrieken would be awesome if you could give us an update on the status of this.

There is a major discussion going on in the Julia language community right now whether most IDE efforts should focus on VS Code. This feature here is by far the most discussed obstacle right now (because the Atom Julia plugin supports it).

I think just a short status update would be great. Is this just on the backlog for a short while? Are there still plans to eventually finish this? Or has this idea be abandoned?

[jrieken]

There are currently no plans to finalise this API (say within the next 3-6 months)

We are currently focusing on native notebooks support (code editors interleaved with rich output, like jupyter notebooks) and we think that a super-set to this proposal. The title of julia-vscode/julia-vscode#178 suggests that it would also benefit from notebooks support. Once notebooks are done, we will revisit this item.

[davidanthoff]

Thanks for clarifying!

Yes, the notebook stuff is fantastic, I've had a notebook branch for the Julia extension for a couple of weeks now and it is probably the feature I'm most excited about in a long time.

[peey]

While I'm liking notebooks, that seems to be suitable for very highly structured UI, where what you write is limited to the cell structure.

This API seems to be much more powerful since you can inline things in a regular text / code file.

I don't want this to die just because notebooks are able to solve a subset of problems that this does. I hope when you re-visit this after notebooks, this API also makes it through.

[a-stewart]

@jrieken, How about exposing the API of updating the height of insets and delegating the UX of inset height entirely to extension authors? It would reduce the number of complaints about the UX to the VS Code dev team. The authors might know better what is good UX for each situation.

	export interface WebviewEditorInset {
		readonly editor: TextEditor;
		readonly line: number;
		readonly height: number;
		readonly webview: Webview;
		readonly onDidDispose: Event<void>;
		dispose(): void;
		updateHeight(height: number): void;
	}

From the technical point of view, because the insets are implemented with iframe, I think the correct height must be sent from the inside of the iframe. The height can not be known outside the frame.

This is an experiment of mine.

• https://github.com/tamuratak/vscode/tree/inset_update_height

-- snipped --

With regards to this proposal, perhaps another option would be to make height mutable, and use get and set methods.
Then the update from the extension would just look like webviewEditorInset.height = 20?

@jrieken - I know that there aren't plans to finalise this API until after the notebooks work is complete, but I would be interested to know if this feature in general is something that is likely to make it to the main api, or is it superseded by the notebooks api and will likely be dropped?

[zimonitrome]

Any updates on this? Is this API still in progress or was similar functionality implemented in another way?

[Zenahr]

Necroing. Would be awesome if developers could build something in the spirit of MemefulComments.

[PEZ]

This would be fantastic. I would use this in similar ways to what @davidanthoff describes, but for Clojure code evaluations. If the view can be scrolled, I could make do without dynamic height.

Also, if it could be configured either as an inset or a floating view, fixed to some document position, it could be be used for #3220.

[jeronimoek]

Any updates so far?

[ihdavids]

Any updates on this? It has been asked for in so many different ways since about 2014+

[zimonitrome]

I am once again asking for this feature.

[thefakeplace]

This has a ton of upvotes, seems to already be implemented in a PR, and has been sitting around for years.

Because it's already implemented + the upvotes, seems like low hanging fruit for big impact.

What work/concerns are left to address? I would be happy to help.

[Ivorforce]

Hiya, for anyone that's actively waiting on this feature: The PR has long been merged, and the feature is available on the main branch.

Do that mean it's available in production? No, but it is available in VSCode Insiders. Use this guide to set it up - the ID of the feature is editorInsets.

Of course, I too hope it will get accepted into production, but if you want to use it for a personal project (like me), this is good enough.

[a-stewart]

The recent release notes for 1.79 included some interesting insets used by the the Github Copilot extension.

https://code.visualstudio.com/updates/v1_79#_editor-chat

I was wondering if that was using inset webviews or some other ui element?
If it is using inset webviews, does that imply that this feature may be being looked again with a view to finalising it?

(Also unrelated but that inline side-by-side diff looks cool too!)

[alexisgauba]

As an extension dev would be awesome to have these kinds of in-line inputs! Is there a timeframe now that it seems to be built but not exposed?

[jrieken]

I was wondering if that was using inset webviews or some other ui element?

It uses the internal editor API to create a view zone (that our lingo here) to host a diff editor and some clever hiding of lines. Editor insets are similar in that they also built in view zones but the contents are designed to be a web view

[a-stewart]

I was wondering if that was using inset webviews or some other ui element?

It uses the internal editor API to create a view zone (that our lingo here) to host a diff editor and some clever hiding of lines. Editor insets are similar in that they also built in view zones but the contents are designed to be a web view

Thanks :)

And how about the comment input below - where the user is writing a comment and there is the "Accept" button. Is that an inset webview?

[jrieken]

where the user is writing a comment and there is the "Accept" button. Is that an inset webview?

No. It also uses the lower-level view zone API. As of today we have no feature that builds on inset webviews

[MaxKless]

It uses the internal editor API to create a view zone (that our lingo here)

@jrieken are there any plans to expose these view zones for rendering custom text in VSCode? Kind of like the webview inset but for just text. Then we wouldn't have to re-implement all the rendering features that are already included in the editor.

[starball5]

Apparently related on Stack Overflow: VScode extension: how can I add custom UI inside the editor?

[hotwater9]

Excuse, are you saying that in-line input plugins like Copilot Chat are not currently supported? @starball5

[Mayanks0307]

@jrieken Hi any update on this if we have any plan to add this in production or any strict timeline in mind like may be 1-2 month types ?

[jrieken]

This item is currently not planned to be finalized. When it happens it will appear on our monthly iteration plan and this issue will be updated accordingly. I have locked this issue because of repeated pings. Sorry, for doing that but we need to minimise distractions. Thanks for understanding