Skip to content

Allow extensions to provide a source mapper for comments on notebooks. #246317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

rehmsen
Copy link
Contributor

@rehmsen rehmsen commented Apr 11, 2025

VSCode can render review comments in the NotebookEditor, but only when they use vscode-notebook-cell: URLs. These URLs require knowing the internal cell ID, which is not exposed to extensions.
Another challenge is that comments and notebook serializers are often provided by different extensions (e.g. comments by the Github Pull Request extension, the NotebookSerializer by the Jupyter extension).
So in practice, it is difficult to provide such comments.

This change proposes a different approach, where NotebookSerializers can optionally provide a NotebookMapper, which can translate file ranges to notebook ranges. If such a mapper is provided, VS Code will, upon adding a notebook document or a document comment, convert the document comment into the corresponding notebook comment, so that it is rendered in the NotebookEditor.

As we cannot change the type of CommentThread.range on the fly, a new NotebookRange2 type is created which inherits from Range and additionally contains the cell index and cell part (source, metadata, maybe later outputs). When a thread is associated with a specific cell via the vscode-notebook-cell: URI, it is sufficient that all downstream components consider only the positions shared with Range. The cell index and cell part are only used to determine the correct cell.

Currently, the NotebookEditor renders notebook comments at cell granularity only and does not care about the exact Range where the comment was left. However, this is a natural extension that would already be supported by the proposed NotebookMapper API.

Issue #214017

VSCode can render review comments in the NotebookEditor, but only when they use vscode-notebook-cell: URLs. These URLs require knowing the internal cell ID, which is not exposed to extensions.
Another challenge is that comments and notebook serializers are often provided by different extensions (e.g. comments by the Github Pull Request extension, the NotebookSerializer by the Jupyter extension).
So in practice, it is difficult to provide such comments.

This change proposes a different approach, where NotebookSerializers can optionally provide a `NotebookMapper`, which can translate file ranges to notebook ranges. If such a mapper is provided, VS Code will, upon adding a notebook document or a document comment, convert the document comment into the corresponding notebook comment, so that it is rendered in the NotebookEditor.

As we cannot change the type of `CommentThread.range` on the fly, a new `NotebookRange2` type is created which inherits from `Range` and additionally contains the cell index and cell part (source, metadata, maybe later outputs). When a thread is associated with a specific cell via the vscode-notebook-cell: URI, it is sufficient that all downstream components consider only the positions shard with `Range`. The cell index and cell part are only used to determine the correct cell.

Currently, the NotebookEditor renders notebook comments at cell granularity only and does not care about the exact Range where the comment was left. However, this is a natural extension that would already be supported by the proposed NotebookMapper API.
@rehmsen
Copy link
Contributor Author

rehmsen commented Apr 11, 2025

@rebornix @alexr00 Here is a proposal for how we could address #214017.
I have tested this end to end with our internal notebook and comments extension. If you think this is promising, I can send a PR how this might be used by the Jupyter extension to support e.g. Github PR comments.

One aspect I am still unsure about is if it would not be better to keep the mapped range independent from the original range. Ideally, I would like the same comment to show both in the the Text and the NotebookEditor, as two different views of the same model. This would require more changes to the NotebookEditor, so it can read and respect the mapped range of comments attached to a file: URL. If that makes sense to you, let me know and I can try that out, too.

Looking forward to your feedback.

Copy link
Member

@bpasero bpasero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Red CI

@rehmsen
Copy link
Contributor Author

rehmsen commented Apr 14, 2025

Thanks for checking. I have adjusted the unit tests to pass undefined for the mapper for now. I am happy to actually add unit tests for the new code if we are aligned on the general design.

I verfied that

npm exec -- npm-run-all -lp core-ci-pr extensions-ci-pr hygiene eslint valid-layers-check property-init-order-check vscode-dts-compile-check tsec-compile-check

Now passes locally.

I also tried

./scripts/test.sh --tfs "Unit Tests"

But that fails for me with:

[3503:0414/094357.620702:INFO:CONSOLE(220)] "TypeError: Failed to fetch dynamically imported module: file:///Users/oler/Code/vscode/out/vs/base/common/errors.js", source: file:///Users/oler/Code/vscode/test/unit/electron/renderer.js (220)

I am trying to figure out why. Maybe that is some Mac issue?

In the meanwhile, is there any way I can trigger CI on Github or does this happen automatically after a while?

@rehmsen
Copy link
Contributor Author

rehmsen commented Apr 14, 2025

Looks like it is run automatically, and the tests still fail. I figured out how to run the tests locally - the main issue was that you first need to run the UI locally for it to create the build/ folder - it seems the test script itself does not actually do the build - it relies on the files already being there.

Now that I figured that out, I will work on fixing all the tests and send a new push when done. Sorry for the delays. In the meantime, any opinions on the design are also very welcome.

@rehmsen rehmsen requested a review from bpasero April 14, 2025 16:13
@bpasero bpasero requested review from rebornix and removed request for bpasero April 14, 2025 16:18
@bpasero
Copy link
Member

bpasero commented Apr 14, 2025

I will yield to @rebornix review, I only chimed in as I saw the build be red.

@rehmsen
Copy link
Contributor Author

rehmsen commented Apr 30, 2025

Gentle ping @alexr00 @rebornix - any thoughts on this?
@Yoyokrazy, you seem to also be working on notebooks - any chance you could take a look if @rebornix is currently unavailable?

@christophermadsen
Copy link

This would be phenomenal to have! 😮 Hopefully you'll be able to merge this soon. Godspeed team 🙏🏻

@alexr00
Copy link
Member

alexr00 commented May 27, 2025

@rehmsen thank you for the PR. We will try to get to this, but our focus isn't on Notebooks + Comments at the moment. Thank you for being so patient!

@rehmsen
Copy link
Contributor Author

rehmsen commented May 28, 2025

Understood. We are not blocked on this as we have patched our version already, but it would of course be nice to get this into upstream so it is also useful to others.
Btw. we are not only using the mapper for comments but also for the inverse mapping to convert cursor positions in the notebook editor to the corresponding positions in file, e.g. for passing that to AI models along side the JSON file content, not only the current cell like upstream seems to do. Happy to extend on this PR if and when we have alignment on the simple case.

@rebornix
Copy link
Member

Like @alexr00 mentioned above, supporting comments in notebooks is a stretch for now. In the meantime, I’ve been exploring other areas where we might leverage source mapping. I noticed that the Search View includes experimental support for searching text in closed notebook documents via search.experimental.closedNotebookRichContentResults.

For example, when a user clicks a match in the Search View (say, the 10th match), the notebook document is opened and we try to locate the same match in the editor. But this isn’t very accurate, we’re just matching the same text and position heuristically. I wonder if we could use a source map instead to accurately navigate to the correct match location.

I will dig into this a bit more and see if I can build a prototype on top of this concept for notebook search. Will keep you all posted. @rehmsen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants