Skip to content

[vscode] Support encoding/decoding from workspace and while opening text documents #15873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 25, 2025

Conversation

rschnekenbu
Copy link
Contributor

What it does

Support the workspace openTextDocument, encode and decode methods with additional encoding options.

fix #15559

How to test

The following extension can be used:

This extension provides:

  • provide an action in the explorer and palette to open a text editor with a chosen encoding
  • provide a contextual menu action to reopen document with encoding in the current active text editor
  • provide several actions to test encoding and decoding actions. These actions basically make a call to the API and displays a webview based on the result. Not much to see here, except that the encoding service is properly invoked on encode or decode methods.

Here is also a sample workspace with a set of files with different encodings. This can be used as a test for the Encoding Sample: open with encoding action or reopen editor with encoding actions
workspace-encoding-sample.zip

15559.sample.encoding.mp4

Follow-ups

None known

Breaking changes

  • This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Contributed on behalf of STMicroelectronics

Review checklist

Reminder for reviewers

@github-project-automation github-project-automation bot moved this to Waiting on reviewers in PR Backlog Jun 23, 2025
@rschnekenbu rschnekenbu requested a review from tsmaeder June 24, 2025 07:01
Copy link
Member

@tsmaeder tsmaeder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks reasonable +/- a couple of questions.

} else {
encoding = options?.encoding ?? UTF8;
}
// see https://github.com/microsoft/vscode/blob/118f9ecd71a8f101b71ae19e3bf44802aa173209/src/vs/workbench/services/textfile/browser/textFileService.ts#L806
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this file contain enough copied code to warrant adding a copyright by MS notice?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a copyright + comment that indicates where some parts of the code comes from.

return { encoding, hasBOM };
}

async $validateDetectedEncoding(uri: UriComponents | undefined, detectedEncoding: string | undefined, options: { encoding: string } | undefined): Promise<string> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I read validate..., I expect a method that returns an error if the parameters are not valid. Why not $getValidEncoding?

@@ -74,6 +76,10 @@ export class DocumentDataExt {
ok(!this.disposed);
this.languageId = langId;
}
acceptEncoding(encoding: string): void {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is never called. Do we need it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is now called when encoding is changed. Thanks for catching this piece of missing code!

const cached = this.editorsAndDocuments.getDocument(uri.toString());
if (cached) {
if (cached && !options?.encoding) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does passing an encoding invalidate the cache? Maybe we read the content with the same encoding before?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked for the current encoding of the cached one. This may ease the reuse of cached version.

@@ -80,6 +84,7 @@ export class WorkspaceExtImpl implements WorkspaceExt {
initialize(): void {
this.proxy = this.rpc.getProxy(Ext.WORKSPACE_MAIN);
this.logger = new PluginLogger(this.rpc, 'workspace');
this.encodingService = new EncodingService();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not injectable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some bindings for the EncodingService, and used the injection rather than creating a new one.

tsmaeder
tsmaeder previously approved these changes Jun 25, 2025
@github-project-automation github-project-automation bot moved this from Waiting on reviewers to Needs merge in PR Backlog Jun 25, 2025
@tsmaeder tsmaeder dismissed their stale review June 25, 2025 07:36

Giving you time to address the questions.

@tsmaeder
Copy link
Member

@rschnekenbu This seems to work O.K. for me, but could you try to get the tests to pass, please?

@rschnekenbu
Copy link
Contributor Author

@tsmaeder , I updated the PR according for your comments, thank for the review!
I do not get which tests I am asked to fix. The current failures do not seem to be caused by my contribution.

@tsmaeder
Copy link
Member

@rschnekenbu could you just quickly rebase on master and push? I'd like to get some data on whether the electron update improved the failing tests.

…ext documents.

fix #15559

Contributed on behalf of STMicroelectronics

Signed-off-by: Remi Schnekenburger <rschnekenburger@eclipsesource.com>
@rschnekenbu
Copy link
Contributor Author

Rebased on top of master

@rschnekenbu rschnekenbu merged commit 12d0bab into master Jun 25, 2025
9 of 14 checks passed
@github-project-automation github-project-automation bot moved this from Needs merge to Done in PR Backlog Jun 25, 2025
@github-actions github-actions bot added this to the 1.63.0 milestone Jun 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

[vscode] TextEncoding support and workspace.encode/decode functions
2 participants