Allow to set `files.encoding` as language specific setting for files on startup #127936

VisionXi · 2021-07-03T21:30:12Z

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.57.1, but also previous versions
OS Version: Windows 10 and 7, all versions until 2021, July

Well know bug:

Language specific resource settings are not resolved for files opening right on startup · Issue #39084 and others.

Please reopen this issue and set priority on it. It's really worse and there are many people running into this issue!

Perhaps you find a way to handle this by a configurable seeting or command line switch.

yvvt0379 · 2021-07-04T12:13:09Z

@vscode-triage-bot I sincerely suggest you to look into this bothering issue that a large number of users strongly require to be solved. Many thanks.

VisionXi · 2021-07-04T13:39:56Z

Sorry, I don't know how to get priority on this issue. It seems that the bot has closed this issue. But it's really annoying and such an important use case.
If it's really because the extension host has to scan all extensions and for performance reasons we do not want to wait on that, then why not trying to solve this by a configurable setting. I could live with a small delay when opening a file with vscode. This delay would only occur if vscode was not opened before.

bpasero · 2021-07-06T17:14:19Z

Renamed to reflect the issue: my understanding is that this only applies to settings which we need right in the beginning when opening a file, specifically: the encoding and the line endings. All other settings should apply after the extensions have been resolved.

I briefly looked into this and still don't have a good solution, other than maybe caching all the languages mapped to file types to have them ready right in the beginning.

We resolve the language of the file here in order to resolve the right settings:

vscode/src/vs/editor/common/services/textResourceConfigurationServiceImpl.ts

Line 114 in 58e840c

return this.modeService.getModeIdByFilepathOrFirstLine(resource);

This code will eventually hit guessMimeTypes:

vscode/src/vs/base/common/mime.ts

Line 114 in b776d80

    
           export function guessMimeTypes(resource: URI | null, firstLine?: string): string[] {

But in order for this to work, we need registerTextMime to be called with all languages, including extensions:

vscode/src/vs/base/common/mime.ts

Line 44 in b776d80

    
           export function registerTextMime(association: ITextMimeAssociation, warnOnOverwrite = false): void {

Which is not the case for files opened right on startup. So we resolve those settings that are not language based.

//cc @sandy081 @alexdima for more ideas

bpasero · 2021-07-07T07:41:20Z

Maybe we could have new API on the IConfigurationService that I can ask whether any language specific configuration exists for files.encoding or files.eol and only if so, prolong the opening of a file on startup until languages have been registered.

However, I just checked our default settings and it looks like we actually define files.eol for shell scripts:

"[shellscript]":  {
   "files.eol": "\n"
}

So we need to be a bit more clever here, otherwise we would always wait for languages being registered from extensions...

sandy081 · 2021-07-07T12:20:45Z

inspect API provides you the override identifiers

vscode/src/vs/platform/configuration/common/configuration.ts

Line 115 in bdd4291

    
           inspect<T>(key: string, overrides?: IConfigurationOverrides): IConfigurationValue<Readonly<T>>;

vscode/src/vs/platform/configuration/common/configuration.ts

Lines 68 to 88 in bdd4291

    
           export interface IConfigurationValue<T> { 
        
           	readonly defaultValue?: T; 
        
           	readonly userValue?: T; 
        
           	readonly userLocalValue?: T; 
        
           	readonly userRemoteValue?: T; 
        
           	readonly workspaceValue?: T; 
        
           	readonly workspaceFolderValue?: T; 
        
           	readonly memoryValue?: T; 
        
           	readonly value?: T; 
        
           	readonly default?: { value?: T, override?: T }; 
        
           	readonly user?: { value?: T, override?: T }; 
        
           	readonly userLocal?: { value?: T, override?: T }; 
        
           	readonly userRemote?: { value?: T, override?: T }; 
        
           	readonly workspace?: { value?: T, override?: T }; 
        
           	readonly workspaceFolder?: { value?: T, override?: T }; 
        
           	readonly memory?: { value?: T, override?: T }; 
        
           	readonly overrideIdentifiers?: string[]; 
        
           }

gjsjohnmurray · 2021-07-28T11:33:07Z

Maybe allow file-extension-specific overrides in addition to language-specific ones.

VisionXi · 2022-03-11T07:16:03Z

Is there any progress in solving this issue? I understand that it's not easy. But it's really annoying if you want to open a file direclty from a file manager and not within VC.

bpasero · 2022-03-31T12:27:01Z

Not super happy with the solution I pushed now and it might backfire, but here is what is happening now:

we now detect changes to the editor language and there is almost always a transition from plaintext in the beginning until languages are resolved to a target language
if we detect that the new language has a configured files.encoding override
and that encoding is different from the current encoding
and the editor is not dirty or in save conflict resolution
we reopen the file with the configured encoding

Unfortunately I cannot tell apart this from happening vs. the user changing the language from the editor status bar. So if the user changes language mode to bat from to ps1 and ps1 has a configured encoding, the encoding will change.

🤷

bpasero · 2022-04-19T15:14:20Z

Verified in #39084 (comment)

Raydir · 2022-04-25T04:51:25Z

sorry, that i did not write soon enough in #39084 soon enough (work/holydays absorbed me a bit...), but for me this does not work the way it "should" to be honestly...

since i opened #39084 we partly changed some of our scripts to UTF8bom.

So while migrating our customers to the latest version of our software (what could take some time...) we are in some kind of mixedmode.

My Extension does set the files.encoding param to OEM850 (due to old files). if there is explicity an file that does deliver a UTF8 with a BOM this should not be overridden by the files.encoding param.

UTF8bom does explicit deliver the BOM so that e.g. Editors can open the file correctly.

-> "files.autoGuessEncoding": true does not work becauses this is setting the OEM850 files to OEM866 (...because that is the first implemented encoding in the OEM sector i gues...)

Thanks for a second insight on this matter.

bpasero · 2022-04-25T04:57:30Z

@Raydir can you open a new issue with some basic steps how to reproduce, I am not sure I understand the issue or if its related to this issue.

sbatten assigned bpasero Jul 6, 2021

bpasero added feature-request Request for new features or functionality file-encoding File encoding type issues labels Jul 6, 2021

bpasero added this to the Backlog milestone Jul 6, 2021

bpasero removed their assignment Jul 6, 2021

bpasero changed the title ~~Please reopen: Language specific resource settings are not resolved for files opening right on startup~~ Allow to set files.encoding and files.eol as language specific setting for files on startup Jul 6, 2021

gjsjohnmurray mentioned this issue Jul 28, 2021

Always open files with extension '*.fvl' (which is xml) with encoding ISO 8859-15 #129670

Closed

bpasero mentioned this issue Mar 30, 2022

Language specific resource settings are not resolved for files opening right on startup #39084

Closed

bpasero changed the title ~~Allow to set files.encoding and files.eol as language specific setting for files on startup~~ Allow to set files.encoding as language specific setting for files on startup Mar 31, 2022

bpasero self-assigned this Mar 31, 2022

bpasero modified the milestones: Backlog, April 2022 Mar 31, 2022

bpasero closed this as completed in 2171820 Mar 31, 2022

bpasero added the author-verification-requested Issues potentially verifiable by issue author label Apr 8, 2022

bpasero added the verified Verification succeeded label Apr 19, 2022

Raydir mentioned this issue Apr 25, 2022

UTF8bom files do not open as UTF8 after fix in #39084 #148025

Closed

joyceerhl added the verification-needed Verification of issue is requested label Apr 25, 2022

github-actions bot locked and limited conversation to collaborators May 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow to set `files.encoding` as language specific setting for files on startup #127936

Allow to set `files.encoding` as language specific setting for files on startup #127936

VisionXi commented Jul 3, 2021 •

edited

yvvt0379 commented Jul 4, 2021

VisionXi commented Jul 4, 2021

bpasero commented Jul 6, 2021 •

edited

bpasero commented Jul 7, 2021

sandy081 commented Jul 7, 2021

gjsjohnmurray commented Jul 28, 2021

VisionXi commented Mar 11, 2022

bpasero commented Mar 31, 2022 •

edited

bpasero commented Apr 19, 2022

Raydir commented Apr 25, 2022 •

edited

bpasero commented Apr 25, 2022

Allow to set files.encoding as language specific setting for files on startup #127936

Allow to set files.encoding as language specific setting for files on startup #127936

Comments

VisionXi commented Jul 3, 2021 • edited

yvvt0379 commented Jul 4, 2021

VisionXi commented Jul 4, 2021

bpasero commented Jul 6, 2021 • edited

bpasero commented Jul 7, 2021

sandy081 commented Jul 7, 2021

gjsjohnmurray commented Jul 28, 2021

VisionXi commented Mar 11, 2022

bpasero commented Mar 31, 2022 • edited

bpasero commented Apr 19, 2022

Raydir commented Apr 25, 2022 • edited

bpasero commented Apr 25, 2022

Allow to set `files.encoding` as language specific setting for files on startup #127936

Allow to set `files.encoding` as language specific setting for files on startup #127936

VisionXi commented Jul 3, 2021 •

edited

bpasero commented Jul 6, 2021 •

edited

bpasero commented Mar 31, 2022 •

edited

Raydir commented Apr 25, 2022 •

edited