Manage encoding files by extension of filenames

[FrancisFaure]

Hello "VS Code" team,

• I do a language extension for "VS Code",
  (https://github.com/FrancisFaure/vfp_tmlanguage_generator)
  (Visual FoxPro)
• I search a setting for the "codepage" on file extensions files (.PRG) on "Windows 1252" and not "UTF-8" by default
• I try (without succes to add "files.encoding": "windows1252" in "package.json"
• "VS Code" website say :
  users can change all codepage "globally" or codepage on "workspace"
  (https://code.visualstudio.com/docs/editor/codebasics#_file-encoding-support)

But in the same directory we can have files in utf8 and others in "Western" (Windows 1252) codepage

I tried to add "files.encoding": "windows1252" in my language pack (package.json) without success..

I googled without solutions

sorry for my bad english

With best regards
Francis

[bpasero]

Currently you can only configure encoding per workspace folder.

[FrancisFaure]

Thank for the quick answer...
Have you plan to manage encoding files by extension of filenames ?
with by example add property "files.encoding" in file "package.json" of VS Code extension language.
Because same folder could have files encoded 1252 and others utf8
with best regards
Francis

[FrancisFaure]

please,
have you planned an VSCode update for allow add "files.encoding": "windows1252" in extension language package.json file?
or a way for open files by extension of filenames encoded 1252 and not utf8 (we need to open *.PRG files in 1252)
Thanks

[buzzzzer]

Any news for files.encoding in extention API or by fileext ?
On my project 3 different encodings in over 1000 files.
And reopen the file each time - just killing me...
Please schedule this feature..;(

[cheery]

I am also affected by this issue.

Please fix it.

[bpasero]

#1587

[frankcai]

@bpasero That didn't fix the issue. We can now change editor.* settings but not the encoding setting.

[ianmaurinsoucy]

I am also affected by this issue. I have many files in the same folders with different encoding... I can't use VSCode because of this.

[FrancisFaure]

I left my volunteer work on VS Code because of this encoding problem that was ignored. cordially

[Somnium7]

We also need possibility to set different encoding for different file extensions. Maybe it's possible to write an extension for this?

[FrancisFaure]

to "Somnium7",
Hi
I work a lot (volunteer) on VS Code for manage encoding files by extension of filenames (in example like .HTML -> UTF8 and .PRG -> 1512 in the same directory)
But Microsoft ignore this request : it seem he want only manage UTF8 encoding or encoding per directories,
(Or bpassero haven't understood my request with my bad english)
The solution can be only an new line in package.json file or other way. but ignored.
if you find a solution for "set different encoding for different file extensions" : I'll happy
with best regards
Francis

[Fred-Vatin]

I tried to add

"[bat]": {
    "files.encoding": "cp437",
    "files.autoGuessEncoding": false,
    "editor.fontFamily": "Consolas, 'Courier New', monospace"
}

to the settings.json

files.encoding and files.autoGuessEncoding triggers an issue displayed in the tooltip on mouseover: Unknown identifier. Use language identifiers.

It is impossible now to set those settings as per language rule, please fix it.

VS Code 1.11.2 on Win 10 x64
Edit: still an big issue in v 1.15.0 insider

[z639]

What I find funny is that my old>old editor of choice (Notepad++) is able to auto detect the file encoding and yet VS Code with all its bells and whistles that make it a far better editor can't even manage to do the same.

At the very least you would think that we would be able to tell the editor to use X encoding for Y file type.

[jhonrocha]

This problem is still not solved? It is the biggest miss of Code right now ... Please, take it seriously for a moment. I never lost an afternoon replacing bad chars due a miss while saving a file on Sublime ...

[Fred-Vatin]

Guys, you have to vote for the first post (thumb up) so this issue may be taken seriously. With only 2 votes everybody won't care even if it's an annoying issue.

[Raydir]

voted for this issue.

[roffelsaurus]

"files.autoGuessencoding": true worked for me auto detecting win1252 on opening. The changes in @workflo link seem to at least have given a workaround for this issue.

[BlackOnion]

Voted for this issue.
I like VSCode very much, but not be able to set the encoding per file extension is a absolute no go. Even every normal text editor need this feature. I used it in the last millennium.

[bpasero]

Trying to understand the requirements here. So far I have seen various requests to associate a file encoding

• by language (Langauge Specific file encoding setting #19890)
• by extension (this issue)
• by path (Allow to associate a file encoding per file path #23570)

It seems to me that being able to define the encoding by path is most powerful because it would allow to define the encoding for particular files in specific folders. And it would also allow to define for all files in all folders of a specific extension.

The beauty however of allowing to define the encoding by language is that we already have support for defining settings per language. The syntax is:

"[markdown]":  {
  "editor.wordWrap": "on",
  "editor.quickSuggestions": false
}

And so I could envision a setting such as:

"[markdown]":  {
  "files.encoding": "some encoding"
}

There is another related setting where you can define which extension maps to which language (files.association).

The downside of this solution though is that you cannot set the encoding for a subset of files of a language. It would always apply to all files of that language.

[Somnium7]

@bpasero I think that defining by path should allow wildcards to be able to match files by extension only. However, it could be harder to implement than detecting by language, because it needs new configuration syntax. For my personal needs, detection by language suffices. Let's see, what other people think.

[ElCondor1969]

I agree with @Somnium7 . Detection by language is the simplest and, at present, the fastest thing to implement.

[workflo]

I would like to add a fourth alternative to @bpasero's proposal:
It would be great if I could annotate an arbitrary text file and explicitly tell VSC which encoding to use. This was one of (X)Emacs' features I really liked:
// -*- coding: latin1 -*-

Emacs also allows overriding other settings like indentation mode, tab width and the like this way.

[bpasero]

I have pushed a change that allows to configure the encoding per language in settings (see #19890 (comment)). Will be available in tomorrows insider release. I think we should see how it goes with this support before we introduce more powerful settings.

[BlackOnion]

Thank you, Benjamin.
I used the insider for a very small test and it worked for me.

[bpasero]

@FrancisFaure ok to close?

[FrancisFaure]

Hi @bpasero,
I don't had/took time to test the solution of this issue (I work on vscode on my free time and my english is poor) I think I can work on that: this week-end.
(If in the same directory we can have some files with differents encoding : ie .PRG in cp1252, .XML in utf8, ... and if I can set encoding for .PRG (in any directories) that could be great)
with best regards
Francis

[bpasero]

@FrancisFaure yes, the current solution allows to change the encoding for language types. This is possible globally or per workspace folder.

[FrancisFaure]

Hi @bpasero
when you write "language types" = that mean by extension of filenames (ie : .PRG, .BAT, .CMD, .XML...) ?
is it a setting in package.json of language extension ? other ? (if other have you a url link for read and test please)
Regards

[bpasero]

@FrancisFaure this is an indirection: multiple extensions can be associated with one language. I suggest you give it a try and report back how it works for you.

[FrancisFaure]

Hi @bpasero
ok I will work on the subject this weekend,
Thank you
I'll let you informed after finding the links on VSCODE to define a "language type" to link the extension .PRG + tests
If you want, you can close the subject and if necessary reopen it if it does not work?
(Sorry for my bad English)
regards

[jhonrocha]

Hello @bpasero
I just tried the insiders version. They're working perfectly on both global and workspace settings!
Thank you, best regards

// ---- My settings.json ----
{
    "[asp]":{
      "files.encoding": "iso88591"
    },
    "[javascript]":{
      "files.encoding": "utf8"
    },
    "[html]":{
      "files.encoding": "utf8"
    }
}

[FrancisFaure]

Hi @bpasero,
I tested with VS Code Insider :

a) I added in my settings (stored in C:\Users\USER\AppData\Roaming\Code - Insiders\User\settings.json):

    "[vfp]":{
        "files.encoding": "windows1252"
    }

that work well.

b) I removed my settings of §a
and added in my package.json language :

	"contributes": {
		"configurationDefaults": {
		  "[vfp]": {
			"files.encoding": "windows1252"
		  }
		}...

and that work ! great ! it's what I needed :)
with best regards
Francis

[Raydir]

works like a bomb

[Somnium7]

@bpasero
Doesn't work search in multiple files without Ripgrep (haven't tested with it, don't need it).
I have files which need "maccenteuro" encoding, let assume it is Javascript.
With following configuration search works in all files except ".js".
"search.useRipgrep": false
With following configuration works in none files except ".js".
"search.useRipgrep": false,
"files.encoding": "maccenteuro"
With following configuration search should work for all files (seems logic), however, it works in all files except ".js" (like first case):
"search.useRipgrep": false,
"[javascript]": {
"files.encoding": "maccenteuro"
}
Note that when opening actual files encoding is working properly, only search in multiple files doesn't work. Also note that when testing, you should use symbols with diacritics, which have different representations in different encodings.

Tested in VSCode version 1.17.0 on Windows 10

[buzzzzer]

@bpasero
Search in multiple files doesn't work with setting by language
Win10 1.17.0 release

[bpasero]

@roblourens is aware of that limitation afaik.

[Somnium7]

@roblourens Are you (or another person) going to fix this? Maybe a new issue should be created?

[roblourens]

This is the library that would have to support those encodings: https://github.com/hsivonen/encoding_rs

It's probably unlikely, as it's not a standard encoding.