Vs Code fails to get the encoding of the file

[awebdeveloper]

Look at the following issue pre-commit/pre-commit#519

VS Code has issue recognising the encoding of pre-commit file

• VSCode Version: 1.12
• OS Version: OSX 10.10

Steps to Reproduce:

1. Configure pre-commit tool for your repo
2. Try committing the a file from gui. Notice the following error

An unexpected error has occurred: LookupError: unknown encoding:
Check the log at ~/.pre-commit/pre-commit.log

[asottile]

From what I understand here, VS code is not passing the LANG environment variable (or passing an invalid LANG environment variable) causing python to misinterpret the default encoding. Usually python will fall back to US-ASCII when not receiving LANG or receiving LANG=, I'm unable to reproduce the python issue outside of VSCODE

A minimal reproduction would be a pre-commit hook which does the following:

#!/usr/bin/env python
import io
# io.open triggers an encoding lookup
with io.open('/dev/null') as f:
    f.read()

Simply write that to .git/hooks/pre-commit, chmod 755 .git/hooks/pre-commit and commit from VS code

[joaomoreno]

This is an issue of pre-commit

[asottile]

@joaomoreno I encourage you to reconsider. My reproduction above does not involve the framework, just vanilla git, vanilla python, and vanilla vscode. Commit with that set up works fine for me outside of vscode but crashes with the LookupError listed above. (the small python script also works just fine outside).

I think you may be confused about my poor choice in naming (heh), pre-commit (the framework) is named after the built in git hook by the same name.

[joaomoreno]

What would a fix in VS Code look like?

[asottile]

I'm not entirely sure without analyzing the problem more. My guess is VS Code needs to pass along the LANG environment variable it was started with to subprocesses.

[asottile]

OK I decided to look into this, here's my conclusions:

I changed the pre-commit hook to print the environment it receives:

import os

for k, v in sorted(os.environ.items()):
    print('{!r}: {!r}'.format(k, v))

import io

print('ohai from pre-commit')
io.open('/dev/null').read()

git commit --quiet --allow-empty-message --file -
'AMD_ENTRYPOINT': 'vs/workbench/node/extensionHostProcess'
'Apple_PubSub_Socket_Render': '/private/tmp/com.apple.launchd.dX0FU7QFRb/Render'
'ELECTRON_NO_ASAR': '1'
'ELECTRON_RUN_AS_NODE': '1'
'GIT_ASKPASS': '/Users/asottile/Desktop/Visual Studio Code.app/Contents/Resources/app/extensions/git/out/askpass.sh'
'GIT_AUTHOR_DATE': '@1494448046 -0700'
'GIT_AUTHOR_EMAIL': 'asottile@yelp.com'
'GIT_AUTHOR_NAME': 'Anthony Sottile'
'GIT_DIR': '.git'
'GIT_EDITOR': ':'
'GIT_INDEX_FILE': '.git/index'
'GIT_PREFIX': ''
'HOME': '/Users/asottile'
'LANG': 'en_US.UTF-8'
'LC_ALL': 'en_US'
'LC_NUMERIC': 'C'
'LOGNAME': 'asottile'
'PATH': '/Applications/Xcode.app/Contents/Developer/usr/libexec/git-core:/Library/Frameworks/Python.framework/Versions/3.5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin:/usr/local/munki'
'PIPE_LOGGING': 'true'
'PWD': '/'
'SHELL': '/bin/bash'
'SHLVL': '1'
'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.5R484ME7FY/Listeners'
'TMPDIR': '/var/folders/cq/_m2zxzyd5qz25qt5c8hhlll9kx_hbz/T/'
'USER': 'asottile'
'VERBOSE_LOGGING': 'true'
'VERSIONER_PYTHON_PREFER_32_BIT': 'no'
'VERSIONER_PYTHON_VERSION': '2.7'
'VSCODE_GIT_ASKPASS_MAIN': '/Users/asottile/Desktop/Visual Studio Code.app/Contents/Resources/app/extensions/git/out/askpass-main.js'
'VSCODE_GIT_ASKPASS_NODE': '/Users/asottile/Desktop/Visual Studio Code.app/Contents/Frameworks/Code Helper.app/Contents/MacOS/Code Helper'
'VSCODE_GIT_ASKPASS_PORT': '49798'
'VSCODE_GIT_COMMAND': 'commit'
'VSCODE_IPC_HOOK': '/Users/asottile/Library/Application Support/Code/1.12.1-main.sock'
'VSCODE_IPC_HOOK_EXTHOST': '/var/folders/cq/_m2zxzyd5qz25qt5c8hhlll9kx_hbz/T/vscode-9a9875ae-a665-46ff-8c5a-aa99d8614e9f.sock'
'VSCODE_NLS_CONFIG': '{"locale":"en-us","availableLanguages":{}}'
'VSCODE_NODE_CACHED_DATA_DIR_1710': '/Users/asottile/Library/Application Support/Code/CachedData/f6868fce3eeb16663840eb82123369dec6077a9b'
'VSCODE_PID': '1710'
'VSCODE_WINDOW_ID': '1'
'XPC_FLAGS': '0x0'
'XPC_SERVICE_NAME': '0'
'_': '/Users/asottile/Desktop/Visual Studio Code.app/Contents/MacOS/Electron'
'__CF_USER_TEXT_ENCODING': '0x65D5417F:0x0:0x0'

The problem is the LC_ALL variable:

$ LC_ALL=en_US python -c 'import io; io.open("/dev/null")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
LookupError: unknown encoding: 

The LC_ALL environment variable should look like LANG, notably en_US.UTF-8.

I've submitted a PR here: #26412

[joaomoreno]

Thanks for taking the time to look into this. 🍻

[awebdeveloper]

@joaomoreno hopefully this will land in next version of vscode