Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
More Windows encoding woes #241
I'm running into problems with the latest (post #201) encoding handling. A "tl;dr" is at the bottom.
Using invoke to run msbuild (the Visual Studio version of
The problem is the default value of
Note how the stdout encoding changes when redirecting. Since invoke does indeed redirect its childs I/O, it's correct to use
Unfortunately, msbuild, or more generally, .NET's
In other words, it recovers the console encoding (
(Chcp changes the active codepage but doesn't alter
This means there is no way to correctly handle both Python apps (which output with
tl;dr: Python and .NET output with different encodings when run from invoke. Invoke can't force an encoding on them and it can't know which one a called program will use. We're hosed.
Can you not set the correct encoding using the
Fundamentally, the problem is that there is no way of determining the encoding of any given stream of bytes without out-of-band information, which simply isn't available.
Regarding the proposed solutions:
My preference is to simply document the encoding that is used, and advise the user to use the
Yes, using the
Re (1), it's only for the output to
referenced this issue
May 3, 2015
I've only skimmed this so far, but wanted to note that I've been doing a massive reorg of the run() internals which should be merging soon; as part of that, I seem to remember adding a config-level option for
Re: encoding/decoding applying to the stored data as well as the printed, it feels cleaner to preserve the raw bytes as seen from the subprocess so that they can be introspected and de/en coded as necessary for e.g. debugging purposes. Whether we should extend this to store both copies (raw and decoded) for convenience, I am unsure of.