Generated scripts can be written in a bogus encoding

[habnabit]

setuptools/setuptools/command/easy_install.py

Lines 836 to 837 in 118edbb

	with open(target, "w" + mode) as f:
	f.write(contents)

open's encoding argument, when unspecified, uses locale.getpreferredencoding , which is not necessarily UTF-8. However, the generated script doesn't include an encoding cookie and can contain non-ASCII data if, say, the shebang path contains non-ASCII components. Either always open the file with encoding='utf-8' or write out an encoding cookie.

This has resulted in an issue filed on a different project: pypa/pipenv#1265

[jaraco]

My intention for issues like this one is to replace Setuptools own script generation with that found in distlib.

Plus, easy_install is generally deprecated.

[habnabit]

Sure. Even if easy_install is deprecated, it seems like you'd want to either delete the functionality surrounding this or fix it.

[jaraco]

The issue reported upstream implicates a script installed by pip and distlib and not setuptools. The issue should be filed with distlib and will only be addressed by setuptools if setuptools adopts distlib instead of its own executable wrapper implementation.

[habnabit]

Regardless of the upstream issue, do you see how this affects setuptools? I can reproduce this, but not exactly right now, as I don't have a windows box readily available.

[jaraco]

I do see how setuptools could be affected, but it's not clear to me that this is a use case we wish to support. I don't fully understand which factors are necessary to bring about these invalid encodings. In fact, if the system's preferred encoding is not UTF-8, perhaps that indicates that the system wishes not to have UTF-8 in the scripts.

I don't feel strongly about it, but I do feel a little uneasy fixing a bug that's never been encountered in a piece of code that's deprecated.

I've gone ahead and pushed a possible change to a branch. Would you review and confirm this is what you had in mind?

[jaraco]

Following up in the parent ticket, I see that in fact easy_install is used regularly for installing source distributions, even when pip is doing the builds unless wheel is used. So this issue is very relevant to common workflows.

[uranusjr]

Being curious here, why does pip use setuptools when installing from source? Would it be plausible to also raise this issue in pip?

[jaraco]

raise this issue in pip?

Looks like it's already done.

[pganssle]

Is this related to #1400?

[habnabit]

@pganssle AFAICT #1400 is a duplicate of this issue yes.