bdist_wininst depends on MBCS codec, unavailable on non-Windows

[merwok]

BPO	10945
Nosy	@loewis, @amauryfa, @vstinner, @tarekziade, @merwok, @jelmer, @hroncok, @miss-islington
PRs	bpo-10945: Drop support for bdist_wininst on non-Windows systems #14506 [3.7] bpo-10945: Drop support for bdist_wininst on non-Windows systems (GH-14506) #14509 [3.8] bpo-10945: Drop support for bdist_wininst on non-Windows systems (GH-14506) #14510

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/merwok'
closed_at = <Date 2021-01-11.12:50:37.691>
created_at = <Date 2011-01-19.15:28:02.210>
labels = ['type-bug', 'library']
title = 'bdist_wininst depends on MBCS codec, unavailable on non-Windows'
updated_at = <Date 2021-01-11.12:50:37.691>
user = 'https://github.com/merwok'

bugs.python.org fields:

activity = <Date 2021-01-11.12:50:37.691>
actor = 'vstinner'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2021-01-11.12:50:37.691>
closer = 'vstinner'
components = ['Distutils', 'Distutils2']
creation = <Date 2011-01-19.15:28:02.210>
creator = 'eric.araujo'
dependencies = []
files = []
hgrepos = []
issue_num = 10945
keywords = ['patch']
message_count = 27.0
messages = ['126528', '126531', '126533', '126536', '126539', '135892', '135894', '139968', '139974', '145213', '145509', '145540', '145548', '145627', '145630', '145680', '155591', '155603', '346998', '346999', '347001', '347009', '347011', '347013', '347015', '347132', '384814']
nosy_count = 12.0
nosy_names = ['loewis', 'amaury.forgeotdarc', 'vstinner', 'techtonik', 'tarek', 'eric.araujo', 'Arfrever', 'jelmer', 'runtux', 'hroncok', 'Jelmer Vernooij', 'miss-islington']
pr_nums = ['14506', '14509', '14510']
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10945'
versions = ['3rd party', 'Python 2.7', 'Python 3.2', 'Python 3.3']

[merwok]

If distutils.commands.bdist_wininst.bdist_wininst.get_inidata returns a unicode string (which is always the case in 3.x and can happen in 2.x if if you pass a unicode object as one setup argument), bdist_wininst will try to use the MBCS codec, which is not available on non-Windows OSes (see http://docs.python.org/dev/library/codecs#module-encodings.mbcs).

If someone wants to make a patch, here are some guidelines: http://wiki.python.org/moin/Distutils/FixingBugs

[techtonik]

I believe it is better to start and commit a testcase .

[merwok]

We don’t commit failing tests :) If you mean you want to write a test case, please go ahead and attach a patch or changeset URI.

[techtonik]

Please don't ask me about patches. Core devs won't accept them.

[merwok]

Please don't ask me about patches. Core devs won't accept them.

One of your patches for diff has been accepted; some distutils bugs have
patches from you (for which we are grateful) that are in various review
stages. The only thing preventing acceptance where I’m concerned is
time, nothing else.

[runtux]

I've just been bitten by this when trying to do a new release of roundup, why not make the mbcs codec available on non-windows platforms as has been proposed (and rejected) in bpo-1251921 -- any non-technical reasons for not including this codec on other platforms?

[amauryfa]

The mbcs codec depends on the Windows installation. On most Western Windows it is similar to cp1252, Japanese Windows will use cp932, and so on.
If we were to provide mbcs on non-windows platform, it should be an alias to ascii.

[vstinner]

Can't you only work with Unicode and avoid the MBCS encoding?

[runtux]

On Thu, Jul 07, 2011 at 10:52:51AM +0000, STINNER Victor wrote:

Can't you only work with Unicode and avoid the MBCS encoding?

I'm trying to build a windows binary package on Linux. This usually
works fine -- as long as the package description doesn't contain
unicode. If it *contains* unicode it will somehow internally use MBCS
encoding.
So if someone fixes windows binary package building on non-windows
platforms to not use MBCS encoding this is fine with me. But maybe the
easier way out here is to include that codec on all platforms.

(and don't tell me I have to install windows for building a windows
binary package :-)

Thanks, Ralf

Dr. Ralf Schlatterbeck Tel: +43/2243/26465-16
Open Source Consulting www: http://www.runtux.com
Reichergasse 131, A-3411 Weidling email: office@runtux.com
osAlliance member email: rsc@osalliance.com

[merwok]

haypo:

Can't you only work with Unicode and avoid the MBCS encoding?
It is not code under the users’ control (i.e. setup.py) that uses MBCS, but the bdist_wininst command itself. It used to be runnable from linux, which is great to provide binary installers for Windows users who typically don’t have a compiler. This is what the bug is about.

[vstinner]

It is not code under the users’ control (i.e. setup.py)
that uses MBCS, but the bdist_wininst command itself.

bdist_command append configuration data to a wininst-xxx.exe binary. Where does this file come from? Can we modify wininst-xxx.exe binaries?

If we can modify the binaries, we can change the format to store the configuration data as UTF-8 instead of the ANSI code page.

It's surprising and "unsafe" (not portable) to use the ANSI code page for an installer: if you build your installer on a french setup (ANSI=cp1252), the configuration was be interpreted incorrectly on a japanese setup (ANSI=cp932). Example:

>>> 'Hé ho'.encode('cp1252').decode('cp932')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'cp932' codec can't decode bytes in position 1-2: illegal multibyte sequence

So if the configuration data (package metadata) contains non-ASCII characters, you will not be able to use your installer on a computer using a different ANSI code page than the code page of the computer used to build the installer... In the best case, you will just get mojibake.

If we cannot modify wininst-xx.exe, an alternative to be able to generate installers on non-Windows platforms is to use the most common ANSI code page (cp1252?), or maybe ASCII.

Use the ASCII encoding is the safest solution because you will be able to use your installer on all Windows setup (all ANSI code pages are compatible with ASCII), but you will not be able to generate an installer if the package metadata contains at least one non-ASCII character...

[merwok]

> It is not code under the users’ control (i.e. setup.py)
> that uses MBCS, but the bdist_wininst command itself.
bdist_command append configuration data to a wininst-xxx.exe binary.
Are you sure? The string that’s encoded with mbcs comes from the get_inidata method in bdist_wininst.py; get_inidata only works with strings (either string literals or str objects (in 3.x) given as attributes of the command object or in the DistributionMetadata object).

Where does this file come from? Can we modify wininst-xxx.exe binaries?
If needed, we can: the code lives in PC/bdist_wininst.

Use the ASCII encoding is the safest solution
I don’t think so. Distutils supports author='Éric', so bdist_wininst should too :)