Distutils-based installer does not detect 64bit versions of Python

[erluk]

BPO	6792
Nosy	@loewis, @pjeby, @jaraco, @tarekziade, @jkloth, @merwok, @bitdancer, @eli-b, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tarekziade'
closed_at = <Date 2021-02-03.18:26:06.868>
created_at = <Date 2009-08-28.08:52:18.338>
labels = ['type-bug', 'library']
title = 'Distutils-based installer does not detect 64bit versions of Python'
updated_at = <Date 2021-02-03.18:26:06.867>
user = 'https://bugs.python.org/erluk'

bugs.python.org fields:

activity = <Date 2021-02-03.18:26:06.867>
actor = 'steve.dower'
assignee = 'tarek'
closed = True
closed_date = <Date 2021-02-03.18:26:06.868>
closer = 'steve.dower'
components = ['Distutils']
creation = <Date 2009-08-28.08:52:18.338>
creator = 'erluk'
dependencies = []
files = []
hgrepos = []
issue_num = 6792
keywords = []
message_count = 23.0
messages = ['92028', '92045', '96921', '96925', '96928', '96930', '96933', '100396', '101094', '112936', '116126', '117899', '125977', '127618', '127627', '127630', '151778', '151827', '151851', '177389', '184357', '184358', '386372']
nosy_count = 25.0
nosy_names = ['loewis', 'pje', 'jaraco', 'ghazel', 'tarek', 'jkloth', 'eric.araujo', 'dontbugme', 'r.david.murray', 'carwyn', 'srid', 'cgohlke', 'erluk', 'Stephen.White', 'santoso.wijaya', 'menekali@gmail.com', 'jerome.radix', 'Eli_B', 'piotr.dobrogost', 'vr.gamer', 'steve.dower', 'Roy.Jacobson', 'epu', 'ChrisBenson', 'editor-buzzfeed']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue6792'
versions = ['Python 2.7']

[erluk]

An installer for source-only modules created using distutils
(bdist_wininst) will only detect 32bit installations of Python on 64bit
Windows machines.

Expected behaviour:
The installer lists 32bit and 64bit installations of Python on the machine.

[loewis]

Unfortunately, I don't think this is possible. When creating the
installer, you have to make a choice whether to make it a 32-bit or a
64-bit MSI file - whether or not you have any architecture dependency.

At least, can't think of an easy way to solve it.

[carwyn]

This prevents numerous packages from installing correctly including the
current 0.6c11 version of setuptools. When the installer runs it reports
that it can't find the version of python installed from
python-2.6.4.amd64.msi.

It seems to be looking for it in:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python\PythonCore\2.6\InstallPath]
@="C:\\Python26\\"

.. rather than the:
[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.6\InstallPath]
@="C:\\Python26\\"

.. that the python installer puts in the registry.

This is related to the conversation in this thread:

http://www.mail-archive.com/distutils-sig@python.org/msg10512.html

And to quote from http://bugs.python.org/setuptools/issue2 :

"The issue is with the .exe header used by the bdist_wininst command;
setuptools doesn't generate this wrapper itself; it uses the one
supplied by the distutils, so non-setuptools packages will have the same
problem. Without a fixed .exe header in the Python installation, *all*
bdist_wininst-generated .exe installers (setuptools or distutils) will
have the same problem."

[loewis]

Carwyn: those packages just need to create two versions of their
installers - one for 32-bit Python, and one for 64-bit Python. Please
report that to the respective packages.

[ssbarnea]

It is possible to create combined x86 and x64 msi files and in fact it
would be a good idea to have only one instead of two.

[loewis]

sorin: can you provide a patch?

[pjeby]

ISTM there may be two ways to fix this problem; one was to change the
.exe header produced by bdist_wininst, but in retrospect, it can't fix
this because it's likely Windows' 64-to-32 bit conversion (Wow6432Node)
that's changing the registry path used, rather than anything that Python
or distutils is doing.

The other way, which has more probability of working, is for the 64-bit
Python installer to include an extra registry entry that would allow the
32 bit versions to find the 64-bit version.

(This might also require an install-time 64/32 compatibility check being
added to the .exe header for bdist_wininst to prevent installing
binary-incompatible extensions.)

[StephenWhite]

32bit apps can query the 64bit registry, using the appropriate security and access rights options such as KEY_WOW64_64KEY (0x0100).

Similarly KEY_WOW64_32KEY can be used for 64bit apps to read/write the 32bit registry without having to have knowledge of how the Wow6432Nodes are arranged .

These mean that a 64bit aware app, whether compiled as 64 or 32 bits, can access the alternative view of the registry.

http://msdn.microsoft.com/en-us/library/ms724897(VS.85).aspx
http://msdn.microsoft.com/en-us/library/ms724878(VS.85).aspx

For example if you have both 64 and 32 bit copies of Python installed then a Python app running under the 32bit copy of Python can query the location of the 64bit copy of Python using code like:
key64 = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, "Software\\Python\\PythonCore\\2.6\\PythonPath", 0, _winreg.KEY_READ + 0x0100)
_winreg.QueryValue(key, "")

C code can do similarly.

[loewis]

One issue to consider is pre/post-install actions. bdist_wininst loads pythonxy.dll from the target system, which would fail if it is a 32-bit installer process that tries to load a 64-bit python DLL.

[santosowijaya]

Does anyone know of any workaround, for now?

[merwok]

Removing 2.6 which only gets security fixes now.

[dontbugme]

you can add InstallPath key with the corresponding value at [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python\PythonCore\2.6\]
if you want disutils installer to detect your python
That makes him detect and install the librarys or scripts to the right directory, but doens't make your library 64bit compatible if it isn't
(means if the library doesn't work on 64 bit i neither will whith this work around) Only possibility of fixing that problem is installing 32bit python

[menekaligmailcom]

There are many updated installers, for many libs for those of us using 64bit windows 7.

http://www.lfd.uci.edu/~gohlke/pythonlibs/

[jeromeradix]

Could you please change the priority of this Issue to 'High' as this problem is a big annoyance for all Windows 64bits users which is a rather large niche of users (a niche getting larger and larger everyday), don't you think ?

[bitdancer]

I'll leave priority setting to tarek, but it doesn't look to me like raising the priority is going to make any difference, since it doesn't sound from reading the ticket like anyone has found a solution yet (other than offering 64bit installers).

[tarekziade]

Yeah I agree. Until we get a solution + patch the priority here does not really matter.

[vrgamer]

I'm not certain that I agree with the argument used to justify keeping this as a 'normal' priority issue. Apparently, since it doesn't effect the entire python community and being as there is no readily available solution, the decision is to treat it as a minor problem. Were one to apply the same logic to, what say, epidemiology, then I suppose the lack of a vaccine for HIV would not be a particularly pressing issue either.

If priority escalation is out of the question, can we at least have an update? After three years of workarounds, I'm beginning to suspect that the idea is to wait until the effected packages become deprecated and then declare 'solved!'.

[briancurtin]

Without a patch or a solution, the priority doesn't really matter (like Tarek said in msg127630). If anyone is actively working on this feel free to say otherwise, but I see no status to update.

[loewis]

If priority escalation is out of the question

It's not out of the question - it's just that setting the priority on
the issue is not a proper way to escalate.

Instead, there are two forms of escalation available:

1. submit a patch that fixes the issue
2. pay somebody to fix the issue if you can't fix it yourself

[RoyJacobson]

This bug is a really annoying one, any chance it will be fixed in 2.7? It's really a matter when you want to deploy a program using distutils (my case), because you cannot really require your clients to edit the registry themselves :/

Is there any problem with just adding the x32 compatibility path entry to the python x64 .msi? It's a very easy fix that shouldn't cause any harm.

[epu]

I would like to investigate this issue, but I need more information regarding the bug and the expected behavior. Is this specifically that an x64 windows python that generates a bdist (msi output) runs and can't find the x64 interpreter because of syswow registry redirection? That is, packaging should be able to find the interpreter bitness that generated the msi in the first place (and no-other bitness)?

There are python sprints this week at PyCon, but I cannot attend them. Clarifying the expected behavior this week will help me write tests and investigate/fix (if it is in my ability).

[loewis]

Erik: the issue is about bdist_wininst, not bdist_msi (bdist_msi has a similar issue, but it is entirely different in its causes and potential resolution, and shall not be discussed here).

The code to find the installations is in PC/bdist_wininst/install.c:GetPythonVersions. The code to run the installscript is in run_installscript.

[zooba]

Distutils is now deprecated (see PEP-632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools