Inconsistent results from platform.machine() on Windows ARM64

[cwalther]

Bug report

Running x86 and AMD64 builds of Python 3.11.0 (official binary releases) in emulation on Windows 11 on an ARM64 machine gives me

Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'AMD64'

Python 3.11.0 (main, Oct 24 2022, 18:13:38) [MSC v.1933 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'ARM64'

This seem inconsistent. I would expect both to return 'ARM64', the actual underlying processor architecture, in the same way as running an x86 build on AMD64 returns 'AMD64'. But even if they were reporting the emulated architecture, it should be 'AMD64' and 'x86', not 'AMD64' and 'ARM64' (or possibly 'AMD64' and 'AMD64', if x86 binaries are also handled by the AMD64 emulator, I have no idea how that works).

This makes it hard to distinguish between an AMD64 build running on ARM64 and an AMD64 build running on AMD64. In fact, the only platform function that gives any indication at all is processor(), and parsing the desired information out of the free-form text returned by that seems fiddly. ('ARMv8 (64-bit) Family 8 Model 0 Revision 0, ' and 'Intel64 Family 6 Model 44 Stepping 2, GenuineIntel' here.)

Relatedly, how do I distinguish between an ARM64 build running on ARM64 and an AMD64 build running on ARM64? The inconsistency above makes them differ in platform.machine(), but what if that were fixed? This seems like a job for platform.architecture(), which is explicitly meant to give information about the running binary, but that just returns ('64bit', 'WindowsPE') in both cases. platform.python_compiler() happens to have that information ('MSC v.1933 64 bit (ARM64)' vs. 'MSC v.1933 64 bit (AMD64)'), but again that is fiddly free-form text and may not work for other compilers than MSC.

Your environment

• CPython versions tested on: 3.11.0
• Operating system and architecture: Windows 11 10.0.22000 ARM64, Windows 10 10.0.19043 AMD64

[eryksun]

Out of curiosity, can you install the x86 and AMD64 versions of 3.12.0a1 and check platform.machine() in each? The new implementation in 3.12 prefers to get the CPU "Architecture" from WMI "Win32_Processor" instead of from the system environment variables "PROCESSOR_ARCHITEW6432" and "PROCESSOR_ARCHITECTURE".

[cwalther]

Success! Thanks for the tip. In 3.12.0a1, all binaries return platform.machine() == 'ARM64' when run on ARM64 and platform.machine() == 'AMD64' when run on AMD64, which is what I would expect.

I notice however that, in a similar issue on macOS, #96993, @rossng seems to have a different expectation.

The problem remains that ARM64-on-ARM64 and AMD64-on-ARM64 are only distinguishable by platform.python_compiler(). Should I open a separate issue for that?

Detailed output

AMD64-on-ARM64

Python 3.12.0a1 (main, Oct 25 2022, 00:24:35) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
...     try:
...             print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
...     except:
...             pass
...
_Processor             <platform._Processor object at 0x0000021960314F50>
_get_machine_win32     'ARM64'
_mac_ver_xml           None
_node                  'M1Win11'
_sys_version           ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:24:35', 'MSC v.1933 64 bit (AMD64)')
_syscmd_ver            ('Microsoft', 'Windows', '10.0.22000')
architecture           ('64bit', 'WindowsPE')
java_ver               ('', '', ('', '', ''), ('', '', ''))
libc_ver               ('', '')
mac_ver                ('', ('', '', ''), '')
machine                'ARM64'
node                   'M1Win11'
platform               'Windows-11-10.0.22000-SP0'
processor              'ARMv8 (64-bit) Family 8 Model 0 Revision   0,  '
python_branch          ''
python_build           ('main', 'Oct 25 2022 00:24:35')
python_compiler        'MSC v.1933 64 bit (AMD64)'
python_implementation  'CPython'
python_revision        ''
python_version         '3.12.0a1'
python_version_tuple   ('3', '12', '0a1')
release                '11'
system                 'Windows'
uname                  uname_result(system='Windows', node='M1Win11', release='11', version='10.0.22000', machine='ARM64')
version                '10.0.22000'
win32_edition          'Professional'
win32_is_iot           False
win32_ver              ('11', '10.0.22000', 'SP0', 'Multiprocessor Free')
>>> ^Z

x86-on-ARM64

Python 3.12.0a1 (main, Oct 25 2022, 00:13:39) [MSC v.1933 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
...     try:
...             print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
...     except:
...             pass
...
_Processor             <platform._Processor object at 0x08BD6C78>
_get_machine_win32     'ARM64'
_mac_ver_xml           None
_node                  'M1Win11'
_sys_version           ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:13:39', 'MSC v.1933 32 bit (Intel)')
_syscmd_ver            ('Microsoft', 'Windows', '10.0.22000')
architecture           ('32bit', 'WindowsPE')
java_ver               ('', '', ('', '', ''), ('', '', ''))
libc_ver               ('', '')
mac_ver                ('', ('', '', ''), '')
machine                'ARM64'
node                   'M1Win11'
platform               'Windows-11-10.0.22000-SP0'
processor              'ARMv8 (64-bit) Family 8 Model 0 Revision   0,  '
python_branch          ''
python_build           ('main', 'Oct 25 2022 00:13:39')
python_compiler        'MSC v.1933 32 bit (Intel)'
python_implementation  'CPython'
python_revision        ''
python_version         '3.12.0a1'
python_version_tuple   ('3', '12', '0a1')
release                '11'
system                 'Windows'
uname                  uname_result(system='Windows', node='M1Win11', release='11', version='10.0.22000', machine='ARM64')
version                '10.0.22000'
win32_edition          'Enterprise'
win32_is_iot           False
win32_ver              ('11', '10.0.22000', 'SP0', 'Multiprocessor Free')
>>> ^Z

ARM64-on-ARM64

Python 3.12.0a1 (main, Oct 25 2022, 00:13:29) [MSC v.1933 64 bit (ARM64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
...     try:
...             print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
...     except:
...             pass
...
_Processor             <platform._Processor object at 0x000001AAC8D78FB0>
_get_machine_win32     'ARM64'
_mac_ver_xml           None
_node                  'M1Win11'
_sys_version           ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:13:29', 'MSC v.1933 64 bit (ARM64)')
_syscmd_ver            ('Microsoft', 'Windows', '10.0.22000')
architecture           ('64bit', 'WindowsPE')
java_ver               ('', '', ('', '', ''), ('', '', ''))
libc_ver               ('', '')
mac_ver                ('', ('', '', ''), '')
machine                'ARM64'
node                   'M1Win11'
platform               'Windows-11-10.0.22000-SP0'
processor              'ARMv8 (64-bit) Family 8 Model 0 Revision   0,  '
python_branch          ''
python_build           ('main', 'Oct 25 2022 00:13:29')
python_compiler        'MSC v.1933 64 bit (ARM64)'
python_implementation  'CPython'
python_revision        ''
python_version         '3.12.0a1'
python_version_tuple   ('3', '12', '0a1')
release                '11'
system                 'Windows'
uname                  uname_result(system='Windows', node='M1Win11', release='11', version='10.0.22000', machine='ARM64')
version                '10.0.22000'
win32_edition          'Professional'
win32_is_iot           False
win32_ver              ('11', '10.0.22000', 'SP0', 'Multiprocessor Free')
>>> ^Z

AMD64-on-AMD64

Python 3.12.0a1 (main, Oct 25 2022, 00:24:35) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
...     try:
...             print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
...     except:
...             pass
...
_Processor             <platform._Processor object at 0x000002AF1B5FA630>
_get_machine_win32     'AMD64'
_mac_ver_xml           None
_node                  'DESKTOP-ILRVE2N'
_sys_version           ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:24:35', 'MSC v.1933 64 bit (AMD64)')
_syscmd_ver            ('Microsoft', 'Windows', '10.0.19043')
architecture           ('64bit', 'WindowsPE')
java_ver               ('', '', ('', '', ''), ('', '', ''))
libc_ver               ('', '')
mac_ver                ('', ('', '', ''), '')
machine                'AMD64'
node                   'DESKTOP-ILRVE2N'
platform               'Windows-10-10.0.19043-SP0'
processor              'Intel64 Family 6 Model 44 Stepping 2, GenuineIntel'
python_branch          ''
python_build           ('main', 'Oct 25 2022 00:24:35')
python_compiler        'MSC v.1933 64 bit (AMD64)'
python_implementation  'CPython'
python_revision        ''
python_version         '3.12.0a1'
python_version_tuple   ('3', '12', '0a1')
release                '10'
system                 'Windows'
uname                  uname_result(system='Windows', node='DESKTOP-ILRVE2N', release='10', version='10.0.19043', machine='AMD64')
version                '10.0.19043'
win32_edition          'Professional'
win32_is_iot           False
win32_ver              ('10', '10.0.19043', 'SP0', 'Multiprocessor Free')
>>> ^Z

x86-on-AMD64

Python 3.12.0a1 (main, Oct 25 2022, 00:13:39) [MSC v.1933 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
...     try:
...             print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
...     except:
...             pass
...
_Processor             <platform._Processor object at 0x0163CE70>
_get_machine_win32     'AMD64'
_mac_ver_xml           None
_node                  'DESKTOP-ILRVE2N'
_sys_version           ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:13:39', 'MSC v.1933 32 bit (Intel)')
_syscmd_ver            ('Microsoft', 'Windows', '10.0.19043')
architecture           ('32bit', 'WindowsPE')
java_ver               ('', '', ('', '', ''), ('', '', ''))
libc_ver               ('', '')
mac_ver                ('', ('', '', ''), '')
machine                'AMD64'
node                   'DESKTOP-ILRVE2N'
platform               'Windows-10-10.0.19043-SP0'
processor              'Intel64 Family 6 Model 44 Stepping 2, GenuineIntel'
python_branch          ''
python_build           ('main', 'Oct 25 2022 00:13:39')
python_compiler        'MSC v.1933 32 bit (Intel)'
python_implementation  'CPython'
python_revision        ''
python_version         '3.12.0a1'
python_version_tuple   ('3', '12', '0a1')
release                '10'
system                 'Windows'
uname                  uname_result(system='Windows', node='DESKTOP-ILRVE2N', release='10', version='10.0.19043', machine='AMD64')
version                '10.0.19043'
win32_edition          'Enterprise'
win32_is_iot           False
win32_ver              ('10', '10.0.19043', 'SP0', 'Multiprocessor Free')
>>> ^Z

[eryksun]

The problem remains that ARM64-on-ARM64 and AMD64-on-ARM64

Have you checked os.environ['PROCESSOR_ARCHITECTURE']? That should be the architecture of the current process.

Since you don't control the environment that was used to create the current process, it would be more reliable to call WinAPI GetSystemInfo() via PyWin32 or ctypes. This should return wProcessorArchitecture as PROCESSOR_ARCHITECTURE_INTEL (0) for x86, PROCESSOR_ARCHITECTURE_AMD64 (9) for AMD64 (x64), and PROCESSOR_ARCHITECTURE_ARM64 (12) for ARM64.

[cwalther]

Confirmed, these methods both work on all combinations, on both 3.11.0 and 3.12.0a1. Thanks! That will do as a workaround, although of course it would be nicer to have it abstracted in the platform module, ideally in a way that works identically on other OSes (although at the moment Windows is all I care about).

(It seems weird that two processes started from the same shell would see different environments, but then I’m used to weirdness from Microsoft.)

Detailed output

Script

import os
print('PROCESSOR_ARCHITECTURE:', os.environ['PROCESSOR_ARCHITECTURE'])
import ctypes
import ctypes.wintypes
class SYSTEM_INFO(ctypes.Structure):
	_fields_ = [('wProcessorArchitecture', ctypes.wintypes.WORD), ('dwPageSize', ctypes.wintypes.DWORD), ('lpMinimumApplicationAddress', ctypes.wintypes.LPVOID), ('lpMaximumApplicationAddress', ctypes.wintypes.LPVOID), ('dwActiveProcessorMask', ctypes.c_void_p), ('dwNumberOfProcessors', ctypes.wintypes.DWORD), ('dwProcessorType', ctypes.wintypes.DWORD), ('dwAllocationGranularity', ctypes.wintypes.DWORD), ('wProcessorLevel', ctypes.wintypes.WORD), ('wProcessorRevision', ctypes.wintypes.WORD)]
info = SYSTEM_INFO()
ctypes.windll.kernel32.GetSystemInfo(ctypes.byref(info))
print('GetSystemInfo:      ', {f[0]: getattr(info, f[0]) for f in SYSTEM_INFO._fields_})
ctypes.windll.kernel32.GetNativeSystemInfo(ctypes.byref(info))
print('GetNativeSystemInfo:', {f[0]: getattr(info, f[0]) for f in SYSTEM_INFO._fields_})

AMD64-on-ARM64

PROCESSOR_ARCHITECTURE: AMD64
GetSystemInfo:       {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}

x86-on-ARM64

PROCESSOR_ARCHITECTURE: x86
GetSystemInfo:       {'wProcessorArchitecture': 0, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 2147418111, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 586, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 2147418111, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 586, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}

ARM64-on-ARM64

PROCESSOR_ARCHITECTURE: ARM64
GetSystemInfo:       {'wProcessorArchitecture': 12, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 0, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 0, 'wProcessorRevision': 0}
GetNativeSystemInfo: {'wProcessorArchitecture': 12, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 0, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 0, 'wProcessorRevision': 0}

AMD64-on-AMD64

PROCESSOR_ARCHITECTURE: AMD64
GetSystemInfo:       {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}

x86-on-AMD64

PROCESSOR_ARCHITECTURE: x86
GetSystemInfo:       {'wProcessorArchitecture': 0, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 2147418111, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 586, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 4294901759, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}

[eryksun]

I just checked the source for QEMU user space emulation, which is used on Linux and BSD to emulate another CPU architecture in user space, relying on native system calls. When translating the uname() system call for an emulated process, it replaces the native machine with the emulated machine. Thus it seems that platform.machine() for an x64 Python that's running under QEMU emulation on ARM64 would report "x86_64" instead of "aarch64" (ARM64). In contrast, on Windows we return the native OS architecture. Should this be made consistent across platforms? If so, which choice is correct? If the choice to return the emulated architecture is correct, we could switch to calling GetNativeSystemInfo() on Windows, which will return the emulated architecture, instead of using a WMI query that returns the true machine architecture.

---

Off-topic comments on using ctypes:

• I would include the wReserved field in SYSTEM_INFO instead of implicitly relying on alignment padding to set the offset of dwPageSize.
• It's better to use kernel32 = ctypes.WinDLL('kernel32', use_last_error=True) instead of ctypes.windll.kernel32. The ctypes.windll loader caches WinDLL instances, which cache function prototypes. This can cause conflicts between packages. Also, a separate WinDLL instance can set use_last_error=True to enable the use of reliable ctypes.get_last_error() instead of less reliable kernel32.GetLastError().
• ctypes opens scripts up to problems with data loss and crashes due to memory and stack corruption. To help avoid this, in general it's better to define function prototypes such as kernel32.GetSystemInfo.argtypes = (ctypes.POINTER(SYSTEM_INFO),). ctypes will type check arguments if argtypes is set, which helps to ensure that low-level C code is never passed bad arguments.

[cwalther]

I’m not sure what is correct. Maybe before deciding about that one should discuss what is useful. I have use cases for wanting to know

• code of what architecture(s) the machine is capable of running, so that when I’m installing software, I can install the version compiled for that architecture. This is what to me platform.machine() is documented to do, and does on Windows in 3.12.0a1: return something that depends only on the machine (including installed OS) and not on the running interpreter. However the documentation is not very clear, and apparently opinions differ on that (see platform.processor() and platform.machine() report inconsistent values under Rosetta on M1 macOS #96993). An ideal function for this purpose would return a list of architectures in order of preference, maybe even marked by whether supported natively or in emulation.
• the architecture of the running interpreter, so that in a setuptools setup.py I know what architecture we are building for (which wouldn’t necessarily need to be the same, but as far as I know there is no support for cross-compilation, so it is in practice, and I have not found any way of querying the latter directly) and can supply the corresponding libraries. This seems what platform.architecture() was intended to do, but does not on Windows (because it bins all architectures into the two categories “32bit” and “64bit”) nor on macOS (because it looks at the binary, not the process, and may choose wrong when the binary is fat).

So far I do not have a use case for wanting to know the emulated architecture, but it may exist.

I would consider consistency across platforms very desirable (and, at the moment, lacking). What would also be useful in that regard is if one could agree on a known set of possible values for an “architecture”, in particular choose one of “x86_64” and “AMD64” and one of “aarch64” and “ARM64” and stick to it.

(Thanks for the ctypes tips! Omitting wReserved was an oversight and I didn’t notice that my code was just working by accident due to alignment. I wasn’t aware of the second point, and left out the third in a quick and minimal script.)