platform() is not able to detect windows 11

[sahsariga111]

BPO	45382
Nosy	@malemburg, @pfmoore, @tjguk, @ambv, @zware, @eryksun, @zooba, @miss-islington, @Evernow
PRs	bpo-45382: test.pythoninfo logs more Windows versions #30817 bpo-45382: test.pythoninfo: set wmic.exe encoding to OEM #30890 [3.10] bpo-45382: test.pythoninfo logs more Windows versions #30891 [3.9] [3.10] bpo-45382: test.pythoninfo logs more Windows versions (GH-30891) #30894
Files	demo.py import subprocess.py main.py main.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-10-05.19:43:57.181>
labels = ['library', '3.10']
title = 'platform() is not able to detect windows 11'
updated_at = <Date 2022-03-28.07:10:50.170>
user = 'https://bugs.python.org/sahsariga111'

bugs.python.org fields:

activity = <Date 2022-03-28.07:10:50.170>
actor = 'tim.golden'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-10-05.19:43:57.181>
creator = 'sahsariga111'
dependencies = []
files = ['50327', '50329', '50330', '50331']
hgrepos = []
issue_num = 45382
keywords = ['patch']
message_count = 38.0
messages = ['403260', '403261', '403262', '403263', '403264', '403265', '403267', '403271', '403272', '403357', '403358', '403379', '403386', '403452', '403459', '403471', '404451', '404591', '411331', '411336', '411474', '411498', '411659', '411661', '411663', '411671', '411694', '411696', '411704', '411711', '411712', '411714', '411762', '411763', '411764', '415333', '416142', '416146']
nosy_count = 10.0
nosy_names = ['lemburg', 'paul.moore', 'tim.golden', 'lukasz.langa', 'zach.ware', 'eryksun', 'steve.dower', 'miss-islington', 'sahsariga111', 'Evernow']
pr_nums = ['30817', '30890', '30891', '30894']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue45382'
versions = ['Python 3.10']

[sahsariga111]

I am updated to windows 11 . Now I am trying to write script that will detect is user use windows 11 or windows 10 .
I was using the simplest way as possible:
import platform
print(platform.platform())
The result I got is : Windows-10-10.0.22000-SP0
That is quite correct . The build version is correct ,but the windows version is still not .

[sahsariga111]

The
platform.win32_ver() returns same answer

[sahsariga111]

The bug comes from Microsoft terminal bug : If I type there : ver it will return Microsoft Windows [Version 10.0.22000.194] only one patch is if that will check the build .
so :
info = subprocess.check_output(cmd,
stdin=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
text=True,
shell=True)
if(int(info.strip(".")[2])==22000):
return "Windows [Version 11.0.22000.194]

[sahsariga111]

demo.py contains dirty hack that can be used as a fix for some time before microsoft will not fix it.

[malemburg]

win32_ver() should be using the internal Windows APIs to figure out the version. I do wonder why those don't return the same version as the "ver" command line tool.

Adding our Windows experts to the noisy list.

[zooba]

The version number for "Windows 11" still starts with 10.0. Just like how Windows 5.x and 6.x were around for a very long time each ;)

There are tables in platform module that map the specific version to the release name. These probably need to be updated to return "11" for versions 10.0.22000 and greater.

[malemburg]

On 05.10.2021 22:30, Steve Dower wrote:

The version number for "Windows 11" still starts with 10.0. Just like how Windows 5.x and 6.x were around for a very long time each ;)

There are tables in platform module that map the specific version to the release name. These probably need to be updated to return "11" for versions 10.0.22000 and greater.

Hmm, but the "ver" output seems to have more information than those
APIs.

Note: The tables for mapping to releases for Windows only take the
major.minor versions as key. Unfortunately, those did not change. It's
actually the build version which provides the indicator, it seems.

Any idea, whether a patch will fix this on Windows soonish ?

[eryksun]

The _WIN32_CLIENT_RELEASES table based on major.minor version number isn't helpful since Windows 10 and 11 have the same version number. win32_ver() needs a workaround to return release "11" if the build number is 22000 or greater. Is there any need/desire to also identify Server 2016, 2019, and 2022?

[zooba]

Hmm, but the "ver" output seems to have more information than those APIs.

It's always had build numbers, which the regular APIs do not, because the regular APIs are meant for detecting incompatibilities rather than reporting.

Since there are some incompatibilities, I hope they'll rev the minor version number, but I have no idea if that's planned yet. In theory I have early access to the release build already, but I haven't installed it on anything.

Eventually I think we're going to need some kind of WMI call in the platform module to get the right data for reporting. Until then, we're making best guesses from heuristics.

[sahsariga111]

systeminfo can be option

[malemburg]

It's probably time to extend the marketing version detection mechanism to use
the build number as reference instead of the major.minor system version numbers.

Here's a good reference for this:

https://en.wikipedia.org/wiki/List_of_Microsoft_Windows_versions

MS resources:

https://docs.microsoft.com/en-us/windows/win32/sysinfo/operating-system-version
https://docs.microsoft.com/en-us/windows/release-health/release-information

[sahsariga111]

That nice idea . So the dist can contain the minimal build required to say that is for example windows 11 . The simplest solution that came in mind is . Is far from perfect but it works.

[sahsariga111]

Beter solution . Using match in python 3.10

[eryksun]

use the build number as reference instead of the major.minor

It could check the (major, minor, build) tuple, which allows reporting 10.1+ as "post11" and minimizes hard coding of build numbers. For example, given win32_ver() iterates by (major, minor, build) thresholds:

    _WIN32_CLIENT_RELEASES = [
        ((10, 1, 0), "post11"),
        ((10, 0, 22000), "11"),
        ((6, 4, 0), "10"),
        ((6, 3, 0), "8.1"),
        ((6, 2, 0), "8"),
        ((6, 1, 0), "7"),
        ((6, 0, 0), "Vista"),
        ((5, 2, 3790), "XP64"),
        ((5, 2, 0), "XPMedia"),
        ((5, 1, 0), "XP"),
        ((5, 0, 0), "2000"),
    ]

    _WIN32_SERVER_RELEASES = [
        ((10, 1, 0), "post2022Server"),
        ((10, 0, 20348), "2022Server"),
        ((10, 0, 17763), "2019Server"),
        ((6, 4, 0), "2016Server"),
        ((6, 3, 0), "2012ServerR2"),
        ((6, 2, 0), "2012Server"),
        ((6, 1, 0), "2008ServerR2"),
        ((6, 0, 0), "2008Server"),
        ((5, 2, 0), "2003Server"),
        ((5, 0, 0), "2000Server"),
    ]

In win32_ver():

    if major >= 5: # NT systems
        if getattr(winver, 'product_type', None) == 3: # VER_NT_SERVER
            release_list = _WIN32_SERVER_RELEASES
        else:
            release_list = _WIN32_CLIENT_RELEASES
        ver = major, minor, build
        for release_ver, release_name in release_list:
            if ver >= release_ver:
                release = release_name
                break

[malemburg]

On 08.10.2021 02:15, Eryk Sun wrote:

> use the build number as reference instead of the major.minor

It could check the (major, minor, build) tuple, which allows reporting 10.1+ as "post11" and minimizes hard coding of build numbers. For example, given win32_ver() iterates by (major, minor, build) thresholds:

Great idea.

Could you prepare a PR for this ?

[vstinner]

macOS has a similar issue in the platform module. Previously, platform gave the darwin kernel version, whereas most users only know macOS versions. See bpo-35344.

Even if Microsoft decided to internally stay at 10.x, IMO users really expect "Windows 11" when requesting the Windows version, especially in platform.platform().

[zooba]

All that said, if we're going to shell out to "ver", we may as well try "wmic os get Caption,Version /value" first and parse its output. For me, it looks like the below (with a number of blank lines around it):

Caption=Microsoft Windows 10 Enterprise
Version=10.0.19043

I would assume the output is in MBCS, and there doesn't appear to be an option to change that - the "/locale" option appears to influence translations, rather than encoding.

We'd still want the "ver" fallback I think. The wmic command is deprecated in favour of PowerShell commands, which means we would have to implement native calls to get equivalent functionality. But I expect neither cmd.exe nor wmic.exe will actually disappear for a long time.

[sahsariga111]

On my Win11 it returns me :

C:\Users\Win10Home>wmic os get Caption,Version /value

Caption=Microsoft Windows 11 Home Single Language
Version=10.0.22000

[vstinner]

I wrote PR 30817 to log "ver" and "wmic os get Caption,Version /value" command output in test.pythoninfo. I would like to check if Windows Server 2022 is deployed on CI used by Python :-) I'm trying to understand why bpo-41682 "[Windows] test_asyncio: Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure" started to fail more often recently.

[vstinner]

New changeset b0898f4 by Victor Stinner in branch 'main':
bpo-45382: test.pythoninfo logs more Windows versions (GH-30817)
b0898f4

[zooba]

Yeah, I pushed the machines up to 2022 the other day while tidying the configs. Wouldn't have expected it to impact that issue though, it's fully within OpenSSL, isn't it? (We also updated that recently...)

Also, as an FYI, if you use "wmic /output:<filename>" then it always writes in utf-16-le with a BOM. It's *very* unlikely you'll ever get output that doesn't fit into MBCS, but if that does ever occur, we can use a temporary file instead of stdout.

[eryksun]

It's *very* unlikely you'll ever get output that doesn't fit into MBCS,

When writing to a pipe, wmic.exe hard codes using the process OEM code page (i.e. CP_OEMCP). If it matters, running wmic.exe with subprocess should use encoding='oem' instead of text=True.

That said, wmic.exe is deprecated. I suggest using PowerShell instead. For example:

    import os
    import json
    import subprocess

    cmd = 'Get-CimInstance Win32_OperatingSystem | Select Caption, Version | ConvertTo-Json'
    p = subprocess.run(f'powershell.exe -c "{cmd}"', capture_output=True, encoding=os.device_encoding(1))
    result = json.loads(p.stdout)

PowerShell uses the console's output code page (i.e. os.device_encoding(1)) when writing to stdout, even if it's a pipe. (If PowerShell is run without a console via DETACHED_PROCESS, then it outputs nothing to stdout.) The only way I know of to make PowerShell write UTF-8 to stdout when it's a pipe is by temporarily changing the console output code page. Assuming the current process has a console, you have to first get the current code page with GetConsoleOutputCP(). Change the code page to UTF-8 via SetConsoleOutputCP(CP_UTF8). Run the PowerShell command. Finally, restore the original code page.

Maybe subprocess should provide a context manager to set the console code pages before a call, and restore the previous console code pages and console modes after a call completes. That's what CLI shells such as CMD do when running an external program.

[vstinner]

When writing to a pipe, wmic.exe hard codes using the process OEM code page (i.e. CP_OEMCP). If it matters, running wmic.exe with subprocess should use encoding='oem' instead of text=True.

Oh ok, I wrote PR 30890 to fix the wmic.exe encoding.

I would prefer to avoid a named temporary file if possible, until it's really needed.

I'm only aware of the Fedora operating system which made fun of non-ASCII users once with "Fedora release 19 (Schrödinger's Cat)": https://lwn.net/Articles/545741/

[vstinner]

New changeset cef0a54 by Victor Stinner in branch 'main':
bpo-45382: test.pythoninfo: set wmic.exe encoding to OEM (GH-30890)
cef0a54

[sahsariga111]

Thanks guys. Microsoft is always pain for us.
Bit wmic seems nice solution.
Is still working for windows lower than 11?

[vstinner]

New changeset 4a57fa2 by Victor Stinner in branch '3.10':
[3.10] bpo-45382: test.pythoninfo logs more Windows versions (GH-30891)
4a57fa2

[zooba]

So I just created a _wmi module that can do basic queries without shelling out to another process. It's still fairly complicated, as it requires COM (which requires running on a separate thread to avoid interfering with user code), but should be very usable.

I'm not entirely clear on what the best mapping to platform module functions is though, as it would almost certainly result in values changing quite significantly (for the better, but still changing). So I'd prefer to leave that to others to think through.

My main concern with adding the module is the security implications - being able to query all of WMI is very useful, but because we don't offer any way to modify things it isn't quite as useful for legitimate purposes. All you can really do is scan the whole system. Having that available on every Python install by default is a bit scary, so I'm considering limiting the API to only run the "SELECT * FROM Win32_OperatingSystem" query.

(There's also likely a significant performance impact if we always do a WMI query, so I'd rather it remain very optional. It has to load up a ton of stuff to do its job.)

[pfmoore]

Having WMI queries available by default would be potentially pretty useful. What are the security implications you are thinking of? Given that it’s possible to shell out to wmic, I’m not clear how this would be notably worse…

[zooba]

Given that it’s possible to shell out to wmic, I’m not clear how this would be notably worse…

It's pretty straightforward to block or remove wmic.exe if you don't want to allow it on your machine. But that's at the level of paranoia where you should also have your own modified CPython that uses audit hooks to block WMI queries.

Did you have anything specific in mind that it might be useful for?

[pfmoore]

Did you have anything specific in mind that it might be useful for?

Not really, no. More just that I don't see any benefit in having the capability in core Python and then hiding it. I was imagining using it for things that would otherwise need a pywin32 dependency - listing free space on mounted drives, for example, or getting running process information (both of which I have done with pywin32 in the past).

[zooba]

Those both seem legitimate, and looking through this doc I see more that aren't terrible. Provided only select queries are supported, I don't really see the harm in letting it be more open. (And it's worth checking for select in the query in case there's some craftable query that can modify the system...)

I'm still not planning to make it a non-private API, or to make it return anything richer than Name=Value\0Name=Value\0 etc., but it's also very unlikely to change so I guess if someone sneakily uses it 🤷‍♂️

[pfmoore]

lol, fair enough. I'll let the rest of the world persuade you that the UI needs improving because it's a bit user-unfriendly 🙂

[zooba]

It's very user-unfriendly, but it's only an internal API for using in the stdlib 😉 Only has to be better than parsing the output of cmd /c ver

The rest of the world can copy my code and write their own module.

[zooba]

PR #96289 is now updated with platform module changes. I kind of threw a lot of changes at it without thoroughly making sure they'll be consistent with what the old values where, so if anyone wants to try it out on their more obscure setups please do.

Thanks @eryksun for the version->release tables from earlier - I used those.

I did just have the thought that I should probably update sys.getwindowsversion() to use _wmi (when it's available), which would mean I could revert a lot of the platform changes. But that's for another night

[zooba]

The fix for this (not including getwindowsversion()) has been merged for 3.12, but should probably be considered for 3.11 and 3.10 as well.

I don't think it's important enough to hold up 3.11.0, so we'd be looking at 3.11.1. And we just missed 3.10.7, so no rush for 3.10.8. But I'll leave this open until the backports are done.

[vstinner]

Because of the side effects of the calling the _wmi module, I would prefer to only target Python 3.12. I would prefer to get more feedback to see if everything goes well. It's too late to add new feateures to Python 3.11.

The workaround on older Python versions is to run the ver command, as done by test.pythoninfo: see collect_windows().

[zooba]

Fair enough, consider this closed.

[vstinner]

Thanks for fixing this issue, it's good to have an easy way to retrieve the real Windows version ;-)

Maybe _wmi can be used for other purposed later. Maybe we can expose it somehow later.

[v-python]

Why shell out or use _wmi rather than the registry HKLM\software\Microsoft\windows nt\CurrentVersion ProductName

[zooba]

Because the registry key is unsupported and primarily for compatibility reasons.

Also, for example, the value of ProductName on my Windows 11 machine is "Windows 10 Enterprise", which is exactly the bug we were trying to fix here.

The platform module is intended to get true information, not compatibility shims.