_contextvars missing in xmaster branch Windows build?

[tim-one]

BPO	34691
Nosy	@tim-one, @pfmoore, @tjguk, @zware, @stevenwinfield, @1st1, @zooba, @willingc, @miss-islington, @HeleneMary, @ossdev07
PRs	bpo-34691: Compile _contextvars module into main Python library #11741 bpo-34691: Compile _contextvars module into main Python library #11741 bpo-34691: Compile _contextvars module into main Python library #11741 [3.7] bpo-34691: Compile _contextvars module into main Python library (GH-11741) #11743 [3.7] bpo-34691: Compile _contextvars module into main Python library (GH-11741) #11743 [3.7] bpo-34691: Compile _contextvars module into main Python library (GH-11741) #11743
Files	Captura de Tela (12).png

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2019-02-03.00:06:06.176>
created_at = <Date 2018-09-15.03:03:00.779>
labels = ['3.7', '3.8', 'type-bug', 'expert-installation']
title = '_contextvars missing in xmaster branch Windows build?'
updated_at = <Date 2019-02-03.00:06:06.176>
user = 'https://github.com/tim-one'

bugs.python.org fields:

activity = <Date 2019-02-03.00:06:06.176>
actor = 'steve.dower'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2019-02-03.00:06:06.176>
closer = 'steve.dower'
components = ['Installation']
creation = <Date 2018-09-15.03:03:00.779>
creator = 'tim.peters'
dependencies = []
files = ['47832']
hgrepos = []
issue_num = 34691
keywords = ['patch', 'patch', 'patch']
message_count = 11.0
messages = ['325419', '325555', '325556', '325560', '325602', '326513', '333822', '333823', '334763', '334772', '334775']
nosy_count = 11.0
nosy_names = ['tim.peters', 'paul.moore', 'tim.golden', 'zach.ware', 'steven.winfield', 'yselivanov', 'steve.dower', 'willingc', 'miss-islington', 'helena.marina', 'ossdev07']
pr_nums = ['11741', '11741', '11741', '11743', '11743', '11743']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34691'
versions = ['Python 3.7', 'Python 3.8']

[tim-one]

Using Visual Studio 2017 to build the current master branch of Python (something I'm trying for the first time in about two years - maybe I'm missing something obvious!), with the x64 target, under both the Release and Debug builds I get a Python that can't even start to run the test suite. Here's output from the Release build - the Debug build output is nearly identical. The problem does not appear with the Win32 target. Of course I'm doing this on a 64-bit box (under Win 10 Pro):

C:\Code\Python\PCbuild>rt -q -x64

C:\Code\Python\PCbuild>"C:\Code\Python\PCbuild\amd64\python.exe"  -u -Wd -E -bb -m test
Traceback (most recent call last):
  File "C:\Code\Python\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Code\Python\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Code\Python\lib\test\__main__.py", line 1, in <module>
    from test.libregrtest import main
  File "C:\Code\Python\lib\test\libregrtest\__init__.py", line 4, in <module>
    from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES, ALL_RESOURCES
  File "C:\Code\Python\lib\test\libregrtest\cmdline.py", line 4, in <module>
    from test import support
  File "C:\Code\Python\lib\test\support\__init__.py", line 6, in <module>
    import asyncio.events
  File "C:\Code\Python\lib\asyncio\__init__.py", line 8, in <module>
    from .base_events import *
  File "C:\Code\Python\lib\asyncio\base_events.py", line 39, in <module>
    from . import events
  File "C:\Code\Python\lib\asyncio\events.py", line 14, in <module>
    import contextvars
  File "C:\Code\Python\lib\contextvars.py", line 1, in <module>
    from _contextvars import Context, ContextVar, Token, copy_context
ModuleNotFoundError: No module named '_contextvars'

_contextvarsmodule.c is in the list of Modules under the pythoncore project, and it appears to compile without incident when the Solution is built.

The executable appears to be fine in other respects I checked; e.g.,

C:\Code\Python\PCbuild>amd64\python.exe
Python 3.8.0a0 (heads/master:24f6846920, Sep 14 2018, 18:55:33) [MSC v.1915 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> import math
>>> from collections import Counter
>>> Counter("abab")
Counter({'a': 2, 'b': 2})
>>> import _contextvars
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_contextvars'

Clue?

[zooba]

It looks like _contextvars has both its own project and is also built as part of pythoncore.vcxproj.

However, _contextvars.vcxproj is not in PCbuild.sln, which means when you build through Visual Studio (rather than with build.bat via pcbuild.proj), you don't get the separate extension module.

And even though it's built into pythoncore.vcxproj, it hasn't been added to PC/config.c, and so it isn't initialised as a regular builtin module.

Yury - is there a specific reason for it to be a standalone module? How much do you think it hurts to have it always loaded into memory, even if it hasn't truly been imported or used?

[1st1]

How much do you think it hurts to have it always loaded into memory, even if it hasn't truly been imported or used?

I'd be fine with that. Would you be able to do the necessary change (I don't have VS at hand)?

[zooba]

Shouldn't need VS for this change (updating the SLN file generally does need it). I'm happy to do it though.

In case someone else gets to it before me:

• delete _contextvars.vcxproj and remove all references from PCbuild and Tools/msi
• add _contextvars entries to PC/config.c

The CI builds will tell you pretty quickly if it worked or not.

[tim-one]

FYI, I bet I didn't see a problem with the Win32 target because I followed instructions ;-) and did my first build using build.bat. Using that for the x64 too target makes the problem go away.

[HeleneMary]

Good night I installed the python program at the start it appears this message
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

>>

[ossdev07]

I am building python for windows on ARM64,
I am getting "No module named '_contextvars'" when trying to import contextvars:
>>> import contextvars
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\ps-emb\Downloads\cpython-master\cpython-master\lib\contextvars.py", line 1, in <module>
    from _contextvars import Context, ContextVar, Token, copy_context
ModuleNotFoundError: No module named '_contextvars'

thanks

[tim-one]

Precisely _how_ are you building it? As noted above, if you're using Visual Studio, try using build.bat instead for the first time.

[stevenwinfield]

(Just updating the issue to note that Python 3.7 is similarly affected)

[zooba]

New changeset 4c70d9f by Steve Dower in branch 'master':
bpo-34691: Compile _contextvars module into main Python library (GH-11741)
4c70d9f

[miss-islington]

New changeset 81eda28 by Miss Islington (bot) in branch '3.7':
bpo-34691: Compile _contextvars module into main Python library (GH-11741)
81eda28