Pegen regenerate project for Windows not working

[tonybaloney]

BPO	40432
Nosy	@pfmoore, @tjguk, @jkloth, @ambv, @zware, @zooba, @tonybaloney, @pablogsal, @miss-islington, @Fidget-Spinner
PRs	bpo-40432 Fix MSBuild project for Pegen grammars #19785 bpo-40432: Use python 3.8 or higher to compile cpython on windows #25389 [3.9] bpo-40432: Use python 3.8 or higher to compile CPython on Windows (GH-25389) #25637

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tonybaloney'
closed_at = <Date 2021-04-26.23:17:22.911>
created_at = <Date 2020-04-29.02:37:10.121>
labels = ['build', '3.9', 'OS-windows']
title = 'Pegen regenerate project for Windows not working'
updated_at = <Date 2021-04-26.23:17:22.910>
user = 'https://github.com/tonybaloney'

bugs.python.org fields:

activity = <Date 2021-04-26.23:17:22.910>
actor = 'steve.dower'
assignee = 'anthonypjshaw'
closed = True
closed_date = <Date 2021-04-26.23:17:22.911>
closer = 'steve.dower'
components = ['Build', 'Windows']
creation = <Date 2020-04-29.02:37:10.121>
creator = 'anthonypjshaw'
dependencies = []
files = []
hgrepos = []
issue_num = 40432
keywords = ['patch']
message_count = 8.0
messages = ['367613', '367634', '390970', '391281', '391397', '391739', '391969', '391982']
nosy_count = 10.0
nosy_names = ['paul.moore', 'tim.golden', 'jkloth', 'lukasz.langa', 'zach.ware', 'steve.dower', 'anthonypjshaw', 'pablogsal', 'miss-islington', 'kj']
pr_nums = ['19785', '25389', '25637']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40432'
versions = ['Python 3.9']

[tonybaloney]

The additional tasks in the MSBuild project for pegen regeneration are not functional:

• Setting PYTHONPATH= inline cannot be done in Windows using that method
• The task does not inherit environment variables that way
• The path to the peg_generator module is in Unix path format

PR to follow

[pablogsal]

New changeset 9b64ef3 by Anthony Shaw in branch 'master':
bpo-40432 Fix MSBuild project for Pegen grammars (#GH-9785)
9b64ef3

[pablogsal]

Seems that the regeneration script is not detecting correctly that needs Python3.8+ for regenerating pegen. Check for instance:

https://buildbot.python.org/all/#/builders/596/builds/2/steps/2/logs/stdio

[jkloth]

Adding windows team to the nosy list

[zooba]

That'll be a change to PCbuild/find_python.bat that needs to be backported then. It probably defaults to 3.7 in all branches right now, but since 3.9 is out we can update them all to that (it downloads on demand if necessary).

[zooba]

Thanks for the PR, kj! Left a suggestion for you, but otherwise it should solve the problem nicely.

[zooba]

New changeset 425434d by Ken Jin in branch 'master':
bpo-40432: Use python 3.8 or higher to compile CPython on Windows (bpo-25389)
425434d

[ambv]

New changeset f65f3f0 by Miss Islington (bot) in branch '3.9':
bpo-40432: Use python 3.8 or higher to compile CPython on Windows (GH-25389) (bpo-25637)
f65f3f0