Windows Tools\buildbot\clean.bat misses some needed pyc/pyo removal

[db3l]

BPO	43709
Nosy	@pfmoore, @db3l, @tjguk, @zware, @zooba
PRs	bpo-43709: Remove pyc/pyo files from Tools/Parser in buildbot clean.bat #25157

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-04-04.17:16:58.886>
created_at = <Date 2021-04-02.20:56:43.051>
labels = ['3.10', 'build', 'OS-windows', 'type-crash']
title = 'Windows Tools\\buildbot\\clean.bat misses some needed pyc/pyo removal'
updated_at = <Date 2021-04-04.17:16:58.885>
user = 'https://github.com/db3l'

bugs.python.org fields:

activity = <Date 2021-04-04.17:16:58.885>
actor = 'db3l'
assignee = 'none'
closed = True
closed_date = <Date 2021-04-04.17:16:58.886>
closer = 'db3l'
components = ['Build', 'Windows']
creation = <Date 2021-04-02.20:56:43.051>
creator = 'db3l'
dependencies = []
files = []
hgrepos = []
issue_num = 43709
keywords = ['patch']
message_count = 4.0
messages = ['390098', '390100', '390164', '390201']
nosy_count = 5.0
nosy_names = ['paul.moore', 'db3l', 'tim.golden', 'zach.ware', 'steve.dower']
pr_nums = ['25157']
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue43709'
versions = ['Python 3.10']

[db3l]

The Tools\buildbot\clean.bat script used on Windows only removes pyc/pyo files from the Lib tree, leaving some files beneath Tools (clinic and peg_generator) and Parser (asdl). This can cause failures following commits that affect those files, such as fcb55c0 (in issue bpo-27129) leading to subsequent crashes or anomalous behavior.

This appears to have been true for a while, though only the 3.10 branch ran into the issue, so any fix might also be back-ported to the other active branches.

[db3l]

Something like this is a quick 'n dirty minimal fix - at least it seems to solve the problem that arose in issue bpo-27129 on the Win10 buildbot:

--- a/Tools/buildbot/clean.bat
+++ b/Tools/buildbot/clean.bat
@@ -11,6 +11,8 @@ call "%pcbuild%\build.bat" -t Clean -k -d %*
 
 echo Deleting .pyc/.pyo files ...
 del /s "%root%\Lib\*.pyc" "%root%\Lib\*.pyo"
+del /s "%root%\Tools\*.pyc" "%root%\Tools\*.pyo"
+del /s "%root%\Parser\*.pyc" "%root%\Parser\*.pyo"
 
 echo Deleting test leftovers ...
 rmdir /s /q "%root%\build"

but since this only covers the buildbot script, someone working within the regular VS IDE and just using the clean target of the projects could presumably run into the same issue. Or is there some other way to fully clean a working tree from within the IDE? Not sure how important that is to address in any event.

[db3l]

Based on further information in issue bpo-27129 as well as issue bpo-43719 it appears a source of the problem prompting this fix was a failure to update the magic number in the original commit for bpo-27129.

The windows clean script does still leave more artifacts than, for example, the Unix Makefile, but it shouldn't have resulted in a problem if the magic number had changed, and it's not clear it has to change after all. So this issue can probably be closed once that is corrected.

[db3l]

No longer needed after commit c368ce7