make regen-all triggers wholesale rebuild

[pitrou]

BPO	31536
Nosy	@Yhg1s, @pitrou, @vstinner, @zware, @serhiy-storchaka
PRs	bpo-31536: Avoid wholesale rebuild after make regen-opcode #3677 bpo-31536: Avoid wholesale rebuild after make regen-all #3678 [3.6] bpo-31536: Avoid wholesale rebuild after make regen-all (GH-3678) #3797

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-09-28.11:21:50.898>
created_at = <Date 2017-09-20.18:37:37.574>
labels = ['type-feature', '3.7', 'build']
title = '`make regen-all` triggers wholesale rebuild'
updated_at = <Date 2017-09-28.11:21:50.869>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2017-09-28.11:21:50.869>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2017-09-28.11:21:50.898>
closer = 'pitrou'
components = ['Build']
creation = <Date 2017-09-20.18:37:37.574>
creator = 'pitrou'
dependencies = []
files = []
hgrepos = []
issue_num = 31536
keywords = ['patch']
message_count = 12.0
messages = ['302649', '302651', '302652', '302653', '302654', '302662', '302663', '302665', '302670', '303203', '303204', '303222']
nosy_count = 5.0
nosy_names = ['twouters', 'pitrou', 'vstinner', 'zach.ware', 'serhiy.storchaka']
pr_nums = ['3677', '3678', '3797']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31536'
versions = ['Python 3.6', 'Python 3.7']

[pitrou]

make regen-opcode always writes the destination unconditionally, after which make thinks it has to rebuild everything.

[pitrou]

Note that at least make regen-ast will also need to be changed in order to reduce build times on Travis-CI.

[pitrou]

Looks like pgen will also have to be changed, which is more annoying as it's written in C...

Perhaps another approach can be used that will use a separate utility to do the copying if the contents changed?

[pitrou]

Posted #3678 which uses a separate utility. It seems to save 1 minute off the build step on Travis-CI.

[pitrou]

Note that GNU install (install -C) would probably do the trick, but is not necessarily installed on the build system.

[vstinner]

New changeset b091bec by Victor Stinner (Antoine Pitrou) in branch 'master':
bpo-31536: Avoid wholesale rebuild after make regen-all (bpo-3678)
b091bec

[vstinner]

If it works well, maybe it's worth it to backport the change to the 3.6 branch. We still backport a lot of changes to 3.6. But maybe wait until 3.6.3 final is released?

[pitrou]

If it works well, maybe it's worth it to backport the change to the 3.6 branch. We still backport a lot of changes to 3.6. But maybe wait until 3.6.3 final is released?

Fair enough. Does someone want to do the backport? :-)

[vstinner]

I may do the backport, or ask the bot to do it. Right now I prefer to check if everything is fine on all CI.

[pitrou]

Victor, are the buildbots fine now?

[vstinner]

Victor, are the buildbots fine now?

I didn't see anything wrong on buildbots, nor Travis CI/AppVeyor on PRs. So we can now backport the "optimization". Do you want to do it, Antoine?

[pitrou]

New changeset ec3d34c by Antoine Pitrou (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31536: Avoid wholesale rebuild after make regen-all (GH-3678) (bpo-3797)
ec3d34c