Add the --duplicate option for timeit

[serhiy-storchaka]

BPO	30080
Nosy	@tim-one, @birkenfeld, @rhettinger, @pitrou, @vstinner, @vstinner, @stevendaprano, @alex, @bitdancer, @serhiy-storchaka
PRs	bpo-30080: Add the --duplicate option for CLI of timeit. #1161

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2022-03-16.17:17:08.638>
created_at = <Date 2017-04-16.06:12:44.636>
labels = ['3.7', 'type-feature', 'library']
title = 'Add the --duplicate option for timeit'
updated_at = <Date 2022-03-16.17:17:08.638>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2022-03-16.17:17:08.638>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2022-03-16.17:17:08.638>
closer = 'serhiy.storchaka'
components = ['Demos and Tools', 'Library (Lib)']
creation = <Date 2017-04-16.06:12:44.636>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 30080
keywords = []
message_count = 4.0
messages = ['291736', '291771', '295317', '295326']
nosy_count = 9.0
nosy_names = ['tim.peters', 'georg.brandl', 'rhettinger', 'pitrou', 'vstinner', 'vstinner', 'steven.daprano', 'alex', 'r.david.murray', 'serhiy.storchaka']
pr_nums = ['1161']
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue30080'
versions = ['Python 3.7']

[serhiy-storchaka]

One of the most used by me option of the "perf timeit" subcommand is --duplicate. It duplicates statements to reduce the overhead of the loop. This is necessary when measure the time of very fast statements. Proposed patch adds this option for CLI of the timeit module.

Similar feature already was proposed in bpo-21988, but it automatically duplicated statements if they executed too fast. This patch does this only on explicit request. And it affects only command-line interface. You need to duplicate statements manually when use programming interface.

[rhettinger]

+1 I've long used this technique when timing fast statements. See https://code.activestate.com/recipes/577834 for an example.

[vstinner]

About the command line interface, I would expect that "timeit -n 10 --duplicate 10 STMT" would run STMT 100 times. I chose this behaviour in perf timeit:
http://perf.readthedocs.io/en/latest/cli.html#timeit-cmd

Current behaviour chosen by Serhiy, -n3 --duplicate=3 runs STMT 3 times:

$ ./python -m timeit --duplicate=3 -r 1 -n 3 'print("bla")'
bla
bla
bla
3 loops, best of 1: 21.8 usec per loop

[serhiy-storchaka]

Ah, I even didn't know that "perf timeit" supports -n since it doesn't report the number of loops. And seems it runs the benchmark much more times, since even with -n1 it is slow. If the number of loops is determined automatically, it would do not matter.

I choose the meaning of --duplicate so that it almost not affect the total time of the benchmarking. Larger value just decreases the overhead of the iteration.