Provide optimized HMAC digest

[tiran]

BPO	32433
Nosy	@gpshead, @tiran
PRs	bpo-32433: Optimized HMAC digest #5023

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-01-27.08:57:12.584>
created_at = <Date 2017-12-27.19:00:28.290>
labels = ['extension-modules', '3.7', 'performance']
title = 'Provide optimized HMAC digest'
updated_at = <Date 2018-01-27.08:57:12.583>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2018-01-27.08:57:12.583>
actor = 'christian.heimes'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-27.08:57:12.584>
closer = 'christian.heimes'
components = ['Extension Modules']
creation = <Date 2017-12-27.19:00:28.290>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 32433
keywords = ['patch']
message_count = 2.0
messages = ['309097', '310849']
nosy_count = 2.0
nosy_names = ['gregory.p.smith', 'christian.heimes']
pr_nums = ['5023']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue32433'
versions = ['Python 3.7']

[tiran]

hmac.HMAC's flexibility comes with a cost. To create the MAC of a message, it has to create between three and four hash instances (inner, outer, key for long keys, result), multiple bound method objects and other temporary objects. For short messages, memory allocation, object handling and GIL release/acquire operations dominate the performance.

I propose to provide a fast one-shot HMAC function: hmac.digest(key, msg, digstmod) -> bytes function. A PoC implementation based on OpenSSL's HMAC() function and a pure Python implementation as inlined HMAC showed promising performance improvements. The C implementation is 3 times faster.

Standard HMAC:
$ ./python -m timeit -n200000 -s "import hmac" -- "hmac.HMAC(b'key', b'message', 'sha256').digest()"
200000 loops, best of 5: 5.38 usec per loop

Optimized Python code:
$ ./python -m timeit -n 200000 -s "import hmac; hmac._hashopenssl = None" -- "hmac.digest(b'key', b'message', 'sha256')"
200000 loops, best of 5: 3.87 usec per loop

OpenSSL HMAC()
$ ./python -m timeit -n 200000 -s "import hmac" -- "hmac.digest(b'key', b'message', 'sha256')"
200000 loops, best of 5: 1.82 usec per loop

[tiran]

New changeset 2f050c7 by Christian Heimes in branch 'master':
bpo-32433: Optimized HMAC digest (bpo-5023)
2f050c7