Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide optimized HMAC digest #76614

tiran opened this issue Dec 27, 2017 · 2 comments

Provide optimized HMAC digest #76614

tiran opened this issue Dec 27, 2017 · 2 comments
3.7 extension-modules performance


Copy link

@tiran tiran commented Dec 27, 2017

BPO 32433
Nosy @gpshead, @tiran
  • #5023
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2018-01-27.08:57:12.584>
    created_at = <Date 2017-12-27.19:00:28.290>
    labels = ['extension-modules', '3.7', 'performance']
    title = 'Provide optimized HMAC digest'
    updated_at = <Date 2018-01-27.08:57:12.583>
    user = '' fields:

    activity = <Date 2018-01-27.08:57:12.583>
    actor = 'christian.heimes'
    assignee = 'none'
    closed = True
    closed_date = <Date 2018-01-27.08:57:12.584>
    closer = 'christian.heimes'
    components = ['Extension Modules']
    creation = <Date 2017-12-27.19:00:28.290>
    creator = 'christian.heimes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32433
    keywords = ['patch']
    message_count = 2.0
    messages = ['309097', '310849']
    nosy_count = 2.0
    nosy_names = ['gregory.p.smith', 'christian.heimes']
    pr_nums = ['5023']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'performance'
    url = ''
    versions = ['Python 3.7']

    Copy link
    Member Author

    @tiran tiran commented Dec 27, 2017

    hmac.HMAC's flexibility comes with a cost. To create the MAC of a message, it has to create between three and four hash instances (inner, outer, key for long keys, result), multiple bound method objects and other temporary objects. For short messages, memory allocation, object handling and GIL release/acquire operations dominate the performance.

    I propose to provide a fast one-shot HMAC function: hmac.digest(key, msg, digstmod) -> bytes function. A PoC implementation based on OpenSSL's HMAC() function and a pure Python implementation as inlined HMAC showed promising performance improvements. The C implementation is 3 times faster.

    Standard HMAC:
    $ ./python -m timeit -n200000 -s "import hmac" -- "hmac.HMAC(b'key', b'message', 'sha256').digest()"
    200000 loops, best of 5: 5.38 usec per loop

    Optimized Python code:
    $ ./python -m timeit -n 200000 -s "import hmac; hmac._hashopenssl = None" -- "hmac.digest(b'key', b'message', 'sha256')"
    200000 loops, best of 5: 3.87 usec per loop

    OpenSSL HMAC()
    $ ./python -m timeit -n 200000 -s "import hmac" -- "hmac.digest(b'key', b'message', 'sha256')"
    200000 loops, best of 5: 1.82 usec per loop

    @tiran tiran added 3.7 extension-modules performance labels Dec 27, 2017
    Copy link
    Member Author

    @tiran tiran commented Jan 27, 2018

    New changeset 2f050c7 by Christian Heimes in branch 'master':
    bpo-32433: Optimized HMAC digest (bpo-5023)

    @tiran tiran closed this as completed Jan 27, 2018
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    3.7 extension-modules performance
    None yet

    No branches or pull requests

    1 participant