Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvements #19

Merged
merged 2 commits into from Jan 16, 2016

Conversation

aectann
Copy link
Contributor

@aectann aectann commented Jan 16, 2016

Re-use hmac object instead of creating a new one from scratch on every generate() call.

40% shorter running time for generate() function with thread_safe set to False, or 30% running time improvement when thread_safe parameter is true.

Code used to measure running time:

from libthumbor import CryptoURL
import cProfile


def generate():
    base_url = 'http://example.com/images{}.jpeg'
    crypto = CryptoURL(key='example_key_here')
    for i in xrange(200001):
        crypto.generate(image_url=base_url.format(i))

if __name__ == '__main__':
    cProfile.run('generate()', sort='cumulative')

Before the change:

   10400079 function calls in 4.910 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    4.910    4.910 <string>:1(<module>)
        1    0.136    0.136    4.910    4.910 load_test.py:5(generate)
   200001    0.179    0.000    4.701    0.000 crypto.py:62(generate)
   200001    0.610    0.000    4.480    0.000 crypto.py:54(generate_new)

After, thread_safe=True:

   8400061 function calls in 3.483 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    3.483    3.483 <string>:1(<module>)
        1    0.153    0.153    3.483    3.483 load_test.py:11(generate)
   200001    0.220    0.000    3.225    0.000 crypto.py:72(generate)
   200001    0.430    0.000    2.966    0.000 crypto.py:62(generate_new)

After, thread_safe=False:

   7600057 function calls in 2.985 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    2.985    2.985 <string>:1(<module>)
        1    0.151    0.151    2.985    2.985 load_test.py:11(generate)
   200001    0.167    0.000    2.732    0.000 crypto.py:72(generate)
   200001    0.355    0.000    2.525    0.000 crypto.py:62(generate_new)

@@ -53,7 +61,9 @@ def generate_old(self, options):

def generate_new(self, options):
url = plain_image_url(**options)
signature = base64.urlsafe_b64encode(hmac.new(b(self.key), text_type(url).encode('utf-8'), hashlib.sha1).digest())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You still need to convert the key to bytes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, addressed that in ea4843a.

@aectann aectann force-pushed the feature/speed-improvements branch 2 times, most recently from cd641fb to ea4843a Compare January 16, 2016 12:21
guilhermef added a commit that referenced this pull request Jan 16, 2016
@guilhermef guilhermef merged commit c0b7922 into thumbor:master Jan 16, 2016
@guilhermef
Copy link
Member

Thanks @aectann, that's an awesome change.
I'll release a new version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants