New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change reversed() to [::-1] for speed #13

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
4 participants
@scragg0x

scragg0x commented Oct 2, 2012

No description provided.

@jeffwidman

This comment has been minimized.

Member

jeffwidman commented Apr 10, 2016

Skimming your PR made me curious... [::-1] only replaces reversed().

Why did you drop the ''.join() from your version?

I haven't dug into the surrounding code so I'm unaware of the overall calling context.

@scragg0x

This comment has been minimized.

scragg0x commented Apr 10, 2016

Ahh, such an old commit of mine. It would appear that I just neglected it. Good thing this wasn't pulled.

@davidism davidism closed this Apr 10, 2016

@davidism

This comment has been minimized.

Member

davidism commented Apr 10, 2016

I timed both versions and there was no significant change, so just going to close this and leave it as is.

@scragg0x

This comment has been minimized.

scragg0x commented Apr 10, 2016

https://gist.github.com/scragg0x/97d2927ce20b0766710bcee43795c2a2

Interestingly, I just did a bench and reversed it much faster anyway with larger lists. 25x faster when using the example gist.

@ThiefMaster

This comment has been minimized.

Member

ThiefMaster commented Apr 10, 2016

That benchmark is incorrect - reversed() doesn't actually reverse anything when called; it just provides an iterator that starts at the end of the collection

In [1]: lst = ['x'] * 1000
In [2]: %timeit lst[::-1]
100000 loops, best of 3: 4.37 µs per loop
In [3]: %timeit list(reversed(lst))
100000 loops, best of 3: 8.27 µs per loop

With strings:

In [1]: s = 'x' * 1000
In [2]: %timeit s[::-1]
1000000 loops, best of 3: 883 ns per loop
In [3]: %timeit ''.join(reversed(s))
10000 loops, best of 3: 20.7 µs per loop
@scragg0x

This comment has been minimized.

scragg0x commented Apr 10, 2016

@ThiefMaster I see, thanks for clarifying. I figured I missed something to have such a large disparity.

@jeffwidman

This comment has been minimized.

Member

jeffwidman commented Apr 11, 2016

Now that I understand what's happening here, I'd vote to reconsider and merge this.

BTW, thanks @ThiefMaster for taking the time to explain, I hadn't realized that reversed() only returns an iterator.

@davidism

This comment has been minimized.

Member

davidism commented Apr 11, 2016

After a (very silly) discussion about this micro-optimization, http://chat.stackoverflow.com/transcript/message/29868921#29868921, it looks like it would be better to replace the whole thing with the following, which outperforms everything else on py2 and py3 and doesn't require compat imports.

import struct

to_int64 = struct.Struct('>Q').pack

def int_to_bytes(num):
    return to_int64(num).lstrip(b'\x00')

timeit int_to_bytes(ts)
1000000 loops, best of 3: 398 ns per loop
@davidism

This comment has been minimized.

Member

davidism commented Apr 11, 2016

@davidism davidism added this to the 1.0.0 milestone Sep 28, 2018

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 10, 2018

Update py-itsdangerous to 1.1.0.
Version 1.1.0
-------------

Released 2018-10-26

-   Change default signing algorithm back to SHA-1. (`#113`_)
-   Added a default SHA-512 fallback for users who used the yanked 1.0.0
    release which defaulted to SHA-512. (`#114`_)
-   Add support for fallback algorithms during deserialization to
    support changing the default in the future without breaking existing
    signatures. (`#113`_)
-   Changed capitalization of packages back to lowercase as the change
    in capitalization broke some tooling. (`#113`_)

.. _#113: pallets/itsdangerous#113
.. _#114: pallets/itsdangerous#114


Version 1.0.0
-------------

Released 2018-10-18

YANKED

*Note*: This release was yanked from PyPI because it changed the default
algorithm to SHA-512. This decision was reverted in 1.1.0 and it remains
at SHA1.

-   Drop support for Python 2.6 and 3.3.
-   Refactor code from a single module to a package. Any object in the
    API docs is still importable from the top-level ``itsdangerous``
    name, but other imports will need to be changed. A future release
    will remove many of these compatibility imports. (`#107`_)
-   Optimize how timestamps are serialized and deserialized. (`#13`_)
-   ``base64_decode`` raises ``BadData`` when it is passed invalid data.
    (`#27`_)
-   Ensure value is bytes when signing to avoid a ``TypeError`` on
    Python 3. (`#29`_)
-   Add a ``serializer_kwargs`` argument to ``Serializer``, which is
    passed to ``dumps`` during ``dump_payload``. (`#36`_)
-   More compact JSON dumps for unicode strings. (`#38`_)
-   Use the full timestamp rather than an offset, allowing dates before
    2011. (`#46`_)
-   Detect a ``sep`` character that may show up in the signature itself
    and raise a ``ValueError``. (`#62`_)
-   Use a consistent signature for keyword arguments for
    ``Serializer.load_payload`` in subclasses. (`#74`_, `#75`_)
-   Change default intermediate hash from SHA-1 to SHA-512. (`#80`_)
-   Convert JWS exp header to an int when loading. (`#99`_)

.. _#13: pallets/itsdangerous#13
.. _#27: pallets/itsdangerous#27
.. _#29: pallets/itsdangerous#29
.. _#36: pallets/itsdangerous#36
.. _#38: pallets/itsdangerous#38
.. _#46: pallets/itsdangerous#46
.. _#62: pallets/itsdangerous#62
.. _#74: pallets/itsdangerous#74
.. _#75: pallets/itsdangerous#75
.. _#80: pallets/itsdangerous#80
.. _#99: pallets/itsdangerous#99
.. _#107: pallets/itsdangerous#107
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment