Speedup UTCDateTimeAttribute deserialization #277
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`UTCDateTimeAttribute.deserialize` spends most of it's time inside of dateutil which attempts to first guess the format. In most cases PynamoDB will have written the data to begin with, so we can first optimistically assume the data is stored in DATETIME_FORMAT before falling back to dateutil. This ~4.5 times faster than always going through dateutil. ```python from dateutil.parser import parse import datetime from dateutil.tz import tzutc DATETIME_FORMAT = '%Y-%m-%dT%H:%M:%S.%f%z' now_str = datetime.datetime.now().replace(tzinfo=tzutc()).strftime(DATETIME_FORMAT) %timeit parse(now_str) # 116 µs ± 9.46 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each) %timeit datetime.datetime.strptime(now_str, DATETIME_FORMAT) # 26 µs ± 7.53 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each) ```
lita
reviewed
May 11, 2017
| """ | ||
| expected_value = datetime(2047, 1, 6, 8, 21, tzinfo=tzutc()) | ||
| attr = UTCDateTimeAttribute() | ||
| self.assertEqual( |
There was a problem hiding this comment.
It would help me a lot if you changed this to assert expected_value == attr.deserialize('January 6, 2047 at 8:21:00AM UTC') as I would like to move this towards pytest style unit tests. but if it feels weird to have two different styles, I can do it later.
There was a problem hiding this comment.
@lita Agreed, but I would do that in a follow on PR. Having two different testing styles in the same file would be a bit weird.
garrettheel
reviewed
May 11, 2017
pynamodb/attributes.py
Outdated
| @@ -394,7 +395,11 @@ def deserialize(self, value): | |||
| """ | |||
| Takes a UTC datetime string and returns a datetime object | |||
| """ | |||
| return parse(value) | |||
| # First attempt to parse the datetime with the | |||
There was a problem hiding this comment.
This comment is incomplete, did you accidentally
garrettheel
approved these changes
May 11, 2017
garrettheel
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
UTCDateTimeAttribute.deserializespends most of it's time inside of dateutil which attempts to firstguess the format. In most cases PynamoDB will have written the data to begin with, so we can first
optimistically assume the data is stored in DATETIME_FORMAT before falling back to dateutil.
This ~4.5 times faster than always going through dateutil.