Skip to content

Don't unnecessarily copy the key list #238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Don't unnecessarily copy the key list #238

wants to merge 2 commits into from

Conversation

dmitchell
Copy link

__iter__, has_key, __contains__, clear, & len all copy the __keys list but don't need to.

Minor backward compatibility change: make __iter__ generate a runtime error if the user attempts to destructively modify the SON during iteration just as dict does and have the others directly use the internal list.

Conflicts:
	doc/contributors.rst
	test/test_son.py
@dmitchell
Copy link
Author

@dmitchell
Copy link
Author

We also need this as tag 2.4.1a based on tag 2.4.1. We can maintain that in our fork for now if that's preferable.

@@ -120,14 +120,20 @@ def copy(self):
# efficient.
# second level definitions support higher levels
def __iter__(self):
for k in self.keys():
"""
Cannot remove nor add keys while iterating
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't appear to be enforced by collections.OrderedDict, so I don't think we want to enforce it here.

http://hg.python.org/releasing/3.4.1/file/ea310ca42bb2/Lib/collections/__init__.py#l83

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. Thanks for noticing. I'll change it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should act the same was as it would if iterating over a list then: that is, any destructive operations can cause unexpected behavior such as this shows (iteration skips an ele due to the removal of a prior ele):

>>> a = [1, 2, 3]
>>> for ele in a:
...     print ele
...     a.remove(ele)
... 
1
3
>>> a
[2]
>>> 

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>>> od = OrderedDict([('a', 1), ('b', 2), ('c', 3)])
>>> for key in od:
...     print key
...     del od[key]
... 
a
b
c
>>> od
OrderedDict()
>>> for key in od:
...     print key
... 
>>> 

>>> from bson.son import SON
>>> s = SON([('a', 1), ('b', 2), ('c', 3)])
>>> for key in s:
...     print key
...     del s[key]
... 
a
b
c
>>> s
SON([])
>>> for key in s:
...     print key
... 

@behackett
Copy link
Member

Thanks for the patch. Most of the changes seem reasonable to me. We won't release a 2.4.x version of the driver for this though. That's pretty old and a ton of bugs have been fixed since 2.4.

and let python handle (identical to iter on list)
@dmitchell
Copy link
Author

Alas, this didn't improve performance enough; so, we're back to using dict
and worrying about instable subdoc order.

On Wed, Jun 4, 2014 at 4:21 PM, Bernie Hackett notifications@github.com
wrote:

Thanks for the patch. Most of the changes seem reasonable to me. We won't
release a 2.4.x version of the driver for this though. That's pretty old
and a ton of bugs have been fixed since 2.4.


Reply to this email directly or view it on GitHub
#238 (comment)
.

@behackett
Copy link
Member

You can use collections.OrderedDict with PyMongo. bson.son.SON really only exists because PyMongo supports python versions older than python 2.7. Using some trivial benchmarks we've found that SON serializes a bit faster than OrderedDict. Your mileage may vary.

@dmitchell
Copy link
Author

We found the same dict << SON < OrderedDict

On Fri, Jun 6, 2014 at 10:51 AM, Bernie Hackett notifications@github.com
wrote:

You can use collections.OrderedDict with PyMongo. bson.son.SON really only
exists because PyMongo supports python versions older than python 2.7.
Using some trivial benchmarks we've found that SON serializes a bit faster
than OrderedDict. Your mileage may vary.


Reply to this email directly or view it on GitHub
#238 (comment)
.

@behackett behackett assigned behackett and ajdavis and unassigned behackett Oct 17, 2014
@ajdavis
Copy link
Member

ajdavis commented Oct 24, 2014

Merged into 2.8 and 3.0 release branches. Thanks for the contribution!

@ajdavis ajdavis closed this Oct 24, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants