In 0.5+, call arguments to a save method broke.

[amcgregor]

I get the following traceback with MongoEngine 0.5.1 (or 0.5.2) installed, but don't get it with 0.5 installed, regardless of the version of pymongo (tested to pymongo<2.0 and against latest on pypi):

Traceback (most recent call last):
  File "template.py", line 47, in <module>
    settings.save() ; settings.attach(root)
  File "/Users/amcgregor/.../model/__init__.py", line 163, in attach
    self.save()
  File "/Users/amcgregor/.../model/__init__.py", line 224, in save
    return super(Asset, self).save(safe=safe, force_insert=force_insert, validate=validate)
  File "/Users/amcgregor/Projects/sample/lib/python2.7/site-packages/mongoengine/document.py", line 177, in save
    _refs=_refs)
TypeError: save() got an unexpected keyword argument 'write_options'

This appears to be a regression in the handling of reference fields.

[rozza]

@amcgregor - apologies for the delay - will chase up

[rozza]

Hi @amcgregor

I can't see anything that would break save:

v0.5...v0.5.1
v0.5.1...v0.5.2

I think write options were added in 0.5

Ross

[amcgregor]

So instead of investigating the crash, which exists, the ticket gets closed. Here is a somewhat minimal demonstration of the crash. Works in 0.4 (with latest pymongo) and doesn't work in 0.5 or above.

Traceback (most recent call last):
  File "test.py", line 133, in <module>
    company.save(); company.attach(root)
  File "test.py", line 57, in attach
    self.save()
  File "test.py", line 80, in save
    return super(Asset, self).save(safe=safe, force_insert=force_insert, validate=validate)
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/document.py", line 177, in save
    _refs=_refs)
TypeError: save() got an unexpected keyword argument 'write_options'

[mjtamlyn]

Hi @amcgregor,

In v0.5 an option was added to the save method (write_options) which your subclass of Document does not permit. If you update the signature (default params) it will work again. In general it's safer to use *args, **kwargs when subclassing.

[amcgregor]

@mjtamlyn

Now I get an infinite loop:

Traceback (most recent call last):
  File "test.py", line 133, in <module>
    company.save(); company.attach(root)
  File "test.py", line 63, in attach
    parent.save()
  File "test.py", line 80, in save
    return super(Asset, self).save(**kw)
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/document.py", line 144, in save
    self.validate()
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/base.py", line 618, in validate
    for name, field in self._fields.items()]
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/base.py", line 179, in __get__
    instance._data.get(self.name), max_depth=1, instance=instance, name=self.name, get=True
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/dereference.py", line 46, in __call__
    self.reference_map = self._find_references(items)
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/dereference.py", line 78, in _find_references
    references = self._find_references(v, depth)
 …
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/dereference.py", line 78, in _find_references
    references = self._find_references(v, depth)
  File "/Users/amcgregor/Projects/crash-test/lib/python2.7/site-packages/mongoengine/dereference.py", line 74, in _find_references
    elif isinstance(v, (dict, pymongo.son.SON)) and '_ref' in v:
RuntimeError: maximum recursion depth exceeded while calling a Python object

Updated the gist with the updated code.

[mjtamlyn]

Hmmm, definitely some sort of issue here. The problem stems from your circular referencing (parent references child references parent etc). I don't know a huge amount about how that works I'm afraid. There was a fix in v0.5.2 which related to circular references, it was using an EmbeddedDocument to do them - perhaps this is the recommended use?

This test case may give you some inspiration. v0.5...v0.5.2#diff-7

[amcgregor]

Could this ticket be re-opened, then? There is clearly a regression. This used to work. Right now I'm having to bind the maximum installed version of MongoEngine to 0.4. Circular references are somewhat integral in how my objects store their graph. (@mjtamlyn Or do you want me to open a new one?)

Also note that those tests don't test single-table circular references; they test two-table circular references.

[mjtamlyn]

I would suggest submitting a new bug, complete with a minimal-breaking test case. It's not really related to the original problem.

[rozza]

hi @amcgregor - to be fair I did investigate as you said the regression was from 0.5 to 0.5.1 and I stated that write_options were introduced in in 0.5 - apologies for missing it off the upgrade document. (http://mongoengine.org/docs/v0.5/upgrade.html?highlight=upgrade)

[rozza]

@amcgregor I've added #373 for the max recursion - thanks for the test case - its a good catch!

[rozza]

This should now be fixed in the dev branch and hopefully is the last dereferencing regression caused by 0.5