Conditional save for atomic-style operations

[FrankBattaglia]

Software often requires an atomic-style database read-and-modify, i.e.:

1. read a record from the database into memory
2. modify the record in some way
3. store the modified record in the database

in a way that ensures no other changes to the database record have occurred between steps 1 and 3. A simple example of this pattern is document versioning. E.g.:

1. read version (n) of a document from the database
2. modify the document, creating version (n+1)
3. store version (n+1) in the database

If two users (e.g., user A and user B) attempt to perform this type of operation at the same time, it it uncertain which new version of the document will be stored in the database, but the other can be lost irrevocably. Furthermore, depending on the nature of the updates (e.g., if the updates are expressed as "deltas" between documents), this situation can result in invalid documents being stored in the database.

Mongo's collection.update method includes a query document and an update document that allows a user to accomplish the aforementioned atomic-style read-and-update. Continuing with the versioning example, a client can include the expected version number in the query document. E.g.:

db.collection.update(
    {_id:document_id, version:n},
    {
        $set: 
        {
            value: changed,
            version: (n+1)
        }
    }
)

If the document version stored in the database does not match (e.g., if another client has updated the database document(, the database document is not altered.

MongoEngine allows a developer to directly construct an update query (see MongoEngine User Guide 2.5.9: Atomic Updates). E.g., one could write:

Document.objects(id=document_id, version=n).update(set__value=changed, set__version=n+1)

However, explicitly constructing the update query using this 'out of band' style, while it is less cumbersome than generating the full PyMongo object, is awkward, can get unwieldily if multiple document fields are to be updated, and removes much of the benefit of an Object-Document Mapper like MongoEngine. What is desired is a way to achieve "atomic-style read-and-modify" operations using MongoEngine's ODM methods

This PR includes a new optional save_condition parameter to the MongoEngine's Document.save() method. This parameter can be a dictionary of conditions that will be added to the collection.update() call embedded in the Document.save() method. For example, continuing the versioning example, one could write:

document = Document.objects.get()
n = document.version
modify(document)
document.version = n+1
document.save(save_condition={'version': n})

With the above code, the modified document would only be saved in the version of the document stored in the Mongo database was still equal to n; i.e., if someone else had modified the document in the interim, then the save would fail.

Of course, this is a trivial example of the pattern, but the general problem of "only save these updates if the underlying document hasn't been changed" is common and I think MongoEngine could benefit from a less cumbersome method to solve it.

Please let me know if you have any question or comments.

[FrankBattaglia]

This seems to the the same as #460
Closes #460

[FrankBattaglia]

For what it's worth, a slightly more complicated example of why this type of feature would be beneficial (admittedly the versioning example could be addressed by other means):

Assume a person can have pants secured by a belt. I.e., if a person has a belt, his pants cannot be removed. Further, a person can't put on a belt unless he is weaning pants (individual proclivities notwithstanding). This might be implemented by the following:

def put_on_pants(person_id):
    person = Persons.objects.get(id=person_id)
    person.pants = True
    person.save()

def put_on_belt(person_id):
    # a belt requires pants
    person = Persons.objects.get(id=person_id)
    if person.pants:
        person.belt = True
        person.save()

def remove_pants(person_id):
    # can't remove pants if wearing a belt
    person = Persons.objects.get(id=person_id)
    if not person.belt:
        person.pants = False
        person.save()

Assume a person with pants = True If simultaneous calls to put_on_belt and remove_pants occur, they reads and writes may be interleaved, so that the resulting record has a belt but no pants! This could be resolved using the save_condition parameter:

def put_on_pants(person_id):
    person = Persons.objects.get(id=person_id)
    person.pants = True
    person.save()

def put_on_belt(person_id):
    # a belt requires pants
    person = Persons.objects.get(id=person_id)
    person.belt = True
    person.save(save_condition={'pants':True})

def remove_pants(person_id):
    # can't remove pants if wearing a belt
    person = Persons.objects.get(id=person_id)
    person.pants = False
    person.save(save_condition={'belt':False})

One of the calls would fail to update the record (determining which could be accomplished by a person.reload()) and the data would remain consistent.

[rozza]

Thanks @franksomething this looks interesting marking it for 0.9!

[yograterol]

Closed by user petition. PR accept #585