flush() seems unsetting embedded document's field

[nicoladarold]

I had this issue today, and i'm trying to describe it in detail:
Foreword: my code did not change, and I had this problem afther this update:

Updating doctrine/mongodb-odm dev-master (9addf53 => b1089b7)

Switching back to 9addf53 the behaviour returns as before.

I have an application that stores some documents, each one based on a structure describing the fields to be displayed.
To do this the DynamicDocument embeds a DynamicDocumentStructure, and the latter embeds some Field.
Templates are store in DynamicDocumentStructureTemplate, that embeds a DynamicDocumentStructure.

When I create a new DynamicDocument based on a particular template, I create a new instance of DynamicDocumentStructure, copy the Fields collection and some other data from the DynamicDocumentStructureTemplate I want to use and assign this DynamicDocumentStructure to the DynamicDocument.

Now the problem: after the update when persisting the newly created DynamicDocument, the Fields collection on the DynamicDocumentStructureTemplate is emptied.

Enabling the verbose log on mongodb i see this:

Fri Jul 10 10:04:38.120 [initandlisten] connection accepted from 192.168.105.1:57921 #397 (2 connections now open)
Fri Jul 10 10:04:38.121 [conn397] run command admin.$cmd { isMaster: 1 }
Fri Jul 10 10:04:38.121 [conn397] command admin.$cmd command: { isMaster: 1 } ntoreturn:1 keyUpdates:0 reslen:115 0ms
Fri Jul 10 10:04:38.122 [conn397] run command admin.$cmd { getnonce: 1 }
Fri Jul 10 10:04:38.122 [conn397] command admin.$cmd command: { getnonce: 1 } ntoreturn:1 keyUpdates:0 reslen:64 0ms
Fri Jul 10 10:04:38.122 [conn397] run command test_prosit.$cmd { authenticate: 1, user: "test_prosit", nonce: "73f98796356801d", key: "7e7532fd87b01df0cfc19d9f4062ab67" }
Fri Jul 10 10:04:38.122 [conn397] authenticate db: test_prosit { authenticate: 1, user: "test_prosit", nonce: "73f98796356801d", key: "7e7532fd87b01df0cfc19d9f4062ab67" }
Fri Jul 10 10:04:38.123 [conn397] command test_prosit.$cmd command: { authenticate: 1, user: "test_prosit", nonce: "73f98796356801d", key: "7e7532fd87b01df0cfc19d9f4062ab67" } ntoreturn:1 keyUpdates:0 locks(micros) r:429 reslen:83 0ms
Fri Jul 10 10:04:38.124 [conn397] run command admin.$cmd { ping: 1 }
Fri Jul 10 10:04:38.124 [conn397] command admin.$cmd command: { ping: 1 } ntoreturn:1 keyUpdates:0 reslen:37 0ms
Fri Jul 10 10:04:38.132 [conn397] query test_prosit.DynamicDocumentStructureTemplate query: { name: "SRD0-b-anamnestici" } ntoreturn:1 ntoskip:0 nscanned:1 keyUpdates:0 locks(micros) r:248 nreturned:1 reslen:29301 0ms
Fri Jul 10 10:04:38.450 [conn397] insert test_prosit.DynamicDocument ninserted:1 keyUpdates:0 locks(micros) w:74561 74ms
Fri Jul 10 10:04:38.450 [conn397] run command test_prosit.$cmd { getlasterror: 1 }
Fri Jul 10 10:04:38.450 [conn397] command test_prosit.$cmd command: { getlasterror: 1 } ntoreturn:1 keyUpdates:0 reslen:67 0ms
Fri Jul 10 10:04:38.460 [journal] lsn set 57818948
**Fri Jul 10 10:04:38.481 [conn397] update test_prosit.DynamicDocumentStructureTemplate query: { _id: ObjectId('52711a707f00416523000001') } update: { $set: { updated: new Date(1436515477000) } } idhack:1 nupdated:1 fastmod:1 keyUpdates:0 locks(micros) w:115 0ms**
Fri Jul 10 10:04:38.481 [conn397] run command test_prosit.$cmd { getlasterror: 1 }
Fri Jul 10 10:04:38.481 [conn397] command test_prosit.$cmd command: { getlasterror: 1 } ntoreturn:1 keyUpdates:0 reslen:85 0ms
**Fri Jul 10 10:04:38.483 [conn397] update test_prosit.DynamicDocumentStructureTemplate query: { _id: ObjectId('52711a707f00416523000001') } update: { $unset: { fields: true } } idhack:1 nupdated:1 keyUpdates:0 locks(micros) w:215 0ms**

The emphasized lines are absent using the bundle in 9addf53 or previous commit.
Now, the Structure uses the @gedmo\Timestampable annotation to keep updated datetime, so the first line should be just the jandling of the event fired.
It seems that someone deletes the Fields from the structure, and at the flush() the DM updates it phisically.

Now, my code when creating a new DynamicDocument is this:

$template = $this->get('doctrine_mongodb')
    ->getRepository('AcmeDynamicDocumentBundle:DynamicDocumentStructureTemplate')
    ->findOneById($templateid);
    // check for errors
    if ($template) {
        $document = new DynamicDocument();
        $structure = new DynamicDocumentStructure();
        $structure->setName($template->getName());
        $structure->setFields($template->getFields());
        $document->setStructure($structure);
        $dm->persist($document);
        $dm->flush();

Or could be that the previous bundle behaviour was wrong? The Fields I'm copying are maybe considered not just clones of Template Fields but the Fields themselves?
In this latter case the problem is my understanding of the document's embedding, and I'm in the wrong place. But this code is working well for 2 years now...

[malarzm]

Does $template->getFields() return collection of embedded documents? If so could you try clone each one of them? So it should become something like

foreach ($template->getFields() as $field) {
    $structure->addField(clone $field);
}

We recently tweaked how embedded documents are handled in #782 and that might be the case.

If that's not fixing your flow could please provide us a failing test case?

[nicoladarold]

Thanks, it could be that case. Monday I'll try to clone the Fields and will update this issue.

[malarzm]

@nicoladarold by the way I'd say clone each and every embedded document you're going to reuse in other document than its original parent :)

[nicoladarold]

I can confirm that, by my first tryings, cloning the embedded documents instead of copying them, the problem seems solved.
Thanks again! :-)