Skip to content
This repository

[WIP] remove duplicate original data tracking (PHPCR-57) #118

Open
wants to merge 6 commits into from

3 participants

Lukas Kahwe Smith David Buchmann Christian Stocker
Lukas Kahwe Smith
Collaborator

the tests pass .. but there is a chance this doesn't cover all edge cases

David Buchmann
Collaborator
dbu commented March 05, 2012

i agree with this. to compensate for the limited test coverage, did you run this in your big project? or in the sandbox? if it does not fail in those, i would be pretty optimistic.

Lukas Kahwe Smith
Collaborator

no i have not .. but i agree we should before merging this .. maybe @chregu can try this as part of his current round of performance improvements. like i said, i think this could potentially improve memory consumption, though i am not sure if it will. i am fairly confident that it shouldn't hurt performance.

Lukas Kahwe Smith
Collaborator

@chregu has run some tests in our app and found some issues related to binary handling. guess i need to dive into this deeper than i had hoped was necessary :-/

David Buchmann
Collaborator
dbu commented March 07, 2012

they probably can't be part of the diff but need to be pushed separatly i guess. or somehow be encoded for json.

Lukas Kahwe Smith
Collaborator

umm maybe i just misunderstood you .. but sounds like you are commenting on the JSOP stuff?

David Buchmann
Collaborator
dbu commented March 07, 2012

indeed. sorry for the confusion, completely out of context.

Christian Stocker

JFTR. they can be part of the diff and they don't have to be encoded. multi-part posts FTW (and therefore saves some bytes, since we don't have to encode it in base64 ;))

But still out of context ;)

Lukas Kahwe Smith Merge remote-tracking branch 'origin/master' into remove_duplicate_or…
…iginal_data

Conflicts:
	lib/Doctrine/ODM/PHPCR/UnitOfWork.php
79fd881
Lukas Kahwe Smith
Collaborator

the issue i see in our project is really bizarre. we iterate over a list and in each iteration we do a flush. basically on the 5th iteration i get the following error:
[PHPCR\ValueFormatException]

Cannot convert value into a binary resource

this is caused by a resource suddenly being an "unknown type". not sure what could cause this.
however one thing to note here is that the node in question didn't really change. all that is being done is that a reference to the grandparent node is being created.

Christian Stocker

"unknown type" is usually after an fclose, JFYI :)

Lukas Kahwe Smith
Collaborator

so in general the issue might be more related to the fact that we are incorrectly detecting a change.
furthermore i wonder why we do "all or nothing" aka either we update everything .. or nothing.

Lukas Kahwe Smith
Collaborator

i changed the code to more precisely compute which fields actually need to be updated. it doesn't fix the issue i was seeing.

i have also made the same change based on current master in #123

David Buchmann
Collaborator
dbu commented March 13, 2012

i am currently refactoring the PHPCR PropertyType::convertType method. not sure if you are using that. but afaikt it should not affect you yet, i did only push minor cleanups.

added some commits March 14, 2012
Lukas Kahwe Smith Merge branch 'master' into remove_duplicate_original_data
Conflicts:
	lib/Doctrine/ODM/PHPCR/UnitOfWork.php
3eb8ca8
Lukas Kahwe Smith Merge branch 'master' into remove_duplicate_original_data
Conflicts:
	lib/Doctrine/ODM/PHPCR/UnitOfWork.php
d202d1a
David Buchmann
Collaborator
dbu commented March 19, 2012

should we merge now? if the tests pass, the cmf sandbox is not broken and you think its correct now, then lets just do it and see what happens :-)

Lukas Kahwe Smith
Collaborator

no, the issue i noticed in the test suite of our project isn't fixed yet. or at least last time i tried it wasn't fixed.

David Buchmann
Collaborator
dbu commented April 12, 2012

is this affected by the fix from uwe in #130 about duplicating calculating changesets?

Lukas Kahwe Smith
Collaborator

i will have to take another look .. there were a lot of changes that might have a positive effect on the goals of this sprint.

David Buchmann
Collaborator
dbu commented January 12, 2013

is this still relevant? if so it needs a rebase.

David Buchmann
Collaborator
dbu commented August 05, 2013

ping

David Buchmann
Collaborator

ping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 6 unique commits by 1 author.

Mar 04, 2012
Lukas Kahwe Smith remove duplicate original data tracking (#PHPCR-57) 7280b41
Mar 12, 2012
Lukas Kahwe Smith Merge remote-tracking branch 'origin/master' into remove_duplicate_or…
…iginal_data

Conflicts:
	lib/Doctrine/ODM/PHPCR/UnitOfWork.php
79fd881
Lukas Kahwe Smith more precisely determine what was actually changed when computing upd…
…ates
bc42a8f
Lukas Kahwe Smith Merge branch 'master' into remove_duplicate_original_data
Conflicts:
	lib/Doctrine/ODM/PHPCR/UnitOfWork.php
84c8742
Mar 14, 2012
Lukas Kahwe Smith Merge branch 'master' into remove_duplicate_original_data
Conflicts:
	lib/Doctrine/ODM/PHPCR/UnitOfWork.php
3eb8ca8
Mar 16, 2012
Lukas Kahwe Smith Merge branch 'master' into remove_duplicate_original_data
Conflicts:
	lib/Doctrine/ODM/PHPCR/UnitOfWork.php
d202d1a
This page is out of date. Refresh to see the latest.

Showing 1 changed file with 39 additions and 63 deletions. Show diff stats Hide diff stats

  1. 102  lib/Doctrine/ODM/PHPCR/UnitOfWork.php
102  lib/Doctrine/ODM/PHPCR/UnitOfWork.php
@@ -99,21 +99,6 @@ class UnitOfWork
99 99
     private $documentLocales = array();
100 100
 
101 101
     /**
102  
-     * PHPCR always returns and updates the whole data of a document. If on update data is "missing"
103  
-     * this means the data is deleted. This also applies to attachments. This is why we need to ensure
104  
-     * that data that is not mapped is not lost. This map here saves all the "left-over" data and keeps
105  
-     * track of it if necessary.
106  
-     *
107  
-     * @var array
108  
-     */
109  
-    private $nonMappedData = array();
110  
-
111  
-    /**
112  
-     * @var array
113  
-     */
114  
-    private $originalData = array();
115  
-
116  
-    /**
117 102
      * @var array
118 103
      */
119 104
     private $documentChangesets = array();
@@ -232,7 +217,6 @@ public function createDocument($className, $node, array &$hints = array())
232 217
         $class = $this->dm->getClassMetadata($className);
233 218
 
234 219
         $documentState = array();
235  
-        $nonMappedData = array();
236 220
         $id = $node->getPath();
237 221
 
238 222
         // second param is false to get uuid rather than dereference reference properties to node instances
@@ -352,11 +336,9 @@ public function createDocument($className, $node, array &$hints = array())
352 336
         }
353 337
 
354 338
         if ($overrideLocalValuesOid) {
355  
-            $this->nonMappedData[$overrideLocalValuesOid] = $nonMappedData;
356 339
             foreach ($class->reflFields as $prop => $reflFields) {
357 340
                 $value = isset($documentState[$prop]) ? $documentState[$prop] : null;
358 341
                 $reflFields->setValue($document, $value);
359  
-                $this->originalData[$overrideLocalValuesOid][$prop] = $value;
360 342
             }
361 343
         }
362 344
 
@@ -801,29 +783,24 @@ private function computeChangeSet(ClassMetadata $class, $document)
801 783
         }
802 784
 
803 785
         $actualData = $this->getDocumentActualData($class, $document);
  786
+        $id = $this->getDocumentId($document);
804 787
         $oid = spl_object_hash($document);
805  
-        $isNew = !isset($this->originalData[$oid]);
806  
-        if ($isNew) {
807  
-            // Document is New and should be inserted
808  
-            $this->originalData[$oid] = $actualData;
809  
-            $this->documentChangesets[$oid] = $actualData;
810  
-            $this->scheduledInserts[$oid] = $document;
811  
-        }
812 788
 
813 789
         if ($class->parentMapping && isset($actualData[$class->parentMapping])) {
814  
-            $parent = $actualData[$class->parentMapping];
815  
-            $parentClass = $this->dm->getClassMetadata(get_class($parent));
816  
-            $state = $this->getDocumentState($parent);
  790
+            $parentClass = $this->dm->getClassMetadata(get_class($actualData[$class->parentMapping]));
  791
+        }
817 792
 
  793
+        if (isset($parentClass)) {
  794
+            $state = $this->getDocumentState($actualData[$class->parentMapping]);
818 795
             if ($state === self::STATE_MANAGED || $state === self::STATE_MOVED) {
819  
-                $this->computeChangeSet($parentClass, $parent);
  796
+                $this->computeChangeSet($parentClass, $actualData[$class->parentMapping]);
820 797
             }
821 798
         }
822 799
 
823  
-        $id = $this->getDocumentId($document);
824 800
         foreach ($class->childMappings as $name => $childMapping) {
825 801
             if ($actualData[$name]) {
826  
-                if ($this->originalData[$oid][$name] && $this->originalData[$oid][$name] !== $actualData[$name]) {
  802
+                $child = $this->getDocumentById($id.'/'.$childMapping['name']);
  803
+                if ($child && $child !== $actualData[$name]) {
827 804
                     throw new PHPCRException('Cannot move/copy children by assignment as it would be ambiguous. Please use the DocumentManager::move() or PHPCR\Session::copy() operations for this: '.self::objToStr($document, $this->dm));
828 805
                 }
829 806
                 $this->computeChildChanges($childMapping, $actualData[$name], $id);
@@ -844,24 +821,26 @@ private function computeChangeSet(ClassMetadata $class, $document)
844 821
             }
845 822
         }
846 823
 
847  
-        if (!$isNew) {
848  
-            if (isset($this->originalData[$oid][$class->nodename])
849  
-                && isset($actualData[$class->nodename])
850  
-                && $this->originalData[$oid][$class->nodename] !== $actualData[$class->nodename]
  824
+        if (!$this->session->nodeExists($id)) {
  825
+            // Document is new and should be inserted
  826
+            $this->documentChangesets[$oid] = $actualData;
  827
+            $this->scheduledInserts[$oid] = $document;
  828
+        } else {
  829
+            $node = $this->session->getNode($id);
  830
+            $originalData = $node->getPropertiesValues();
  831
+
  832
+            if (isset($actualData[$class->nodename])
  833
+                && $node->getName() !== $actualData[$class->nodename]
851 834
             ) {
852  
-                throw new PHPCRException('The Nodename property is immutable ('.$this->originalData[$oid][$class->nodename].' !== '.$actualData[$class->nodename].'). Please use DocumentManager::move to rename the document: '.self::objToStr($document, $this->dm));
  835
+                throw new PHPCRException('The Nodename property is immutable ('.$node->getName().' !== '.$actualData[$class->nodename].'). Please use DocumentManager::move to rename the document: '.self::objToStr($document, $this->dm));
853 836
             }
854  
-            if (isset($this->originalData[$oid][$class->parentMapping])
855  
-                && isset($actualData[$class->parentMapping])
856  
-                && $this->originalData[$oid][$class->parentMapping] !== $actualData[$class->parentMapping]
857  
-            ) {
858  
-                throw new PHPCRException('The ParentDocument property is immutable ('.$this->getDocumentId($this->originalData[$oid][$class->parentMapping]).' !== '.$this->getDocumentId($actualData[$class->parentMapping]).'). Please use PHPCR\Session::move to move the document: '.self::objToStr($document, $this->dm));
  837
+            if (isset($parentClass) && ($parent = $this->getDocumentById(dirname($id))) && $parent !== $actualData[$class->parentMapping]) {
  838
+                throw new PHPCRException('The ParentDocument property is immutable ('.self::objToStr($parent, $this->dm).' !== '.self::objToStr($actualData[$class->parentMapping], $this->dm).'). Please use PHPCR\Session::move to move the document: '.self::objToStr($document, $this->dm));
859 839
             }
860  
-            if (isset($this->originalData[$oid][$class->identifier])
861  
-                && isset($actualData[$class->identifier])
862  
-                && $this->originalData[$oid][$class->identifier] !== $actualData[$class->identifier]
  840
+            if (isset($actualData[$class->identifier])
  841
+                && $id !== $actualData[$class->identifier]
863 842
             ) {
864  
-                throw new PHPCRException('The Id is immutable ('.$this->originalData[$oid][$class->identifier].' !== '.$actualData[$class->identifier].'). Please use DocumentManager::move to move the document: '.self::objToStr($document, $this->dm));
  843
+                throw new PHPCRException('The Id is immutable ('.$id.' !== '.$actualData[$class->identifier].'). Please use DocumentManager::move to move the document: '.self::objToStr($document, $this->dm));
865 844
             }
866 845
 
867 846
             if (!isset($this->documentLocales[$oid])
@@ -884,7 +863,9 @@ private function computeChangeSet(ClassMetadata $class, $document)
884 863
                                 $this->visitedCollections[] = $fieldValue;
885 864
                                 continue;
886 865
                             }
887  
-                        } elseif ($this->originalData[$oid][$fieldName] !== $fieldValue) {
  866
+                        } elseif (!array_key_exists($fieldName, $originalData)
  867
+                            || $originalData[$fieldName] !== $fieldValue
  868
+                        ) {
888 869
                             continue;
889 870
                         } elseif ($fieldValue instanceof ReferenceManyCollection && $fieldValue->changed()) {
890 871
                             continue;
@@ -901,8 +882,8 @@ private function computeChangeSet(ClassMetadata $class, $document)
901 882
             }
902 883
 
903 884
             foreach ($class->referrersMappings as $name => $referrerMapping) {
904  
-                if ($this->originalData[$oid][$name]) {
905  
-                    foreach ($this->originalData[$oid][$name] as $referrer) {
  885
+                if (isset($originalData[$name])) {
  886
+                    foreach ($originalData[$name] as $referrer) {
906 887
                         $this->computeReferrerChanges($referrer);
907 888
                     }
908 889
                 }
@@ -1380,16 +1361,16 @@ private function executeUpdates($documents, $dispatchEvents = true)
1380 1361
                             }
1381 1362
                         }
1382 1363
                     }
1383  
-                } elseif (isset($class->childMappings[$fieldName])) {
1384  
-                    if ($fieldValue === null) {
1385  
-                        if ($node->hasNode($class->childMappings[$fieldName]['name'])) {
1386  
-                            $child = $node->getNode($class->childMappings[$fieldName]['name']);
1387  
-                            $childDocument = $this->createDocument(null, $child);
1388  
-                            $this->purgeChildren($childDocument);
1389  
-                            $child->remove();
1390  
-                        }
1391  
-                    } elseif ($this->originalData[$oid][$fieldName] && $this->originalData[$oid][$fieldName] !== $fieldValue) {
1392  
-                        throw new PHPCRException('Cannot move/copy children by assignment as it would be ambiguous. Please use the DocumentManager::move() or PHPCR\Session::copy() operations for this.');
  1364
+                } elseif (isset($class->childMappings[$fieldName])
  1365
+                    && $node->hasNode($class->childMappings[$fieldName]['name'])
  1366
+                    && $fieldValue === null
  1367
+                ) {
  1368
+                    $childDocument = $this->getDocumentById($node->getPath().'/'.$class->childMappings[$fieldName]['name']);
  1369
+                    if (empty($childDocument) || $childDocument instanceof Proxy) {
  1370
+                        $child = $node->getNode($class->childMappings[$fieldName]['name']);
  1371
+                        $childDocument = $this->createDocument(null, $child);
  1372
+                        $this->purgeChildren($childDocument);
  1373
+                        $child->remove();
1393 1374
                     }
1394 1375
                 }
1395 1376
             }
@@ -1450,7 +1431,6 @@ private function executeMoves($documents)
1450 1431
                     $class = $this->dm->getClassMetadata(get_class($document));
1451 1432
                     if ($class->identifier) {
1452 1433
                         $class->setIdentifierValue($document, $newId);
1453  
-                        $this->originalData[$oid][$class->identifier] = $newId;
1454 1434
                     }
1455 1435
                 }
1456 1436
             }
@@ -1659,12 +1639,10 @@ private function unregisterDocument($document)
1659 1639
             $this->scheduledMoves[$oid],
1660 1640
             $this->scheduledInserts[$oid],
1661 1641
             $this->scheduledAssociationUpdates[$oid],
1662  
-            $this->originalData[$oid],
1663 1642
             $this->documentIds[$oid],
1664 1643
             $this->documentState[$oid],
1665 1644
             $this->documentTranslations[$oid],
1666 1645
             $this->documentLocales[$oid],
1667  
-            $this->nonMappedData[$oid],
1668 1646
             $this->documentChangesets[$oid],
1669 1647
             $this->documentHistory[$oid],
1670 1648
             $this->documentVersion[$oid]
@@ -1823,8 +1801,6 @@ public function clear()
1823 1801
         $this->documentState =
1824 1802
         $this->documentTranslations =
1825 1803
         $this->documentLocales =
1826  
-        $this->nonMappedData =
1827  
-        $this->originalData =
1828 1804
         $this->documentChangesets =
1829 1805
         $this->scheduledUpdates =
1830 1806
         $this->scheduledAssociationUpdates =
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.