DDC-211: Exception is thrown after many calls to flush()

[doctrinebot]

Jira issue originally created by user baumgartl:

Two classes having a many-to-many association:

namespace Entity;

/****
 * @Entity
 * @Table(name="user")
*/
class User
{
    /****
     * @Id
     * @Column(name="id", type="integer")
     * @GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /****
     * @Column(name="name", type="string")
     */
    protected $name;

    /****
    * @ManyToMany(targetEntity="Group")
    *   @JoinTable(name="user_groups",
    *       joinColumns={@JoinColumn(name="user_id", referencedColumnName="id")},
    *       inverseJoinColumns={@JoinColumn(name="group_id", referencedColumnName="id")}
    *   )
    */
    protected $groups;

    public function **construct() {
        $this->groups = new \Doctrine\Common\Collections\ArrayCollection();
    }

    public function setName($name) { $this->name = $name; }

    public function getGroups() { return $this->groups; }
}

namespace Entity;

/****
 * @Entity
 * @Table(name="groups")
 */
class Group
{
    /****
     * @Id
     * @Column(name="id", type="integer")
     * @GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /****
     * @Column(name="name", type="string")
     */
    protected $name;

    /****
    * @ManyToMany(targetEntity="User", mappedBy="groups")
    */
    protected $users;

    public function **construct() {
        $this->users = new \Doctrine\Common\Collections\ArrayCollection();
    }

    public function setName($name) { $this->name = $name; }

    public function getUsers() { return $this->users; }
}

Fill the database with some data:

$em = EntityManager::create($connectionOptions, $config);

$user = new \Entity\User();
$user->setName('John Doe');

$em->persist($user);
$em->flush();

$groupNames = array('group 1', 'group 2', 'group 3', 'group 4');
foreach ($groupNames as $name) {

    $group = new \Entity\Group();
    $group->setName($name);
    $em->persist($group);
    $em->flush();

    if (!$user->getGroups()->contains($group)) {
        $user->getGroups()->add($group);
        $group->getUsers()->add($user);
        $em->flush();
    }
}

After the user was added to the third group, an exception is thrown:

PHP Fatal error:  Uncaught exception 'PDOException' with message 'SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-1' for key 'PRIMARY'' in /lib/Doctrine/DBAL/Connection.php:623
Stack trace:
#0 /lib/Doctrine/DBAL/Connection.php(623): PDOStatement->execute(Array)
#1 /lib/Doctrine/ORM/Persisters/AbstractCollectionPersister.php(121): Doctrine\DBAL\Connection->executeUpdate('INSERT INTO use...', Array)
#2 /lib/Doctrine/ORM/Persisters/AbstractCollectionPersister.php(101): Doctrine\ORM\Persisters\AbstractCollectionPersister->insertRows(Object(Doctrine\ORM\PersistentCollection))
#3 /lib/Doctrine/ORM/UnitOfWork.php(306): Doctrine\ORM\Persisters\AbstractCollectionPersister->update(Object(Doctrine\ORM\PersistentCollection))
#4 /lib/Doctrine/ORM/EntityManager.php(288): Doctrine\ORM\UnitOfWork->commit()
#5 [...] in /lib/Doctrine/DBAL/Connection.php on line 623

Using the EchoSqlLogger a strange SQL statement becomes visible (after the user was added to "group 3"):

INSERT INTO groups (name) VALUES (?)
array(1) {
  [1]=>
  string(7) "group 4"
}
DELETE FROM user*groups WHERE user_id = ? AND group*id = ?
array(2) {
  [0]=>
  int(1)
  [1]=>
  int(3)
}
INSERT INTO user*groups (user_id, group*id) VALUES (?, ?)
array(2) {
  [0]=>
  int(1)
  [1]=>
  int(1)
}

Where does the delete statement come from? There is none of that in the code.

Interestingly the example works fine if the user is just added to three groups. If flush() is just called once at the end of the script, everything is fine too.

Many calls to flush() are expected to slow down execution time and/or increase memory consumption, but should work properly.

[doctrinebot]

Comment created by nicokaiser:

The delete statement seems to come from \Doctrine\ORM\PersistentCollection#getDeleteDiff(), which returns the entities from the association that should be deleted (in this case this happens iff you delete associations between M:N entities):

This is done by an *array_udiff($this->_snapshot, $this->_coll->toArray(), array($this, '_compareRecords'))*, where _compareRecords is nothing more than $a === $b.

I modified getDeleteDiff() so it displays what it does and pasted the output here: http://pastie.org/744175
(the DIFF sections are only the DeleteDiffs). After successfully adding 3 Categories (303, 304, 305) Doctrine suddenly decides that the association Group[134]/Category[305] should be deleted, and then Group[134]/Category303 is re-inserted, which produces an exception of course.

The full log with SQL is here http://pastie.org/744183 (here the Group has id 138 instead of 134).

[doctrinebot]

Comment created by romanb:

array_udiff is very weird. If anyone can explain me this behavior, I would be delighted:

class Foo {
    public function **construct($val) {
        $this->name = $val;
    }
    public $name;
}

function compare($a, $b) {
    return $a === $b ? 0 : 1;
}

$foo1 = new Foo('one');
$foo2 = new Foo('two');
$foo3 = new Foo('three');
$foo4 = new Foo('four');
$foo5 = new Foo('five');

$snapshot = array();
$coll = array($foo1);
var*dump(array*udiff($snapshot, $coll, 'compare'));
// => array(0) { }

$snapshot = array($foo1);
$coll = array($foo1, $foo2);
var*dump(array*udiff($snapshot, $coll, 'compare'));
// => array(0) { }

$snapshot = array($foo1, $foo2);
$coll = array($foo1, $foo2, $foo3);
var*dump(array*udiff($snapshot, $coll, 'compare'));
// => array(0) { }

$snapshot = array($foo1, $foo2, $foo3);
$coll = array($foo1, $foo2, $foo3, $foo4);
var*dump(array*udiff($snapshot, $coll, 'compare'));
// => array(1) { [2]=>  object(Foo)#3 (2) { ["name"]=>  string(5) "three" } } 

$snapshot = array($foo1, $foo2, $foo3, $foo4);
$coll = array($foo1, $foo2, $foo3, $foo4, $foo5);
var*dump(array*udiff($snapshot, $coll, 'compare'));
// => array(2) { [1]=>  object(Foo)#2 (2) { ["name"]=>  string(3) "two" } [3]=>  object(Foo)#4 (2) { ["name"]=>  string(4) "four" } } 

The array_udiff behavior is the reason for this problem. I just dont understand it.

According to the manual: "Returns an array containing all the values of array1 ($snapshot in this case) that are not present in any of the other arguments. "

[doctrinebot]

Comment created by romanb:

This should now be fixed. We're using array_udiff_assoc instead which behaves as expected and is faster. Also, this is a necessary preparation for DDC-213. This can now lead to join-table elements being deleted and reinserted when they change position in the collection, however this is expected and indeed needed for DDC-213. I don't see this as an issue for "unordered" collections.

[doctrinebot]

Issue was closed with resolution "Fixed"