DBAL-106: Doctrine\DBAL\Schema\Comparator false positives

[doctrinebot]

Jira issue originally created by user lfeistel:

I am using the model class annotations for my Doctrine schema definition and a MySQL database for my target. When testing code that uses Doctrine\DBAL\Schema\Comparator, either by using $schemaTool->updateSchema($classes) which is defined in the core functionality or by running doctrine migrations:diff from the Doctrine Migrations project, a set of bugs are revealed. The symptoms of this are that immediately after calling $schemaTool->createSchema($classes) any call that uses the Comparator class to calculate the diffs between the real database and the schema definition produce a large set of false positives. Most of these false positives can be traced back to the logic in Doctrine\DBAL\Schema\Comparator::diffColumn(Column $column1, Column $column2). There are two issues here I can see right away:

One is that if the model annotations rely on defaults rather than explicitly defining certain attributes, this function sees them as different. For example, if I have a column defined like this: @Column(type="decimal", scale=2, nullable=true) then the Comparator believes it is different from the real database because precision is 10 in the real data and 0 in my schema. If I change the line to this: if (($column1->getPrecision() ?: 10) != ($column2->getPrecision() ?: 10)) then it works, but I am sure a real solution needs to determine what the defaults actually are.

The second issue is when I use the 'columnDefinition' attribute with a custom type to declare an enum or set, for instance. Since this function does not actually check what the columnDefinition field is set to at all, it has no way to know if I have actually made a change to the list of values, or anything else about that columnDefinition. I have been reading that using the comments field might be another way to do this, but I am not yet clear on what the best practice really is here. I know that http://www.doctrine-project.org/jira/browse/[DBAL-42](http://www.doctrine-project.org/jira/browse/DBAL-42) touches on this with regard to storing Array type metadata in the comments field. I need more direction on how this should work in custom field types.

In addition, I am getting this strange set of diffs on the index for a ManyToMany association's join table:

        $this->addSql("CREATE INDEX IDX*44B8C6D89AF7860 ON firm_firm (firm*id)");
        $this->addSql("DROP INDEX primary ON firm_firm");
        $this->addSql("CREATE UNIQUE INDEX primary ON firm*firm (firm_id, firm*id)");

I am not sure why this only happens on this one table and not others, but it seems to be another case of the Comparator getting a false positive for some reason this time in Comparator::diffIndex(Index $index1, Index $index2). The model annotations for this association are:

  /****
   * @var Firm
   * @ManyToMany(targetEntity="Firm")
   */
  protected $Children = array();

I will add more to this as I learn more details about the issues, but hopefully this is enough to start grokking the issues in Comparator. Getting the Comparator class in the core Doctrine\DBAL library working properly for calculating diffs will go a long way toward making well defined migrations a reality for Doctrine2 projects running in production. Hopefully, someone will also be able to address this minor issue in the Doctrine Migrations (http://www.doctrine-project.org/jira/browse/[DMIG-21](http://www.doctrine-project.org/jira/browse/DMIG-21)) so that it can be deployed without needing a patch.

Thanks to everyone for Doctrine2, and I am looking forward to being among the first to put this new library into production on a real world project in a few months.

[doctrinebot]

Comment created by @beberlei:

Can you tell me how the indexes on the join table look like?

[doctrinebot]

Comment created by lfeistel:

Now that I look at this schema, I think I need to apologize for my oversight because I didn't explain the problem properly. The real issue is that this self-referential ManyToMany association needs to have the id fields explicitly defined or it will not work. Doctrine2 is generating this table by apparently assuming both id fields have the same name, and so the second index that is created gets a randomly generated name assigned to it. There is no way this association is actually working properly at all, so I'll talk to the programmer that wrote this. I think we can fix this part of the problem ourselves, but perhaps some error checking here would be helpful. The first half of the problem (in diffColumn) remains. Thanks again for looking into this.

CREATE TABLE IF NOT EXISTS firm_firm (
firm_id int(11) NOT NULL,
PRIMARY KEY (firm_id),
KEY IDX*44B8C6D89AF7860 (firm*id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

ALTER TABLE firm_firm
ADD CONSTRAINT firm*firm_ibfk_1 FOREIGN KEY (firm*id) REFERENCES firm (id) ON DELETE CASCADE;

[doctrinebot]

Comment created by @beberlei:

Fixed the precision issue, please open a new ticket for the many to many.

[doctrinebot]

Issue was closed with resolution "Fixed"

[doctrinebot]

Comment created by lfeistel:

Thanks. At least the precision issue was easy, so glad to have that one out of the way. Is there any hope for a way to handle the use of columnDefiniton? I guess the alter statements being generated are just changing it to the same value, so it might not actually be broken in and of itself. Those statements could take a small amount of time to execute what is essentially a noop. Maybe we just have to manually go through and clean out the unwanted alter statements for the time being. It might be workable.

[doctrinebot]

Comment created by @beberlei:

Updated version

[doctrinebot]

Comment created by @beberlei:

columnDefinition always comes up in the diff. There is just no way around it. The problem is that there is no generic way to create a "columnDefinition" from the existing databse, so no good value to compare it too. If you can think of a solution for this problem i would be very happy, but we havent come up with one yet.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.