DoctrineMetadataCacheWarmer must load metadata first, check priority of you warmers when updated from 2.2.4 -> 2.3.0

[ivanbogomoloff]

Hello all.
I got this message when run bin/console cache:clear or bin/console cache:warmup.

Conditions:

I have symfony 5.2.4 project out box
Then i run composer update doctrine-bundle updated to 2.2.4 -> 2.3.0.

Then i got error.

DoctrineMetadataCacheWarmer must load metadata first, check priority of you warmers

[dmaicher]

I cannot reproduce this on any of my projects. I also tested it on symfony-demo and it works fine.

Can you provide a reproducer @ivanbogomoloff ?

[ivanbogomoloff]

I cannot reproduce this on any of my projects. I also tested it on symfony-demo and it works fine.

Can you provide a reproducer @ivanbogomoloff ?

No, i can't. It's magically gone.
But now It occurs only in project with a lot of code, but when i copy composer.json and try to create raw new project (i mean out box is composer command create project) it works fine.
Little notice, that error occurs only after rm -rf var/cache/prod and when APP_ENV=prod

[alexrozz]

I have the same issue on cache:clear, doctrine:migrations:migrate and any other.
Could anybody explain the reason of this exception? What warmers must run before?
In DoctrineMetadataCacheWarmer:47 there is a check if metadata exists, then crush. What the purpose, anybody knows?

[dmaicher]

Could anybody explain the reason of this exception? What warmers must run before?
In DoctrineMetadataCacheWarmer:47 there is a check if metadata exists, then crush. What the purpose, anybody knows?

@alexrozz

With DoctrineBundle 2.3.0 we introduced a new and faster caching mechanism for the Doctrine meta data. For this to work properly we have to ensure that DoctrineMetadataCacheWarmer is the first to actually load all meta data so the cache can be properly warmed up.

Are you able to create a reproducer? Happy to take a look then

[jkabat]

I'm facing the issue as well. In my project it has relation to "msgphp/msgphp" package. In my understanding problematic is the change of warmers priority introduced in #1260 - other packages may be bitten as well if they depend on specific priority value.

[dmaicher]

In my understanding problematic is the change of warmers priority introduced in #1260

we did not change any priority of cache warmers 😉 We introduced a new DoctrineMetadataCacheWarmer with a high priority of 1000.

Again: if someone can provide a reproducer I can re-open this and take a look.

[jkabat]

I have created a reproducer, but I think it is related with the library Im using (it processes/loads entity metadata and should probably be run before doctrine warmup step which is not true anymore as new warmer has priority 1000, while msgphp one only 100).

https://github.com/jkabat/doctrine-bundle-test

bin/console cache:clear --env=prod --no-debug # http://prntscr.com/10zwd8u

In my application setup I had better insight with cache:warmup command: http://prntscr.com/10zwfkn. Unfortunately in reproducer I do not see this kind of error.

[ostrolucky]

Reproducer doesn't work for me

❯ /usr/local/Cellar/php@7.4/7.4.16/bin/php bin/console cache:clear --env=prod --no-debug -vvv

 // Clearing the cache for the prod environment with debug false

 // Clearing outdated warmup directory...

 // Warming up cache...

[info] User Deprecated: The "metadata_cache_driver" configuration key is deprecated. PHP Array cache is now automatically registered when %kernel.debug% is false.

[info] User Deprecated: The "metadata_cache_driver" configuration key is deprecated. PHP Array cache is now automatically registered when %kernel.debug% is false.


In MappingException.php line 65:

  [Doctrine\Persistence\Mapping\MappingException]
  No mapping file found named 'EmailPassword.orm.xml' for class 'MsgPhp\User\Credential\EmailPassword'.


Exception trace:
  at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/MappingException.php:65
 Doctrine\Persistence\Mapping\MappingException::mappingFileNotFound() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/SymfonyFileLocator.php:230
 Doctrine\Persistence\Mapping\Driver\SymfonyFileLocator->findMappingFile() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/FileDriver.php:92
 Doctrine\Persistence\Mapping\Driver\FileDriver->getElement() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/Driver/XmlDriver.php:67
 Doctrine\ORM\Mapping\Driver\XmlDriver->loadMetadataForClass() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/MappingDriverChain.php:79
 Doctrine\Persistence\Mapping\Driver\MappingDriverChain->loadMetadataForClass() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:149
 Doctrine\ORM\Mapping\ClassMetadataFactory->doLoadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:306
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:79
 Doctrine\ORM\Mapping\ClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:184
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->getMetadataFor() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:163
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->getMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:100
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->processFieldMapping() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:77
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->processClassFields() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:81
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->processClassFields() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:62
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->loadClassMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/doctrine-bridge/ContainerAwareEventManager.php:64
 Symfony\Bridge\Doctrine\ContainerAwareEventManager->dispatchEvent() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:239
 Doctrine\ORM\Mapping\ClassMetadataFactory->doLoadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:306
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:79
 Doctrine\ORM\Mapping\ClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:184
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->getMetadataFor() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:90
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->getAllMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/doctrine-bundle/CacheWarmer/DoctrineMetadataCacheWarmer.php:51
 Doctrine\Bundle\DoctrineBundle\CacheWarmer\DoctrineMetadataCacheWarmer->doWarmUp() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/CacheWarmer/AbstractPhpFileCacheWarmer.php:49
 Symfony\Bundle\FrameworkBundle\CacheWarmer\AbstractPhpFileCacheWarmer->warmUp() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/CacheWarmer/CacheWarmerAggregate.php:97
 Symfony\Component\HttpKernel\CacheWarmer\CacheWarmerAggregate->warmUp() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/Kernel.php:633
 Symfony\Component\HttpKernel\Kernel->initializeContainer() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/Kernel.php:136
 Symfony\Component\HttpKernel\Kernel->boot() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/Kernel.php:153
 Symfony\Component\HttpKernel\Kernel->reboot() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Command/CacheClearCommand.php:189
 Symfony\Bundle\FrameworkBundle\Command\CacheClearCommand->warmup() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Command/CacheClearCommand.php:129
 Symfony\Bundle\FrameworkBundle\Command\CacheClearCommand->execute() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Application.php:1027
 Symfony\Component\Console\Application->doRunCommand() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Console/Application.php:97
 Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Application.php:273
 Symfony\Component\Console\Application->doRun() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Console/Application.php:83
 Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Application.php:149
 Symfony\Component\Console\Application->run() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/bin/console:42

cache:clear [--no-warmup] [--no-optional-warmers] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-e|--env ENV] [--no-debug] [--] <command>

❯

[Eydamos]

I have the same issue. Unfortunately the project is private so I can't share it. I also, up to this point, failed to reproduce the behaviour in a new project. But after hours of debugging I can already give some insight why this is happening.

In my project I have several connections/entity manager and therefore multiple cache warmer are registered all with priority 1000.

For simplicity let's say we have the connections conn_a and conn_b which have the entity managers em_a and em_b and their cache warmers are also registered in this order. So we have two iterations of cache warmers first the cache warmer who gets the entity manager em_a and then the cache warmer who gets the entity manager em_b.

Now when I clear the cache the following happens:

Iteration 1 (em_a)

In DoctrineMetadataCacheWarmer in line 59 we have $metadataFactory->getAllMetadata(); which will load all the metadata for all entities of the entity manager em_a.
During this method call the ClassMetadataFactory will call in line 244 if ($this->evm->hasListeners(Events::loadClassMetadata)) {.

And here comes the tricky part I was not able to reproduce yet as I don't know how symfony determines which event manager should get a certain subscriber but the following happens:

The event manager of em_a has a doctrine event subscriber assigned that I have written.
This subscriber has a dependency on a custom repository.
This custom repository will get the entity manager em_b and calls $entityManager->getRepository(SomeEntityFromConnB::class) in it's constructor.
Now the entity_manager em_b will generate the metadata for SomeEntityFromConnB class.

Iteration 2 (em_b)

Now the cache warmer of entity manager em_b will do it's run and will fetch from it the meta data factory in line 47 of the DoctrineMetadataCacheWarmer and as the entity manager has already created metadata for SomeEntityFromConnB even though there are still several entities that do not have meta data generated the if condition in line 48 will throw an exception.

I will continue my quest to provide an example so you can debug it yourself but I hope I have already clearly outlined the cause for the bug.

[dmaicher]

@TimWerdin this sounds indeed like it could be the issue...

But then creating a reproducer should be possible when simply following your comments?

[Eydamos]

I guess it will not take too much time to be able to produce a demo but I'm currently busy with testing and deploying a big project. I will continue my work next Thuesday

[Eydamos]

I was a little bit busy last week but now I could prepare a demo application which shows the cause of the bug. You can find it here: https://github.com/TimWerdin/doctrine-bug-demo

[dmaicher]

The reproducer looks good to me. Thanks @TimWerdin