A way to enable auto_mapping with multipe Entity Manager

[goetas]

Currently i have 50 active bundles in my project.

10 bundles do not require database interaction.
39 bundles uses the default entity manager, that uses the default dbal connection.
1 bundle use the 'custom' entity manager, that uses the 'custom' dbal connection.

to do this, i need to disable auto_mapping and specify all mappings.

doctrine:
    dbal:
        default_connection: default
        connections:
                default:
                    driver:   %database_driver%
                   ....
                custom:
                    driver:   pdo_pgsql
                    ...
    orm:
        entity_managers:
            default:
                mappings:
                    Bundle1: ~
                    Bundle2: ~
                    Bundle3: ~
                    Bundle4: ~
                    Bundle4: ~
                    ........
            superdpi:
                mappings: 
                    SuperDpiCoreBundle: ~
                connection: custom     

this is very annoying...
currently i've resolved the problem in this way:
1)

doctrine:
    orm:
        entity_managers:
            default:
                auto_mapping : true
            superdpi:
                mappings: 
                    SuperDpiCoreBundle: ~
                connection: custom     

removed auto_mapping check inside DoctrineExtension::loadOrmEntityManager()
3)
removed auto mapping expansion inside AbstractDoctrineExtension::loadMappingInformation()
4) added DoctrineExtension::collectAutoMappings()

protected function collectAutoMappings (array $entityManagerConfigs, ContainerBuilder $container){
    $defindedBundles = array_keys($container->getParameter('kernel.bundles'));
    foreach ($entityManagerConfigs as $name => $entityManager) {
        if(isset($entityManager['auto_mapping']) && $entityManager['auto_mapping']){
            foreach ($defindedBundles as $bundle){
                foreach ($entityManagerConfigs as $name2 => $entityManager2) {
                    if($entityManager2['mappings'][$bundle]){
                        continue 2;
                    }
                }
                $entityManagerConfigs[$name]['mappings'][$bundle]['mapping']=true;
            }
        }
    }
    return $entityManagerConfigs;
}

5. added

$config['entity_managers'] = $this->collectAutoMappings($config['entity_managers'], $container);

inside DoctrineExtension::ormLoad() just before

foreach ($config['entity_managers'] as $name => $entityManager) {
       $entityManager['name'] = $name;
       $this->loadOrmEntityManager($entityManager, $container);
}

suggestions?

[stof]

How would you determine which bundles are mapped to each entity manager ?

[goetas]

If a bundle is listed inside an entity manager definition, then associate it to that entity manager, else associate it to entity manager that uses "auto_mapping" attribute.

[goetas]

should also decide what to do if two or more entity managers use auto_mapping" attribute...

1. prohibit this auto_mapping combination
2. allow, bundle will be associated to the last entity manager that appears in the list

[stof]

2 entity managers using auto_mapping should be prohibited to avoid WTF for the user. Associating bundles to the entity manager using auto_mapping if they are not used by others would make the code far more complex in the DI extension as we would need to look at all other managers to be able to find the bundles instead of simply getting the full list. I'm not sure how much complexity it will add (especially as the code handling the mapping is partially in the bridge to be shared).

Thus, how would you consider a bundle listed in another entity manager but setting the mapping to false to disable it ?

[goetas]

I also consider that allow 2 two EM with auto_mapping should be a bad idea.

Regarding a bundle listed in another entity manager, we have to skip always that bundle.
To do this, we need to change

if($entityManager2['mappings'][$bundle]){
   continue 2;
}

to

if($name2 === $name || isset($entityManager2['mappings'][$bundle])){
   continue 2;
}

[goetas]

About complexity, your are talking about performance or about 'code complexity'?

(sorry for my bad english)

[stof]

code complexity. Performance is not an issue as the DI extension code is only used when building the container (i.e. only when deploying for the prod environment)

[goetas]

all code complexity is enclosed inside collectAutoMappings() method...

should be added this feature (or something similar) inside DoctrineBundle? What about AbstractDoctrineExtension that is enclosed inside Symfony2?

[stof]

@goetas but the code calling collectAutoMappings only knows about the config of the entity manager being configured. It does not have access to the config of others currently

[goetas]

collectAutoMappings requires the entire $config['entity_managers'] array to work, as explained at point (5) of my first post

[HadesArchitect]

Hi. I have this problem too. Do you have any simple solutions, or i should make a PR with @goetas changes?

[goetas]

I'm waiting for some inputs from symfony community... currently i'm using the suggested implementation inside my forked version of symfony/doctrine bundle

[HadesArchitect]

Well, I'm got it. I don't see any problems to make a PR, but i don't want to do it if it will be declined =)

Waiting for decision...

[stephen-leavitt-sonyatv-com]

While I see this hasn't received any traction recently, here is a commit against a forked branch from the v1.2.0 tag. It is completely self-contained and requires no modification of AbstractDoctrineExtension, and it also handles the logic if auto_mapping is on in two entity manager at the same time.

SonyATV@c2c5559

[lsmith77]

I very much agree that we should make this easier. I think its a very common case to need one EntityManager for most Bundles and only another EM for a few other Bundles.

@stephen-leavitt-sonyatv-com I would recommend to open a PR, its always easier to provide feedback with an open PR

[stof]

@lsmith77 Can you find a way to make the auto_mapping work for multiple managers without registering the same entity in several managers ?
I don't wan tto push users into the broken cases by default (managing the same class with several managers causes all sorts of weird issues later)

[lsmith77]

I have not dug into it .. but why can we not simply have a step before the current call that is aware of the entire config of all managers, that finds out which Bundles are manually mapped and then automatically maps all others to a default EM?

[stephen-leavitt-sonyatv-com]

@lsmith77 I will look into providing a PR request for this as soon as time permits. As I recall the commit was clean and self-contained. I applied it against 1.2.0, since that requires Doctrine 2.4, whereas the work on 1.3.0 requires Doctrine 2.5, which is something I am not prepared to support on the application I am working on. What branch/tag would you prefer the PR to be against? Thanks.

[stof]

@stephen-leavitt-sonyatv-com the PR should be based on the master branch of the bundle (and btw, it is still compatible with Doctrine 2.4)

[stephen-leavitt-sonyatv-com]

@stof Thanks, I will try to work up a PR against master as soon as time permits. If the commit I referenced earlier applies cleanly against master, I will submit a PR as soon as I can. If not, it may take me a little more time to do so.

FYI, I think the issue I encountered in 1.3.0 (beta1 last I looked) was trying to use the support for entity listener resolvers that is present in 1.3.0 (a separate issue entirely not related to this issue), and every time I tried to use 1.3.0, composer tried to pull in Doctrine 2.5 (which may or may not have been from master, I forget now; I've tried to sleep since then). Since I had no desire to use Doctrine from master or an unreleased version, I simply back-ported the support into a forked version of doctrine-bundle instead against the tagged 1.2.0 version.

[stof]

the beta1 is indeed broken with 2.4. the master rbanch works

[lsmith77]

ping

[stephen-leavitt-sonyatv-com]

@lsmith Sorry, I just haven't had much time this last week to work up the PR, my apologies. It's still on my radar to do time permitting... :)

[helmut-hoffer-von-ankershoffen]

Would be really great if auto_mapping would be allowed when using multiple entity managers by marking one of the entity managers as the default one.

The current solution (having to manually specify all mappings) is very failure prone in large sf2 projects e.g. when requiring new bundles - as we have to check if those bundles add additional ones as a dependency and if those bundles use mappings ...

[lsmith77]

+1 for @helmuthva suggestion

[lsmith77]

see #322

[vishalmelmatti]

Hi All,

I am using multiple entity managers for different databases, below is my configuration

orm:
    auto_generate_proxy_classes: %kernel.debug%
    default_entity_manager: default
    entity_managers:
        default:
            connection: default
            mappings:
              MyDemoBundle : ~
              MyMessagingBundle: ~
              SonataNotificationBundle: ~
              ApplicationSonataNotificationBundle: ~
              MyCustome1Bundle : ~
        custom1:
            connection: custom1
            mappings:
              MyCustome1Bundle : ~
              MyDemoBundle : ~

If you see MyCustome1Bundle and MyDemoBundle are under both entity managers. I had to add them under both as I was not able to access Custom bundle repositories under Demo bundle and vise versa. There is associations between entities in different entity managers so I have to access them in bundles in different entity managers.

I works but the problem is, doctrine schema features doesn't works as it tries to find tables under custom entity managers in default database and vise versa.

What is the best way to handle fix this issue ?

[stof]

what do you mean by I was not able to access Custom bundle repositories under Demo bundle and vise versa ?

[lsmith77]

#321 will hopefully soon be merged which will enable auto mapping with multiple entity managers in symfony 2.6+

[vishalmelmatti]

@stof Sorry, I will explain use case in detail.

I mean, entity in other namespace is not accessible if we try to map them (cross bundle entity association in case where bundles are configured in different namespaces) . I get this type of error e.g

[Doctrine\Common\Persistence\Mapping\MappingException]
The class 'Acme\UserBundle\Entity\User' was not found in the chain configured namespaces Acme\DemoBundle\Entity

To demonstrate the use case I have created sample symfony project with AcmeDemoBundle (Comment Entity) and AcmeUserBundle (Use Entity)

https://github.com/vishalmelmatti/symfony_doctrine_namespace

# Doctrine Configuration
doctrine:
    dbal:
        default_connection: default
        connections:
            default:
                driver:   "%database_driver%"
                host:     "%database_host%"
                port:     "%database_port%"
                dbname:   "%database_name%"
                user:     "%database_user%"
                password: "%database_password%"
                charset:  UTF8
            user_db:
                driver:   "%database_driver%"
                host:     "%database_host%"
                port:     "%database_port%"
                dbname:   "%database_name1%"
                user:     "%database_user%"
                password: "%database_password%"
                charset:  UTF8

    orm:
        default_entity_manager: default
        entity_managers:
            default:
                connection: default
                mappings:
                    AcmeDemoBundle:  ~
            user_db:
                connection: user_db
                mappings:
                    AcmeUserBundle: ~

For this use case, none of the doctrine commands work. It fails with above error.

if I add both bundles (AcmeDemoBundle, AcmeUserBundle) under both namespaces (default, user_db), like this

    orm:
        default_entity_manager: default
        entity_managers:
            default:
                connection: default
                mappings:
                    AcmeDemoBundle:  ~
                    AcmeUserBundle: ~
            user_db:
                connection: user_db
                mappings:
                    AcmeUserBundle: ~
                    AcmeDemoBundle:  ~

It tries to create or find both tables (fos_user and comment) in both databases. if we create table manually, It also creates problem at the time of loading fixtures as it tries to find tables in both databases to truncate them.

[stof]

Well, if you use different entity managers for both entities, you indeed cannot map associations between them. But if you use separate DB connections, you won't be able to map relations either (unless both connections are pointing to the same DB, in which case things may work)

[CharlyPoppins]

Hi,

I tried to enable auto_mapping on multiple EM but I've got this error :

You cannot enable "auto_mapping" on more than one manager at the same time (found in "default" and client1").

Did I miss something in the config.yml ?

doctrine:
    dbal:
        default_connection:   default
        connections:
            default:
                ...
            client1:
                ...
    orm:
        auto_generate_proxy_classes: true
        default_entity_manager: default
        entity_managers:
            default:
                connection: default
                auto_mapping: true
            client1:
                connection: client1
                auto_mapping: true

[stof]

@CharlyPoppins which version of DoctrineBundle are you using ? Use composer show -i doctrine/doctrine-bundle to get it if you use composer

[stof]

And which version of Symfony ? This feature requires Symfony 2.6+

[CharlyPoppins]

Hi, I'm on Symfony 2.8.0 and I'm using Doctrine :

name     : doctrine/doctrine-bundle
descrip. : Symfony DoctrineBundle
keywords : database, dbal, orm, persistence
versions : * dev-master, 1.6.x-dev
type     : symfony-bundle

[xabbuh]

@CharlyPoppins What is the output of composer show -i symfony/symfony and composer show -i symfony/doctrine-bridge?

[CharlyPoppins]

*php composer.phar show -i symfony/symfony *

name     : symfony/symfony
descrip. : The Symfony PHP framework
keywords : framework
versions : * v2.8.0
type     : library
license  : MIT License (MIT) (OSI approved) https://spdx.org/licenses/MIT.html#licenseText
source   : [git] https://github.com/symfony/symfony.git 5615b92cd452cd54f1433a3f53de87c096a1107f
dist     : [zip] https://api.github.com/repos/symfony/symfony/zipball/5615b92cd452cd54f1433a3f53de87c096a1107f 5615b92cd452cd54f1433a3f53de87c096a1107f
names    : symfony/symfony, symfony/asset, symfony/browser-kit, symfony/class-loader, symfony/config, symfony/console, symfony/css-selector, symfony/debug, symfony/debug-bundle, symfony/dependency-injection, symfony/doctrine-bridge, symfony/dom-crawler, symfony/event-dispatcher, symfony/expression-language, symfony/filesystem, symfony/finder, symfony/form, symfony/framework-bundle, symfony/http-foundation, symfony/http-kernel, symfony/intl, symfony/ldap, symfony/locale, symfony/monolog-bridge, symfony/options-resolver, symfony/process, symfony/property-access, symfony/property-info, symfony/proxy-manager-bridge, symfony/routing, symfony/security, symfony/security-bundle, symfony/security-core, symfony/security-csrf, symfony/security-guard, symfony/security-http, symfony/serializer, symfony/stopwatch, symfony/swiftmailer-bridge, symfony/templating, symfony/translation, symfony/twig-bridge, symfony/twig-bundle, symfony/validator, symfony/var-dumper, symfony/web-profiler-bundle, symfony/yaml

autoload
psr-4
Symfony\Bridge\Doctrine\ => src/Symfony/Bridge/Doctrine/
Symfony\Bridge\Monolog\ => src/Symfony/Bridge/Monolog/
Symfony\Bridge\ProxyManager\ => src/Symfony/Bridge/ProxyManager/
Symfony\Bridge\Swiftmailer\ => src/Symfony/Bridge/Swiftmailer/
Symfony\Bridge\Twig\ => src/Symfony/Bridge/Twig/
Symfony\Bundle\ => src/Symfony/Bundle/
Symfony\Component\ => src/Symfony/Component/
classmap
src/Symfony/Component/Intl/Resources/stubs
exclude-from-classmap

requires
doctrine/common ~2.4
php >=5.3.9
psr/log ~1.0
symfony/polyfill-intl-icu ~1.0
symfony/polyfill-mbstring ~1.0
symfony/polyfill-php54 ~1.0
symfony/polyfill-php55 ~1.0
symfony/polyfill-php56 ~1.0
symfony/polyfill-php70 ~1.0
symfony/polyfill-util ~1.0
symfony/security-acl ~2.7
twig/twig ~1.23|~2.0

requires (dev)
doctrine/data-fixtures 1.0.*
doctrine/dbal ~2.4
doctrine/doctrine-bundle ~1.2
doctrine/orm ~2.4,>=2.4.5
egulias/email-validator ~1.2
monolog/monolog ~1.11
ocramius/proxy-manager ~0.4|~1.0
phpdocumentor/reflection ^1.0.7

conflicts
phpdocumentor/reflection <1.0.7

replaces
symfony/asset self.version
symfony/browser-kit self.version
symfony/class-loader self.version
symfony/config self.version
symfony/console self.version
symfony/css-selector self.version
symfony/debug self.version
symfony/debug-bundle self.version
symfony/dependency-injection self.version
symfony/doctrine-bridge self.version
symfony/dom-crawler self.version
symfony/event-dispatcher self.version
symfony/expression-language self.version
symfony/filesystem self.version
symfony/finder self.version
symfony/form self.version
symfony/framework-bundle self.version
symfony/http-foundation self.version
symfony/http-kernel self.version
symfony/intl self.version
symfony/ldap self.version
symfony/locale self.version
symfony/monolog-bridge self.version
symfony/options-resolver self.version
symfony/process self.version
symfony/property-access self.version
symfony/property-info self.version
symfony/proxy-manager-bridge self.version
symfony/routing self.version
symfony/security self.version
symfony/security-bundle self.version
symfony/security-core self.version
symfony/security-csrf self.version
symfony/security-guard self.version
symfony/security-http self.version
symfony/serializer self.version
symfony/stopwatch self.version
symfony/swiftmailer-bridge self.version
symfony/templating self.version
symfony/translation self.version
symfony/twig-bridge self.version
symfony/twig-bundle self.version
symfony/validator self.version
symfony/var-dumper self.version
symfony/web-profiler-bundle self.version
symfony/yaml self.version

With command php composer.phar show -i symfony/doctrine-bridge

[InvalidArgumentException]                 
  Package symfony/doctrine-bridge not found

[xabbuh]

This looks weird to me. Can you debug locally why this code gets executed?

[CharlyPoppins]

I replace my DoctrineExtension.php by the one you linked but I've still the same exception.

[xabbuh]

Well, I meant that you try to figure out why that code is executed on your system thus triggering the error message (which should not happen with the version of the packages you have installed).

[Padam87]

The exception is not thrown from https://github.com/doctrine/DoctrineBundle/blob/master/DependencyInjection/DoctrineExtension.php#L358-L363.

fixManagersAutoMappings calls validateAutoMapping, which throws the exception at https://github.com/symfony/doctrine-bridge/blob/master/DependencyInjection/AbstractDoctrineExtension.php#L485