Memory Issue

[renegare]

Probably not an issue but! ... I have 9000 documents I am sending through the this bundle.

My Memory is set to 256M but would like to investigate if there could be an option set the memory limit that the bundle will be aware of and keep with (unset objects no longer needed). Is this an issue or a request ... you decide.

See below error ouput:

PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 76 bytes) in /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php on line 2386
PHP Stack trace:
PHP 1. {main}() /Project/code/client_symfony/symfony/app/console:0
PHP 2. Symfony\Component\Console\Application->run() /Project/code/client_symfony/symfony/app/console:22
PHP 3. Symfony\Bundle\FrameworkBundle\Console\Application->doRun() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:106
PHP 4. Symfony\Component\Console\Application->doRun() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:78
PHP 5. Symfony\Component\Console\Command\Command->run() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:193
PHP 6. FOQ\ElasticaBundle\Command\PopulateCommand->execute() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:239
PHP 7. FOQ\ElasticaBundle\Command\PopulateCommand->populateIndex() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Command/PopulateCommand.php:79
PHP 8. FOQ\ElasticaBundle\Doctrine\AbstractProvider->populate() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Command/PopulateCommand.php:105
PHP 9. FOQ\ElasticaBundle\Doctrine\ORM\Provider->fetchSlice() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Doctrine/AbstractProvider.php:44
PHP 10. Doctrine\ORM\AbstractQuery->getResult() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Doctrine/ORM/Provider.php:46
PHP 11. Doctrine\ORM\AbstractQuery->execute() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php:538
PHP 12. Doctrine\ORM\Internal\Hydration\AbstractHydrator->hydrateAll() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php:747
PHP 13. Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateAllData() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php:111
PHP 14. Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateRowData() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php:149
PHP 15. Doctrine\ORM\Internal\Hydration\ObjectHydrator->_getEntity() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php:393
PHP 16. Doctrine\ORM\UnitOfWork->createEntity() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php:245

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 76 bytes) in /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php on line 2386

Call Stack:
0.0002 232752 1. {main}() /Project/code/client_symfony/symfony/app/console:0
0.0122 1601280 2. Symfony\Component\Console\Application->run() /Project/code/client_symfony/symfony/app/console:22
0.0142 1795640 3. Symfony\Bundle\FrameworkBundle\Console\Application->doRun() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:106
0.0660 6231832 4. Symfony\Component\Console\Application->doRun() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:78
0.0674 6233568 5. Symfony\Component\Console\Command\Command->run() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:193
0.0696 6451184 6. FOQ\ElasticaBundle\Command\PopulateCommand->execute() /Project/code/client_symfony/symfony/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:239
0.0696 6452136 7. FOQ\ElasticaBundle\Command\PopulateCommand->populateIndex() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Command/PopulateCommand.php:79
0.2031 6910296 8. FOQ\ElasticaBundle\Doctrine\AbstractProvider->populate() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Command/PopulateCommand.php:105
176.7703 263556120 9. FOQ\ElasticaBundle\Doctrine\ORM\Provider->fetchSlice() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Doctrine/AbstractProvider.php:44
176.7703 263556496 10. Doctrine\ORM\AbstractQuery->getResult() /Project/code/client_symfony/symfony/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Doctrine/ORM/Provider.php:46
176.7703 263556592 11. Doctrine\ORM\AbstractQuery->execute() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php:538
178.6325 265793096 12. Doctrine\ORM\Internal\Hydration\AbstractHydrator->hydrateAll() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php:747
178.6325 265792336 13. Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateAllData() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php:111
178.7052 268184144 14. Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateRowData() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php:149
178.7063 268228664 15. Doctrine\ORM\Internal\Hydration\ObjectHydrator->_getEntity() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php:393
178.7063 268228584 16. Doctrine\ORM\UnitOfWork->createEntity() /Project/code/client_symfony/symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php:245

In the mean time ... increase my memory should suffice. If not I'll be making this a rel bug!

Great bundle though +1!

[richardmiller-zz]

Try using the --no-debug option when running the populate command that should reduce memory usage to some extent. You could also try reducing the batch size used to be lower than the default 100 (this is the number of objects that are retrieved at a time from the database for indexing, so lowering should use less memory at a time):

persistence:
  driver: orm
  model: Application\UserBundle\Entity\User
  provider:
    batch_size: 100

[renegare]

Oh wonderful.

Cool, thanks.

M.

On Tue, Aug 14, 2012 at 9:50 AM, Richard Miller notifications@github.comwrote:

Try using the --no-debug option when running the populate command that
should reduce memory usage to some extent. You could also try reducing the
batch size used to be lower than the default 100 (this is the number of
objects that are retrieved at a time from the database for indexing, so
lowering should use less memory at a time):

persistence:
driver: orm
model: Application\UserBundle\Entity\User
provider:
batch_size: 100

—
Reply to this email directly or view it on GitHubhttps://github.com/Exercise/FOQElasticaBundle/issues/153#issuecomment-7719980.