Allowed memory size of XXX bytes exhausted

[infectormp]

Get some memory size error when scan code. I try it with different memory_limit.

Allowed memory size of 268435456 bytes exhausted (tried to allocate 528384 bytes) in phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/TokenCollection.php on line 118

Allowed memory size of 536870912 bytes exhausted (tried to allocate 262144 bytes) in phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/TokenCollectionCompatibility.php on line 56

Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/TokenCollectionCompatibility.php on line 34

Want suggest add option to ignore files by size --max-size

[YuraLukashik]

@infectormp it's the very huge amount of memory, so i don't think the reason is a huge size of a file. May be there is a bug with recursion or something like it.
Could you provide the file causing the failure?
Could you provide your full terminal output?

[infectormp]

this is full output
98/184 [==============>-------------] 53%PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/TokenCollectionCompatibility.php on line 34

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/TokenCollectionCompatibility.php on line 34

Look like tcpdf_5.0.002\fonts cause this error. But when i try check files one by one all work fine.

[YuraLukashik]

@infectormp which php version do you use? try php 7.0.0, it might provide us some useful info

[Halleck45]

PhpMetrics use the token_get_all() native PHP function to parse code. We cannot change the way this function parse code and use memory. But we can try to optimize the PhpMetrics's code...

Meanwhile, you can:

• avoid to parse tcpdf files (tcpdf is a library, if you want to get metrics about your code, avoid to parse vendors and libraries)
• increase your memory_limit.

[infectormp]

@YuraLukashik i try PHP7 with same result.
@Halleck45 i increase memory_limit to 512M.

[infectormp]

When i test PhpCodeFixer i get similar error. It was infinite loop during file scanning.
Current error look very similar because does not appear to single file.

[YuraLukashik]

@infectormp I suggest you to debug it with xdebug, ping me in skype(YuraLukashik) and I'll help you :)

[infectormp]

additional info with xdebug

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/TokenCollectionCompatibility.php on line 34

Call Stack:
    0.0001     220992   1. {main}() /usr/local/bin/phpmetrics:0
    0.0323    2784496   2. Symfony\Component\Console\Application->run() /usr/local/bin/phpmetrics:17
    0.0350    3138416   3. Symfony\Component\Console\Application->doRun() phar:///usr/local/bin/phpmetrics/vendor/symfony/console/Application.php:123
    0.0351    3139064   4. Symfony\Component\Console\Application->doRunCommand() phar:///usr/local/bin/phpmetrics/vendor/symfony/console/Application.php:192
    0.0352    3139640   5. Symfony\Component\Console\Command\Command->run() phar:///usr/local/bin/phpmetrics/vendor/symfony/console/Application.php:841
    0.0355    3145248   6. Hal\Application\Command\RunMetricsCommand->execute() phar:///usr/local/bin/phpmetrics/vendor/symfony/console/Command/Command.php:259
    0.0461    4305392   7. Hal\Application\Command\Job\Queue->execute() phar:///usr/local/bin/phpmetrics/src/Hal/Application/Command/RunMetricsCommand.php:125
    0.0461    4306320   8. Hal\Application\Command\Job\DoAnalyze->execute() phar:///usr/local/bin/phpmetrics/src/Hal/Application/Command/Job/Queue.php:51
   85.7652  387506016   9. Hal\Application\Command\Job\Analyze\FileAnalyzer->execute() phar:///usr/local/bin/phpmetrics/src/Hal/Application/Command/Job/DoAnalyze.php:132
   88.7531  487496912  10. Hal\Metrics\Complexity\Text\Length\Loc->calculate() phar:///usr/local/bin/phpmetrics/src/Hal/Application/Command/Job/Analyze/FileAnalyzer.php:124
   88.7531  487497056  11. Hal\Component\Token\Tokenizer->tokenize() phar:///usr/local/bin/phpmetrics/src/Hal/Metrics/Complexity/Text/Length/Loc.php:47
   88.7531  487497184  12. Hal\Component\Token\TokenCollection->__construct() phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/Tokenizer.php:45
   89.4853  520740536  13. Hal\Component\Token\TokenCollectionCompatibility->decorate() phar:///usr/local/bin/phpmetrics/src/Hal/Component/Token/TokenCollection.php:38

[pavarnos]

same. I doubled the ram to 1024M (for a project with 1400 files) and it completed ok. but there were (possibly unrelated) javascript problems with the generated output. Will post a separate bug report (#189)

[theofidry]

Maybe a little profiling could help: blackfire profile.

FYI once installed it just requires to do blackfire run phpmetrics ... and here you go.

[Halleck45]

I don't know how to help you on this issue...

The main problem with RAM used by PhpMetrics concerns the token_get_all() function... I've tried to optimize the use of this function, but without success (I've introduced bugs).

Sadly, today my only recommendation is to increase the RAM :/

@pavarnos , do not hesitate to open new issue.

I close this issue. Feel free to reopen it if you think that is useful.

[pavarnos]

I've spent a bit of time reading the code (which it beautiful btw!)

src/Hal/Application/Command/RunMetricsCommand.php does

        gc_disable();
        $queue->execute($collection, $aggregatedResults);
        gc_enable();

Is this for performance?
I'm not sure what you have tried... but i wonder if this is the cause of the large memory consumption? Blackfire may tell you in detail...

One possible (naive) solution is to make it optional. Eg just before the end of QueueFactory::factory() decorate the queue,

   if ($this->config->fastModeSelected()) {
        $queue = new FastQueue();
   } else {
        $queue = new Queue();
   }

where

class FastQueue extends Queue {
    public function execute(ResultCollection $collection, ResultCollection $aggregatedResults) {
        foreach($this->jobs as $job) {
            gc_disable();
            $job->execute($collection, $aggregatedResults);
            gc_collect_cycles();
            gc_enable();
        }
    }