Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

phpDoc runs out of memory? #637

Closed
KariTrace opened this Issue · 11 comments

2 participants

@KariTrace

So I've moved on to a larger organization recently. They use a home grown framework that is used to support a number of client sites.

I can Document one site at a time and it phpDoc turns out some very nice documentation.

However, when I try to run Docs on the entire framework I run into this:

Storing structure.xml in "C:\phpDoc\Testing\structure.xml" .. PHP Fatal error: Allowed memory size of 134217728 bytes exhausted
(tried to allocate 58926488 bytes) in C:\php5\PEAR\phpDocumentor\src\phpDocumentor\Command\Project\ParseCommand.php on line 274

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 58926488 bytes) in C:\php5\PEAR\phpDocumentor\s
rc\phpDocumentor\Command\Project\ParseCommand.php on line 274

Something just shy of 4000 files total. Who knows how many objects and classes...

Any resolution for this issue?

@mvriel
Owner

128mb of memory is unfortunately not enough to run most big projects; it is a restriction we are sometimes bound to.
In this case I can see that phpDocumentor tried to claim 60mb at once; this probably means that it tried to use the token_get_all function on a very large file (more than 10 to 30.000 lines of code).

This is a most unfortunate answer but the only thing I can tell you is to raise your memory limit in the cli's php.ini to a value of 512mb or even 1GB. Especially regarding the tokenizer of PHP is this an issue we cannot resolve.

@KariTrace

Got closer. But of course new issues:

Storing structure.xml in "C:\phpDoc\W1\structure.xml" .. OK
Initializing transformer ..PHP Warning:  DOMDocument::load(): Input is not proper UTF-8, indicate encoding !
Bytes: 0xFF 0xD8 0xFF 0xE0 in file:///C:/phpDoc/W1/structure.xml, line: 449378 in C:\php5\PEAR\phpDocumentor\src\phpDocumentor\T
ransformer\Transformer.php on line 156

Warning: DOMDocument::load(): Input is not proper UTF-8, indicate encoding !
Bytes: 0xFF 0xD8 0xFF 0xE0 in file:///C:/phpDoc/W1/structure.xml, line: 449378 in C:\php5\PEAR\phpDocumentor\src\phpDocumentor\T
ransformer\Transformer.php on line 156
PHP Warning:  DOMDocument::load(): xmlSAX2Characters: huge text node: out of memory in file:///C:/phpDoc/W1/structure.xml, line:
 801562 in C:\php5\PEAR\phpDocumentor\src\phpDocumentor\Transformer\Transformer.php on line 156

Warning: DOMDocument::load(): xmlSAX2Characters: huge text node: out of memory in file:///C:/phpDoc/W1/structure.xml, line: 8015
62 in C:\php5\PEAR\phpDocumentor\src\phpDocumentor\Transformer\Transformer.php on line 156
PHP Warning:  DOMDocument::load(): Extra content at the end of the document in file:///C:/phpDoc/W1/structure.xml, line: 801562
in C:\php5\PEAR\phpDocumentor\src\phpDocumentor\Transformer\Transformer.php on line 156

Warning: DOMDocument::load(): Extra content at the end of the document in file:///C:/phpDoc/W1/structure.xml, line: 801562 in C:
\php5\PEAR\phpDocumentor\src\phpDocumentor\Transformer\Transformer.php on line 156
 OK
Processing behaviours .. OK
Executing transformations
  5 [===========>----------------]


  [phpDocumentor\Plugin\Core\Exception]
  Specified DOMDocument lacks documentElement, cannot transform



project:run [-t|--target[="..."]] [-f|--filename[="..."]] [-d|--directory[="..."]] [-e|--extensions[="..."]] [-i|--ignore[="..."
]] [--ignore-tags[="..."]] [--hidden] [--ignore-symlinks] [-m|--markers[="..."]] [--title[="..."]] [--force] [--validate] [--vis
ibility[="..."]] [--defaultpackagename[="..."]] [--sourcecode] [-p|--progressbar] [--template[="..."]] [--parseprivate] [-c|--co
nfig[="..."]]

...

The referenced structure.xml weighs in at ~57.5Mb...

@mvriel
Owner

It would appear that you have reached crossed a memory limit of libxml; whether the loading of that node exceeds your physical memory (not the one set in php.ini!) or libxml crashes; I do not know.

Thus far I have not done any testing with a file this big; it is possible for you to generate the documentation in parts instead of in whole? (this sucks, I know).

Our test and performance scenarios should be expanded to try and take this into account; can you please run phploc (https://github.com/sebastianbergmann/phploc) on your project and post that here, or mail it to me?
This will help us to attempt to recreate (or find an OSS) project which we can use as testcase.

@KariTrace

I can do this Monday when I'm back in the office. Whats the process for doing it in parts?

@mvriel
Owner

You could use the -d argument to select a series of folders that make sense to you as a module. Unfortunately phpDocumentor cannot link documentations together; that would have been cool

@KariTrace

phploc fails for me locally. Submitted a help request with the phploc guys. I'll get you the info as soon as possible.

@KariTrace

Got phploc working. Here's the dump from it:

phploc 1.7.1 by Sebastian Bergmann.

Directories: 1085
Files: 3994

Lines of Code (LOC): 1116504
Cyclomatic Complexity / Lines of Code: 0.13
Comment Lines of Code (CLOC): 350760
Non-Comment Lines of Code (NCLOC): 765744

Namespaces: 0
Interfaces: 34
Traits: 0
Classes: 3709
Abstract: 151 (4.07%)
Concrete: 3558 (95.93%)
Average Class Length (NCLOC): 205
Methods: 40551
Scope:
Non-Static: 39018 (96.22%)
Static: 1533 (3.78%)
Visibility:
Public: 32551 (80.27%)
Non-Public: 8000 (19.73%)
Average Method Length (NCLOC): 18
Cyclomatic Complexity / Number of Methods: 3.32

Anonymous Functions: 148
Functions: 661

Constants: 6786
Global constants: 325
Class constants: 6461

@KariTrace

Any movement on this one guys/gals?

@mvriel
Owner

I've been trying to reproduce the issue and with a big enough dataset I went out of memory on the inheritance; your application probably doesn't have deep inheritance levels so that may be why it continues somewhat longer perhaps.

What you can try is to disable inheritance by commenting this line:
https://github.com/phpDocumentor/phpDocumentor2/blob/develop/src/phpDocumentor/Plugin/Core/Listener.php#L48

This will reduce the amount of memory used by phpDocumentor and lixbml. Beyond that it is all about improving performance and memory usage; which takes quite some time to do

@KariTrace

Hay, thanks for the assistance. I'll give that a try and let you know what happens.

@mvriel
Owner

Haven't head the results for this one yet, if there is still an issue: let me know and I'll reopen the issue

@mvriel mvriel closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.