-deps -unittest causes cataclysmic memory usage

[dlangBugzillaToGithub]

FeepingCreature (@FeepingCreature) reported this on 2018-05-08T11:48:08Z

Transferred from https://issues.dlang.org/show_bug.cgi?id=18843

CC List

• Seb

Description

In case you think I'm exaggerating:

echo 'import std.stdio; void main() { writeln("Hello World"); }' > helloworld.d
/usr/bin/time -v dmd -deps -unittest helloworld.d
[...]
        User time (seconds): 27.24
[...]
        Maximum resident set size (kbytes): 11550624

That's 11 gigabytes rss for Hello World.

The background issue is that -deps -unittest forces DMD to recurse into Phobos unittests and all their imports, eventually semantically evaluating every single unittest in Phobos.

Quickfur had a good proposal in https://github.com/dlang/phobos/pull/6159 , that may generalize to -deps: only consider files that are explicitly listed on the command line. That may help.

[dlangBugzillaToGithub]

greensunny12 commented on 2018-05-08T12:13:27Z

Please see and help out on https://github.com/dlang/dmd/pull/8124

[dlangBugzillaToGithub]

default_357-line (@FeepingCreature) commented on 2024-03-27T13:01:25Z

Note: This has gotten a lot better at some point, great job! But running with `-deps` still brings the time for a simple Hello World up from 250ms wallclock to 440ms wallclock. It is thus still unusable, as its whole point would be speeding up compiles, and as it stands I can get the import graph of projects a lot faster by just grepping for `import`.

[dlangBugzillaToGithub]

default_357-line (@FeepingCreature) commented on 2024-03-27T13:11:41Z

Actually, I just tried this on a bit bigger project. And if there's a lot of imports, it looks more like 1s (no -deps=) vs 5s (with -deps=). Looks like it's mostly Semantic3OnDependencies, which makes sense if `-deps` forces compilation so it can look inside for more imports. So I'd still need a way to limit the scope of `-deps` from the commandline.