New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
emacs eats up all my RAM while processing imports #499
Comments
Curiously, in |
Any chance you could collect a memory profile? |
emacs is unresponsive once the RAM usage skyrockets and I have to send it a SIGKILL. So, I can't run any more |
I'm happy to throw other debuggers at it though if you know anything that could be useful.^^ |
Any chance you could collect a memory profile? `M-x profiler-start`, select
`mem`, open and reproduce the issue, then `M-x profiler-report` and `M-x
profiler-report-write-profile`
I just want to clarify that this is not really a memory profile.
It's more like a normal CPU profile but where memory allocation is used
as a kind of time. It was originally introduce the compensate the fact
that the timer-based sampling was not implemented on all systems, so the
allocation-based "time" was used as a poor man's approximation.
We keep it because it can occasionally give relevant info, but don't
read too much into it.
Stefan
|
Ralf,
does this happen with the default Proof General customization? Do
you have any auto-compilation features enabled?
When you open the *Messages* buffer in a second frame, do you see
any pattern of messages scrolling by?
When I try (with auto vos compilation) I immediately see an error
about some library not being in the load path and nothing goes
crazy, but I probably have a completely different Proof General
customization.
Hendrik
|
Good idea! When I remove the "customization" block, the bug goes away. I'll try to bisect which is the "bad" option.
Did you do the submodule init? Also, does it help to run |
So the problem is this line:
When I comment this out, it all works. But of course, I quite like this feature... Here are my other coq/proof-related settings:
|
I am sorry, I cannot reproduce. I did git clone https://github.com/mit-pdos/perennial
cd perennial/
git submodule update --init --recursive
make -j 4 src/program_logic/crash_weakestpre.vo
emacs -q -l ~/src/pg/499.el src/program_logic/crash_weakestpre.v with 499.el containing (setq coq-compile-parallel-in-background t
coq-compile-quick (quote no-quick)
coq-one-command-per-line nil
proof-follow-mode (quote followdown)
proof-next-command-insert-space nil
proof-splash-enable nil
proof-three-window-enable t)
(load-file (expand-file-name "~/src/pg/master/generic/proof-site.el")) In At Please provide more detailed instructions, preferably for a file And please do answer all the questions, in particular the one |
As mentioned before, I am using I tried getting the |
Looks like you did not set the one setting which I identified as problematic above,
Here's the minimal
With that file, the following reproduces the problem, using Coq 8.11.2 from opam:
In emacs, I got to the bottom of the file and do C-c C-Return. This still takes a while to build, and interestingly it's not all files that are broken. I'll walk up the dependency chain. |
Here's something strange:
This doesn't eat up all my RAM, but it tops out at 11% of my 16GB of RAM. When I open the same file in a separate clone of Iris, RAM usage is much lower. For Looks like the further I go up the dependency chain, the smaller RAM usage becomes, but it's still way bigger than I would expect. One thing that is peculiar about the perennial repository is that it's a ton of files that are all built in one makefile. The overall dependency graph is rather big. So maybe this is not PG going into an infinite loop, maybe this is just exposing that PG is wasting a lot of RAM on dependency tracking, and this repository is big enough to actually make that a problem? |
Thanks for your patience, I can now reproduce with |
It will take a while to debug this. What I can see now is that the problem immediately starts with processing the coqdep output of Meanwhile I have another question: Does the makefile also generate coq project files? Because, when I process some imports in a fresh checkout, it immediately fails, because libraries cannot be found. I would guess now, that the load path is incomplete, maybe, because the project files are not there yet? |
With some more debug output the problems start one file earlier at |
Hm, not that I know of.
Looks like the topmost |
Looks like the topmost _CoqProject is generated by the Makefile, yes. Cc @tchajed
OK, thanks. After `make _CoqProject` auto compilation works. But
this does not give any new insights.
|
We do have a weird setup where the dependencies are hooked into a single tree and then we have a custom Makefile that just uses coqdep and has its own compilation rule. One issue this does raise is that there are multiple _CoqProject files, and only the root one should be used (which is what the Makefile does). @RalfJung if your hypothesis is right that it's just the magnitude of the dependencies, you could try disabling compiling the large ones with this: (setq coq-compile-ignored-directories
'("external/iris/.*" "external/stdpp/.*" "external/coqutil/.*")) |
I found the problem: It's the naive way of propagating ancestors upwards in the dependency tree. When I programmed this, I simply appended all ancestors of all dependees - this leads to an exponential duplication of ancestors in tightly coupled dependency graphs. If you want a quick solution, replace (let ((ancestors (get job 'ancestors)) in function (let ((ancestors (and coq-lock-ancestors (get job 'ancestors))) and disable I will replace the lists with a hash, I guess. |
@hendriktews that patch and the config change does it, thanks a lot. :) |
The hash avoids an exponentially growing number of duplicates in the ancestor collection. Fixes ProofGeneral#499
I can confirm that this fixes the problem, thanks a lot :) |
I am having trouble using PG on a particular Coq development, where when I step over the
Require
commands at the top of the file it never completes, instead eating up all of my RAM until the machine starts thrashing. When I killed it the emacs process was at more than 80% of my 16GB RAM. Runningmake
in the same project to build it all withcoqc
works fine.To reproduce, clone https://github.com/mit-pdos/perennial, initialize submodules (
git submodule update --init --recursive
), and then open pretty much any file and step to the end of the buffer in PG. I triedexternal/Goose/github_com/mit_pdos/perennial_examples/replicated_block.v
andsrc/program_logic/crash_weakestpre.v
.This is with latest PG master and emacs 1:26.3+1-2 on Debian testing.
The text was updated successfully, but these errors were encountered: