Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory consumption and crashes when running t/spec/S07-slip/slip.t #2606

Closed
dogbert17 opened this Issue Jan 13, 2019 · 5 comments

Comments

Projects
None yet
4 participants
@dogbert17
Copy link
Contributor

dogbert17 commented Jan 13, 2019

The Problem

The memory consumption when running

MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1 /usr/bin/time ./perl6  t/spec/S07-slip/slip.t

and

MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1 /usr/bin/time make  t/spec/S07-slip/slip.t

differs markedly on my system. On my 32 bit Linux running the first variant reports

1..23
ok 1 - simple | middle
ok 2 - simple | left
ok 3 - simple | right
ok 4 - simple slip middle
ok 5 - simple slip left
ok 6 - simple slip right
ok 7 - simple Slip coercion middle
ok 8 - simple Slip coercion left
ok 9 - simple Slip coercion right
ok 10 - simple Slip coercion middle
ok 11 - simple Slip coercion left
ok 12 - simple Slip coercion right
ok 13 - simple slip listop middle
ok 14 - simple slip listop left
ok 15 - simple slip listop right
ok 16 - slip flat listops middle
ok 17 - slip flat listops left
ok 18 - slip flat listops right
ok 19 - .Slip can slip a long lazy list
ok 20 - Slip() can slip a one-arg long lazy list
ok 21 - slip works with one-arg long lazy
ok 22 - prefix:<|> works with one-arg long lazy
ok 23 - can .perl.EVAL roundtrip an itemized slip
10.98user 0.17system 0:11.13elapsed 100%CPU (0avgtext+0avgdata 156748maxresident)k
0inputs+0outputs (0major+32553minor)pagefaults 0swaps

while the second variant crashes with the following output

/usr/bin/perl tools/build/check-nqp-version.pl /home/dogbert/repos/rakudo/install/bin/nqp-m
rm -f -- perl6
cp -- perl6-m perl6
chmod -- 755 perl6
/usr/bin/perl t/harness5 --fudge --moar --keep-exit-code --verbosity=1 t/spec/S07-slip/slip.t

ok 1 - simple | middle
ok 2 - simple | left
ok 3 - simple | right
ok 4 - simple slip middle
ok 5 - simple slip left
ok 6 - simple slip right
ok 7 - simple Slip coercion middle
ok 8 - simple Slip coercion left
ok 9 - simple Slip coercion right
ok 10 - simple Slip coercion middle
ok 11 - simple Slip coercion left
ok 12 - simple Slip coercion right
ok 13 - simple slip listop middle
ok 14 - simple slip listop left
ok 15 - simple slip listop right
ok 16 - slip flat listops middle
ok 17 - slip flat listops left
ok 18 - slip flat listops right
Dubious, test returned 1 (wstat 256, 0x100)
Failed 5/23 subtests 

Test Summary Report
-------------------
t/spec/S07-slip/slip.t (Wstat: 256 Tests: 18 Failed: 0)
  Non-zero exit status: 1
  Parse errors: Bad plan.  You planned 23 tests but ran 18.
Files=1, Tests=18, 22 wallclock secs ( 0.03 usr  0.01 sys + 10.81 cusr 11.13 csys = 21.98 CPU)
Result: FAIL
make: *** [t/spec/S07-slip/slip.t] Error 1
Command exited with non-zero status 2
10.98user 11.17system 0:22.10elapsed 100%CPU (0avgtext+0avgdata 2994652maxresident)k
0inputs+8outputs (0major+10027340minor)pagefaults 0swaps

As can be seen the maxresident numbers are vastly different and this leads to an out of memory failure.

On 64 bit systems the second variant doesn't crash, instead it hangs bringing my vm to a crawl.

With the help of AlexDaniel's++ bisectable wizardry a suspect commit has been identified, i.e. 3a68cc9

Expected Behavior

Both variants, see above, should behave more or less the same wrt to memory consumption, the time it takes to run them and the result of the run

Actual Behavior

See above

Steps to Reproduce

See above

Environment

  • Operating system: Ubuntu 14.04.3 LTS
  • Compiler version (perl6 -v): This is Rakudo version 2018.12-206-g7be075eb5 built on MoarVM version 2018.12-29-g1ff55bf1c implementing Perl 6.d.
@AlexDaniel

This comment has been minimized.

Copy link
Member

AlexDaniel commented Jan 13, 2019

With the help of AlexDaniel's++ bisectable wizardry a suspect commit has been identified, i.e. 3a68cc9

Note that this is technically when the issue became visible, see proof. But the commit itself is likely innocent.

@jnthn

This comment has been minimized.

Copy link
Member

jnthn commented Jan 13, 2019

A couple of questions:

  1. Does this happen with a standard configuration (e.g. no NODELAY)?
  2. Does setting PERL6LIB=lib cause the high memory use? I believe the harness sets that.
@MasterDuke17

This comment has been minimized.

Copy link
Contributor

MasterDuke17 commented Jan 14, 2019

  1. It happens for me with both NO_DELAY and BLOCKING, but not with either one of those alone.

  2. Setting PERL6LIB=lib does cause high memory use in the non-make variant. Both NO_DELAY and BLOCKING have to be set, behavior is normal otherwise.

This is Rakudo version 2018.12-206-g7be075eb5 built on MoarVM version 2018.12-26-g7adb8090d
Linux alexandria 4.19.9-arch1-1-ARCH #1 SMP PREEMPT Fri Dec 14 00:58:26 UTC 2018 x86_64 GNU/Linux

@dogbert17

This comment has been minimized.

Copy link
Contributor Author

dogbert17 commented Jan 15, 2019

Setting MVM_SPESH_INLINE_DISABLE=1 seems to resolve the problem, i.e.

MVM_SPESH_NODELAY=1 MVM_SPESH_BLOCKING=1 MVM_SPESH_INLINE_DISABLE=1 PERL6LIB=lib ...
@jnthn

This comment has been minimized.

Copy link
Member

jnthn commented Jan 17, 2019

First of all, I've put in a general change to prevent runaway memory usage if this kind of problem occurs.

Then, I also tracked down the root cause of the problem, which was a bug that did indeed affect inlining of spesh plugins that gained extra guards in the guard set after specialization. I fixed it also.

@jnthn jnthn closed this Jan 17, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.