Intermitent problems with some builds

[JJ]

The problem

From time to time, builds fail with error of this type

 xt/examples-compilation.t STDERR: Use of uninitialized value $repo-id of type Any in string context.
# Methods .^name, .perl, .gist, or .say can be used to stringi
# xt/examples-compilation.t STDERR: fy it to something meaningful.
# 
t/pod-valid.t ........ 340/347 # xt/examples-compilation.t STDERR:   in any statement_control at /tmp/whateverable/rakudo-moar/a138dcf50df2b2bf571db96dc14d2ea3e81e8522/share/nqp/lib/Perl6/Grammar.moarvm line 1
# Use of uninitialized value $repo-id of type Any in string context.
# Methods .^name, .perl, .gist, or .say can be 
# xt/examples-compilation.t STDERR: used to stringify it to something meaningful.
#   in any statement_control at /tmp/whateverable/rakudo-moar/a138dcf50df2b2bf571db96dc14d2ea3e81e8522/share/nqp/lib/Perl6/Grammar.moarvm line 1
# 
# Failed test 'xt/examples-compilation.t has clean POD6'

I happens in all tests, in all binaries, be it Docker file, packages or HEAD binary.

Suggestions

Since there is another build running in parallel, it's not too important, but the fact that it consistently fails in the same place, although intermitently, might indicate some problem of some sort there.

[AlexDaniel]

Hmmm…

[JJ]

It's really intermitent. No actual changes to code or pods, and how it passes

[JJ]

I times out sometimes... https://travis-ci.org/perl6/doc/jobs/370171158

[JJ]

It can be any of the two environments, although it happens more often in the binary version. It happens ~ test 337 in pod-valid.t, that is, towards the end... Could it be a memory issue with MoarVM?

[JJ]

@AlexDaniel could it be a memory issue?

[JJ]

$repo-id is definitely part of CompUnit, possibly PrecompilationRepository. So could this be fixed by #1952?

[AlexDaniel]

Which file is it failing on? Maybe that single test in a loop and see what happens.

[JJ]

Several files, as you can see in the screenshot.

[AlexDaniel]

OK well, take any single one of them and run it in a loop. If you can reproduce it alone like that, then please rakudobug it.

[JJ]

Since it's intermitent, not even running it on the actual repo will always reproduce the bug. It's also got to do with precompilation, since the variables mentioned are in that area. I'm not sure what will be easier: to reproduce the bug or just make the .pod6 be precompiled once and for all, instead of precompiling them all over again. My latest issues are going in that direction, indeed...

[JJ]

They seem to happen in shippable too. Same kind of error, same number of files.

[JJ]

It's less frequent, but still happens. Sometimes it just hangs, too.

[JJ]

Come to think of it, we should just eliminate testing the .t files. They are not part of the documentation, and if they are, well, there is all the stuff in ./lib that should be tested too. Invalid pod will simply be invalid Perl 6 if I'm not wrong, so the .t will simply yleld an error.

[JJ]

Right, we will have something like this:

===SORRY!=== Error while compiling /home/jmerelo/Code/forks/perl6/doc/t/02-pod-valid.t
Preceding context expects a term, but found infix = instead.
Did you make a mistake in Pod syntax?
at /home/jmerelo/Code/forks/perl6/doc/t/02-pod-valid.t:52

if there's an error somewhere in the Pod. There will be warnings if there's an error of another type, so unless it's really essential to test the pod content in the tests, I would eliminate them from this test. It will make tests faster, too.

[coke]

I would much rather we fix the underlying issue than not run the test.

[JJ]

The underlying issue is probably somewhere deep in Rakudo, either on the threading or the CompUnit code. And I'm not saying we don't run the test; I say we keep running it where it makes sense, in the .pod6 files.

[JJ]

We could make different tests for the tests themselves, if testing the tests is an issue. Not only valid pod, but also some other stuff.

[JJ]

Now it's showing up in installation of modules too

[JJ]

Now this is weird:

An operation first awaited:
  in sub process-pod-dir at htmlify.p6 line 269
  in sub MAIN at htmlify.p6 line 200
  in block <unit> at htmlify.p6 line 1006
Died with the exception:
    ctxlexpad needs an MVMContext
      in sub extract-pod at /home/travis/build/perl6/doc/lib/Pod/Convenience.pm6 (Pod::Convenience) line 131
      in block  at htmlify.p6 line 264

Seen here

[AlexDaniel]

Rakudobug: rakudo/rakudo#1810

[JJ]

Errors came back. Now in a completely different test.

[JJ]

There have been no errors for some time now: check the Travis history Could this have fixed itself?

[JJ]

Well, no, they are back

[JJ]

The intermitent error now says there's an error in r.moarvm: https://travis-ci.org/perl6/doc/jobs/498560848

[JJ]

This is the last time it happened, 4 months ago. Since we don't really know what causes it, we'll wait for a while before closing it. Besides, it's happening also in shippable

[nkh]

I get this problem quite often.

it's difficult to reproduce but it seems to happen often when I switch branches.

I get error:
Cannot iterate over a VMNull type object
in method get_dump_lines at ....

re-running the test always succeeds.

Another way to always succeed is to remove the .precompiled directory so it may be cache related.

This is Rakudo version 2019.03.1-616-g6565810f5 built on MoarVM version 2019.05-53-gcce71cdbf
implementing Perl 6.d.

[coke]

May be related to #3466

[JJ]

Don't think so. Comes from way before we even started to use Pod::Cache.