[RacketCS] intermittent build failure in Cover

[florence]

Ive been seeing this intermittent build failure with Cover and RacketCS. I haven't been able to reproduce it locally, so I haven't managed to reduce it at all. Every now and then an arbitrary test fails with vector-ref: #<evt> is not a vector. The stack tracks seems to contain condition->exn at the top, but the rest of the stack trace (and which test actually fails) keeps changing.

The failures can be found here:

https://travis-ci.org/florence/cover/jobs/533597539#L1003
https://travis-ci.org/florence/cover/jobs/531008063#L733
https://travis-ci.org/florence/cover/jobs/535576209#L1084

[florence]

Just swinging back here to say this failure happened again:

https://travis-ci.org/florence/cover/jobs/594908100#L466

Its so intermittent that I haven't been able to reproduce it locally enough to narrow it down to a more minimal example, sorry :(

[pmatos]

Just a quick idea. Are you able to run one of the single failing tests locally, let's say 10000 times? Does it ever fail? If you don't have time, I am happy to give it a go, if you tell me how to run one of the tests you seen failing here.

[florence]

Unfortunately that doesn't reproduce it locally.

FWIW, based on the previous failures the easiest way to try this is to run just run:

racocs test cover/cover/tests/do-eval.rkt

Which is the smallest of the tests that I've seen fail.

One possible difference is that I'm on Mac. I'll try spinning up an Ubuntu VM later this week to see if I can reproduce it there, since that's closer to the environment on travis.

[florence]

I'm noticing now that the most recent failure has a source location and that source location is in a file under coverage. Based on this I would guess that the actual error here is that sometimes sync on a log-receiver? gives back structure rather than a 4-vector (since thats the only vector-ref that cover inserts into user code).

[florence]

Based on that hunch I ran this program:

repeat.rkt:

#lang racket
(begin-for-syntax
  (define s1 (make-log-receiver (current-logger) 'info 'receive))
  (thread
   (lambda ()
     (vector-ref (sync s1) 0)
     (log-message (current-logger) 'info 'send "a" 1)))
  
  (define s (make-log-receiver (current-logger) 'info 'send))
  (log-message (current-logger) 'info 'receive "a" 1)
  (vector-ref (sync s) 0))

overnight, and the error occurred after a few hours. But it only happenned when running racocs test repeat.rkt, not when running racketcs repeat.rkt.

[mflatt]

Thank you! This example helped me track down the problem and create a stress test (that's simpler and faster than this example).