Regression test failure

[bookmoons]

Recently regression tests have been failing. It seems to happen after a fuzzing run ends with Failed. I've seen in 3 separate orgs now.

Here's an example.

https://travis-ci.org/bookmoons/jaeger/jobs/582957346

running regression...
INFO: Seed: 46614417
INFO: 65536 Extra Counters
Not a directory: seed_dir/empty; exiting
2019/09/10 01:42:01 Waiting for container
2019/09/10 01:42:01 fuzzer exited with 1
The command "./scripts/travis/fuzzit.sh local-regression" exited with 1.

That "not a directory" line is present in all failures, always with the same path.

Not a directory: seed_dir/empty; exiting

Deleting the target and creating a new one with a different name resolves the error. The "not a directory" line disappears and the build succeeds. Recreating with the same name doesn't work, it seems to preserve the settings and whatever state is causing the error.

Status of the last fuzzing job always seems to be Failed. The end of the worker log always has a Killed line.

https://app.fuzzit.dev/orgs/jaeger/targets/deserialize-zipkin/XrYqKhDNLJ7K9jO1bwCK

FUZZER: #29677	NEW    ft: 1096 corp: 76/688b lim: 14 exec/s: 899 rss: 378Mb L: 14/14 MS: 3 ChangeBit-CrossOver-CrossOver-
FUZZER: #29684	NEW    ft: 1099 corp: 77/702b lim: 14 exec/s: 899 rss: 378Mb L: 14/14 MS: 2 ChangeByte-ChangeBinInt-
FUZZER: #29698	NEW    ft: 1102 corp: 78/716b lim: 14 exec/s: 899 rss: 378Mb L: 14/14 MS: 4 EraseBytes-ChangeBinInt-EraseBytes-CrossOver-
FUZZER: #29710	NEW    ft: 1114 corp: 79/730b lim: 14 exec/s: 873 rss: 720Mb L: 14/14 MS: 2 InsertRepeatedBytes-CrossOver-
FUZZER: #29712	REDUCE ft: 1117 corp: 80/744b lim: 14 exec/s: 873 rss: 720Mb L: 14/14 MS: 2 CMP-ChangeBinInt- DE: "\xff\xff\xff\xff"-
Killed

[bookmoons]

Thinking about where it's happened, seems like it's only been Rust packages.

[bookmoons]

No, that's wrong. 1 is Rust, 1 is Go.

[bookmoons]

The error seems to reappear after doing that target recreation. The new job eventually shows status Failed with a Killed line in the log, and the error starts happening in fuzz regression again.

[bookmoons]

There's another example from cranelift. The fuzz regression phase fails with the same error.

https://dev.azure.com/bookmoons/cranelift/_build/results?buildId=32

This one doesn't seem to have the Failed status. The erroring target parse-ir was canceled.

[yevgenypats]

I think an update of the CLI is needed to https://github.com/fuzzitdev/fuzzit/releases/tag/v2.4.44

[bookmoons]

Upgrading the client fixed this everywhere. Thanks a lot @yevgenypats.