Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MIRI check is failing on master #879

Closed
alamb opened this issue Oct 29, 2021 · 9 comments · Fixed by #892, #895, #899 or #922
Closed

MIRI check is failing on master #879

alamb opened this issue Oct 29, 2021 · 9 comments · Fixed by #892, #895, #899 or #922
Labels

Comments

@alamb
Copy link
Contributor

alamb commented Oct 29, 2021

Describe the bug
The MIRI check has started to fail on master:

https://github.com/apache/arrow-rs/commits/master

Screen Shot 2021-10-29 at 9 14 17 AM

Example failure: https://github.com/apache/arrow-rs/runs/4023917285?check_suite_focus=true

To Reproduce
Run CI MIRI test on master

Expected behavior
Test should pass

Additional context
The MIRI version was recently updated in #875 by @jimexist

@jimexist
Copy link
Member

@alamb i merely bumped the version.

i suspect it's because the 1.56 ver won't play nice with miri

i tried to fix it in https://github.com/apache/arrow-rs/pull/591/files#diff-621a5063e07f71e640bc5af8989b652fb7f6f183e7ef81b819a64f9670bea7f0R2044

@alamb
Copy link
Contributor Author

alamb commented Oct 30, 2021

I could not reproduce this locally (on MacOS):

Here is what i tried:

rustup toolchain install nightly-2021-10-23
rustup component add miri --toolchain nightly-2021-10-23
MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly-2021-10-23  miri test -p arrow -- --skip csv --skip ipc --skip json

Results in

...
test src/record_batch.rs - record_batch::RecordBatch::try_from_iter (line 284) ... ok
test src/record_batch.rs - record_batch::RecordBatch::try_from_iter_with_nullable (line 321) ... ok
test src/compute/kernels/window.rs - compute::kernels::window::shift (line 32) ... ok
test src/util/test_util.rs - util::test_util::arrow_test_data (line 73) ... ok
test src/util/test_util.rs - util::test_util::parquet_test_data (line 95) ... ok
test src/util/string_writer.rs - util::string_writer (line 25) ... ok
test src/compute/kernels/comparison.rs - compute::kernels::comparison::like_utf8 (line 239) ... ok

test result: ok. 117 passed; 0 failed; 0 ignored; 0 measured; 14 filtered out; finished in 5.76s


Compilation finished at Sat Oct 30 09:58:38

Going to try it on Linux next

@jimexist
Copy link
Member

Nor can I on MacBook Pro intel CPU

@alamb
Copy link
Contributor Author

alamb commented Oct 31, 2021

🤔 the same commands passed for me on Ubuntu 20.04 on x86 as well

test src/util/string_writer.rs - util::string_writer (line 25) ... ok
test src/compute/kernels/comparison.rs - compute::kernels::comparison::like_utf8 (line 239) ... ok

test result: ok. 117 passed; 0 failed; 0 ignored; 0 measured; 14 filtered out; finished in 5.85s

@jimexist
Copy link
Member

Can it because of memory consumption?

@alamb
Copy link
Contributor Author

alamb commented Oct 31, 2021

Can it because of memory consumption?

"Great minds think alike" -- lol I tried setting to --test-threads=1 in this commit to test that theory but that didn't seem to work

0dd0d62

However, I have had better luck clearing out the old cache. I'll get a new PR up shortly

@alamb
Copy link
Contributor Author

alamb commented Nov 1, 2021

It appears to have resurfaced: https://github.com/apache/arrow-rs/commits/master

Screen Shot 2021-11-01 at 6 53 13 AM

I am going to try to disable caching entirely for MIRI job and see if that looks promising

@alamb
Copy link
Contributor Author

alamb commented Nov 1, 2021

Still happening: https://github.com/apache/arrow-rs/runs/4069047142?check_suite_focus=true

Passes sometimes on rerun....

@alamb
Copy link
Contributor Author

alamb commented Nov 2, 2021

I did an experiment and confirmed the symptoms we are seeing are the same symptoms as happens when MIRI is killed on github runner due to OOM: https://github.com/alamb/oom_canary#question-what-is-the-symptom-when-cargo-miri-exceeds-its-memory-limits-on-github-runner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment