Tests fail locally at purge_chain_works

[justinfrevert]

When following the instructions to run the tests for Polkadot locally using cargo test --all, it's observed that one of the tests fails. This occurs regardless of whether the --all flag is passed.

I have not yet looked back in the commit history to see where it could have been introduced, though this was my first time running the tests.

• Observed on two machines:
  MacOS Monterey 12.0.1 (21A559)
  Ubuntu 20.04

Toolchain info:
active toolchain

stable-x86_64-apple-darwin (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Logs:

cargo test

...

     Running tests/invalid_order_arguments.rs (target/debug/deps/invalid_order_arguments-45929d8313462a8d)

running 1 test
error: Found argument 'invalid_order_arguments' which wasn't expected, or isn't valid in this context

USAGE:
    polkadot --dev

For more information try --help
test invalid_order_arguments ... ok

     Running tests/purge_chain_works.rs (target/debug/deps/purge_chain_works-2b51cd6b4a13ec3d)

running 1 test
2022-01-10 20:37:06 Parity Polkadot
2022-01-10 20:37:06 ✌️  version 0.9.13-d1db652634-x86_64-macos
2022-01-10 20:37:06 ❤️  by Parity Technologies <admin@parity.io>, 2017-2022
2022-01-10 20:37:06 📋 Chain specification: Development
2022-01-10 20:37:06 🏷  Node name: tenuous-basin-1133
2022-01-10 20:37:06 👤 Role: AUTHORITY
2022-01-10 20:37:06 💾 Database: RocksDb at /var/folders/bt/dqgx8c7d3b55rv8j13rxm4rr0000gn/T/.tmp0EDHqq/chains/dev/db/full
2022-01-10 20:37:06 ⛓  Native runtime: polkadot-9140 (parity-polkadot-0.tx9.au0)
2022-01-10 20:37:12 [0] 💸 generated 1 npos voters, 1 from validators and 0 nominators
2022-01-10 20:37:12 Took active validators from set with wrong size
2022-01-10 20:37:12 Took active validators from set with wrong size.
2022-01-10 20:37:12 Took active validators from set with wrong size
test purge_chain_works ... FAILED

failures:

---- purge_chain_works stdout ----
thread 'purge_chain_works' panicked at 'assertion failed: common::wait_for(&mut cmd, 30).map(|x| x.success()).unwrap_or_default()', tests/purge_chain_works.rs:48:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    purge_chain_works

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 31.14s

error: test failed, to rerun pass '--test purge_chain_works'

[ordian]

Is it reproducible with the current master? (after #4639)

[bkchr]

Please run tests in release mode.

[justinfrevert]

Please run tests in release mode.

Seems to work with the correct arguments, thank you.

[justinfrevert]

Noting a separate error, but it's likely unrelated.

running 1 test
error: Found argument 'invalid_order_arguments' which wasn't expected, or isn't valid in this context

USAGE:
    polkadot --dev

[justinfrevert]

@bkchr, @ordian in light of the requirement to run in release mode, should we update the readme to reflect this?

[bkchr]

Yeah create a pr please.

[justinfrevert]

I started a PR here #4703. There are some separate errors now when running the tests, my only guess is it's something other than what I was reporting here.