Prefer waiting over eventually in migration tests

[Anviking]

Issue Number

ADP-970, #2699

Overview

• waitForTxImmutability should now only need to be 10s instead of 16s
• Use waitForTxImmutability instead of eventually to protect against rollbacks in migration tests

Comments

• An alternative solution would be to use the retrying it with eventually, but we should moving away from the retrying it, not re-adding it.

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• buildkite/cardano-wallet

#expected

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• ci/hydra-build:required

Cardano.Wallet.DB.MVar
  MVar
    Extra Properties about DB initialization
building of '/nix/store/020dwr201i4z6s3xrrdlcr2ybbran40m-cardano-wallet-core-test-unit-2021.5.26-check' timed out after 900 seconds of silence

#2472

Mac

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• ci/hydra-build:required

  src/Test/Integration/Scenario/API/Shelley/Transactions.hs:335:22:
  1) API Specifications, SHELLEY_TRANSACTIONS, TRANS_CREATE_01x - Single Output Transaction
       While verifying (Status {statusCode = 200, statusMessage = "OK"},Right (ApiWallet {id = ApiT {getApiT = WalletId {getWalletId = ead1025d0402c19c08d32d8382db3dcff3a795be}}, addressPoolGap = ApiT {getApiT = AddressPoolGap {getAddressPoolGap = 20}}, balance = ApiWalletBalance {available = Quantity {getQuantity = 1869500}, total = Quantity {getQuantity = 1869500}, reward = Quantity {getQuantity = 0}}, assets = ApiWalletAssetsBalance {available = ApiT {getApiT = TokenMap (fromList [])}, total = ApiT {getApiT = TokenMap (fromList [])}}, delegation = ApiWalletDelegation {active = ApiWalletDelegationNext {status = NotDelegating, target = Nothing, changesAt = Nothing}, next = []}, name = ApiT {getApiT = WalletName {getWalletName = "Empty Wallet"}}, passphrase = Just (ApiWalletPassphraseInfo {lastUpdatedAt = 2021-06-10 13:02:43.829011 UTC}), state = ApiT {getApiT = Ready}, tip = ApiBlockReference {absoluteSlotNumber = ApiT {getApiT = SlotNo 2021}, slotId = ApiSlotId {epochNumber = ApiT {getApiT = EpochNo {unEpochNo = 20}}, slotNumber = ApiT {getApiT = SlotInEpoch {unSlotInEpoch = 21}}}, time = 2021-06-10 13:03:14.2 UTC, block = ApiBlockInfo {height = Quantity {getQuantity = 917}}}}))
       expected: Quantity {getQuantity = 0}
        but got: Quantity {getQuantity = 1869500}

  To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/TRANS_CREATE_01x - Single Output Transaction/"

  src/Test/Integration/Scenario/CLI/Shelley/Wallets.hs:754:18:
  2) CLI Specifications, SHELLEY_CLI_WALLETS, WALLETS_UTXO_02 - Utxo statistics works properly
       Waited longer than 90s to resolve action: "Wallet balance is as expected".
       expected: Quantity {getQuantity = 1562000000}
        but got: Quantity {getQuantity = 0}

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_WALLETS/WALLETS_UTXO_02 - Utxo statistics works properly/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:129:18:
  3) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_01x - Restoration from account public key preserves funds
       Waited longer than 90s to resolve action: "Wallet balance is as expected".
       expected: Quantity {getQuantity = 1000000}
        but got: Quantity {getQuantity = 0}

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_01x - Restoration from account public key preserves funds/"

Randomized with seed 109606617

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• buildkite/cardano-wallet

Completed 8 action(s).
--
  | /tmp/stack-8f9c70c9dfa1921e/test-ghc-env: openBinaryFile: resource busy (file is locked)
  |  
  | /tmp/stack-8f9c70c9dfa1921e/test-ghc-env: openBinaryFile: resource busy (file is locked)
  |  
  | /tmp/stack-8f9c70c9dfa1921e/test-ghc-env: openBinaryFile: resource busy (file is locked)
  |  
  | /tmp/stack-8f9c70c9dfa1921e/test-ghc-env: openBinaryFile: resource busy (file is locked)
  |  
  | /tmp/stack-8f9c70c9dfa1921e/test-ghc-env: openBinaryFile: resource busy (file is locked)
  | error: Command exited with code 1!
  | Continuing...

[rvl]

So the reason for this change is that between checking one wallet condition and the next, the chain rolls back and invalidates the first condition, such that the second condition fails?

Before making a solution, do we have logging which proves conclusively that this is the case?

And what about other ways to solve this? We need some methods to make our tests less fragile - otherwise this flakiness will keep on happening.

My two ideas so far are:

1. Provide API endpoints to pause and resume syncing for a wallet - preventing rollback while assertions are being checked.
2. Provide a way for assertions to be contingent on a certain TxId having status=InLedger. If an assertion fails, and the Tx has gone back to Pending, then restart the eventually block from the beginning.

[rvl]

Why not

[Anviking]

bors r+

[iohk-bors]

Build succeeded:

• ci/hydra-build:required
• buildkite/cardano-wallet