CI Win64 - "The parameter is incorrect" / "The handle is invalid"

[mratsim]

#415 is blocked on an Windows + 64-bit + -d:release only error in PR #408 #422 #431

Some tests seem to fail for no apparent reason, probably while reading the file. Windows 32-bit does not exhibit this behaviour.

Example: #422 (comment) (https://ci.appveyor.com/project/nimbus/nim-beacon-chain/builds/27293393/job/0b600abqo06snpgi#L3130)

[Suite] Official - Operations - Transfers  [Preset: minimal]
  [OK] [Valid]   success_non_activated
  [OK] [Valid]   success_withdrawable
  [OK] [Valid]   success_active_above_max_effective
  [OK] [Valid]   success_active_above_max_effective_fee
NOT 2019-09-09 23:14:47+00:00 Transfer: incorrect signature              tid=2716
  [OK] [Invalid] invalid_signature
NOT 2019-09-09 23:14:47+00:00 Transfer: only senders who either 1) have never been eligible for activation or 2) are withdrawable or 3) have a balance of MAX_EFFECTIVE_BALANCE after the transfer are valid. tid=2716
  [OK] [Invalid] active_but_transfer_past_effective_balance
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] incorrect_slot
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Valid]   transfer_clean
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Valid]   transfer_clean_split_to_fee
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_for_fee
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_for_amount_result_full
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_for_combined_result_dust
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_for_combined_result_full
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_for_combined_big_amount
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_for_combined_big_fee
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_off_by_1_fee
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_off_by_1_amount
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] insufficient_balance_duplicate_as_fee_and_amount
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] no_dust_sender
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] no_dust_recipient
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] non_existent_sender
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] non_existent_recipient
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] [Invalid] invalid_pubkey

Unfortunately, it seems like a non-trivial issue to debug by just relying on the CI as we cannot even reproduce the issue with --opt:size instead of --d:release.

It may even be triggered randomly by the kernel see an unrelated codepath (COM+ objects) that trigger the same error randomly.

At first glance looking through those issues, and given the Nim code it should be related to opening/reading files, and maybe to LFS?

• Windows Storage Space issue
• Reading a drive
• Mounting a volume

status-im/nim-serialization#9 has been open to rule out running of file descriptors.

[tersec]

It seems reasonable to wrap some of them in

when not 64-bit windows:
   runTest()

or its moral equivalent in terms of maintaining the list of such tests, in such a way as they can be easily found and stripped out later when this issue is resolved.

[jangko]

I can reproduce it locally. I will take a look

[mratsim]

A new one from #436 - https://ci.appveyor.com/project/nimbus/nim-beacon-chain/builds/27346548/job/gki7ew7tkda8ou8w#L242

This BLS test has always been working for month and the next tests work.

[Suite] Official - Shuffling tests [Preset:  [Preset: minimal]
  [OK] Shuffling a sequence of N validators [Preset: minimal]
[Suite] Official - BLS tests
    Unhandled exception: The parameter is incorrect.
 [OSError]
  [FAILED] Private to public key conversion
  [OK] Message signing
  [OK] Aggregating signatures
  [OK] Aggregating public keys

The change was just:

status-im/nim-serialization@e874ae6...master

template loadFile*(Format: distinct type,
                   filename: string,
                   RecordType: distinct type,
                   params: varargs[untyped]): auto =
  mixin init, ReaderType
  var stream = openFile(filename)

  # TODO:
  # Remove this when statement once the following bug is fixed:
  # https://github.com/nim-lang/Nim/issues/9996
  when astToStr(params) != "":
    var reader = init(ReaderType(Format), stream, params)
  else:
    var reader = init(ReaderType(Format), stream)

    reader.readValue(RecordType)

template loadFile*(Format: distinct type,
                   filename: string,
                   RecordType: distinct type,
                   params: varargs[untyped]): auto =
  reader.readValue(RecordType)
  try:
    # TODO:
    # Remove this when statement once the following bug is fixed:
    # https://github.com/nim-lang/Nim/issues/9996
    when astToStr(params) != "":
      var reader = init(ReaderType(Format), stream, params)
    else:
      var reader = init(ReaderType(Format), stream)

    reader.readValue(RecordType)
  finally:
    # TODO: destructors
    if not stream.isNil:
      stream[].close()

[mratsim]

We got a new one: https://ci.appveyor.com/project/nimbus/nim-beacon-chain/build/job/tfvhv77u93mkf88y#L3212 for PR #442

[Suite] Official - Sanity - Blocks  [Preset: mainnet]
  [OK] [Invalid] Previous slot block transition (prev_slot_block_transition)
  [OK] [Valid]   Same slot block transition (same_slot_block_transition)
  [OK] [Valid]   Empty block transition (empty_block_transition)
  [OK] [Valid]   Skipped Slots (skipped_slots)
  [OK] [Valid]   Empty epoch transition (empty_epoch_transition)
    Unhandled exception: The handle is invalid.
 [OSError]
  [FAILED] [Valid]   Empty epoch transition not finalizing (empty_epoch_transition_not_finalizing)
  [OK] [Valid]   Proposer slashing (proposer_slashing)
  [OK] [Valid]   Attester slashing (attester_slashing)
  [OK] [Valid]   Deposit top up (deposit_top_up)
  [OK] [Valid]   Voluntary exit (voluntary_exit)
  [OK] [Valid]   Balance-driven status transitions (balance_driven_status_transitions)
  [OK] [Valid]   Historical batch (historical_batch)
    Unhandled exception: The handle is invalid.
 [OSError]
  [FAILED] [Valid]   ETH1 data votes consensus (eth1_data_votes_consensus)
    Unhandled exception: The handle is invalid.
 [OSError]
  [FAILED] [Valid]   ETH1 data votes no consensus (eth1_data_votes_no_consensus)

[jangko]

I have found our source of bug. see nim-lang/Nim#12186

[mratsim]

closed by #513