Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use random port number when starting cardano-testnet #5791

Merged
merged 3 commits into from
Apr 23, 2024

Conversation

carbolymer
Copy link
Contributor

@carbolymer carbolymer commented Apr 19, 2024

Description

This PR makes cardano-testnet use random port range when starting the testnet. The issue arises in Hydra, when cardano-testnet is executed for multiple architectures at once, using separate invocations of cabal test. This can result in clashes in occupied ports by different testnets and test failure similar to this (taken from https://ci.iog.io/build/3734298/nixlog/1):

Cardano node process did not start: 
cardano-node: DiffusionError Network.Socket.bind: resource busy (Address already in use)
Cardano node process did not start: 
cardano-node: DiffusionError Network.Socket.bind: resource busy (Address already in use)
Cardano node process did not start: 
cardano-node: DiffusionError Network.Socket.bind: resource busy (Address already in use)

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. These may include:
    • golden tests
    • property tests
    • roundtrip tests
    • integration tests
      See Runnings tests for more details
  • Any changes are noted in the CHANGELOG.md for affected package
  • The version bounds in .cabal files are updated
  • CI passes. See note on CI. The following CI checks are required:
    • Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
    • Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
    • Code builds on Linux, MacOS and Windows for ghc-8.10.7 and ghc-9.2.7
  • Self-reviewed the diff

Note on CI

If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges. Please contact IOG node developers to do this
for you.

@carbolymer carbolymer force-pushed the mgalazyn/test/randomize-testnets-ips branch from a0dbad2 to 5dc7f6f Compare April 19, 2024 12:29
@carbolymer carbolymer force-pushed the mgalazyn/test/randomize-testnets-ips branch from b4c70b2 to 6944a55 Compare April 19, 2024 13:46
@carbolymer carbolymer force-pushed the mgalazyn/test/randomize-testnets-ips branch from 6944a55 to e22de5a Compare April 19, 2024 13:54
@carbolymer carbolymer changed the title Use random IP addresses when starting cardano-testnet Use random Port when starting cardano-testnet Apr 19, 2024
@carbolymer carbolymer changed the title Use random Port when starting cardano-testnet Use random port number when starting cardano-testnet Apr 19, 2024
@carbolymer carbolymer marked this pull request as ready for review April 19, 2024 15:07
@carbolymer carbolymer requested a review from a team as a code owner April 19, 2024 15:07
@smelc
Copy link
Contributor

smelc commented Apr 22, 2024

@carbolymer

when cardano-testnet is executed for multiple architectures at once

Multiple architectures are trying to use the same port number of one machine? How can this happen? Are the architectures being emulated and so are sharing the same pool of ports of one physical machine?

when cardano-testnet is executed for multiple architectures at once, using separate invocations of cabal test. This can result in clashes in occupied ports by different testnets and test failure similar to this

So wouldn't it be simpler to multiply numberOfPorts by the number of architectures?

@carbolymer
Copy link
Contributor Author

carbolymer commented Apr 22, 2024

@smelc

Multiple architectures are trying to use the same port number of one machine? How can this happen? Are the architectures being emulated and so are sharing the same pool of ports of one physical machine?

Hydra. It's cross-compiling and running tests on the same host.

So wouldn't it be simpler to multiply numberOfPorts by the number of architectures?

Do you mean to use that instead of maxPortsPerRequest? I'd have to find out and hash the architecture currently used to get different starting port offset too. But it wouldn't protect you against running multiple test suites in different hydra jobs (different cabal test invocations) at the same time.

All in all it's just a hack. You'd have to implement some fancy synchronisation in the operating system state (using files in /run maybe) and coordinate between different cabal test executions. I don't know how much Hydra isolates evaluations and if that's even possible.

@carbolymer carbolymer added this pull request to the merge queue Apr 23, 2024
Merged via the queue into master with commit be33ee4 Apr 23, 2024
22 checks passed
@carbolymer carbolymer deleted the mgalazyn/test/randomize-testnets-ips branch April 23, 2024 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants