Add Mithril end to end instructions for --skip-cardano-bin-download command

[dlachaume]

Content

This PR includes a modification in the README file of mithril-end-to-end.

The paragraph explains how to use the --skip-cardano-bin-download command to run the tests.

This option was added with the merge of the PR #1052.

Pre-submit checklist

• Branch
  • Tests are provided (if possible)
  • Crates versions are updated (if relevant)
  • Commit sequence broadly makes sense
  • Key commits have useful messages
• PR
  • No clippy warnings in the CI
  • Self-reviewed the diff
  • Useful pull request description
  • Reviewer requested
• Documentation
  • Update README file (if relevant)
  • Update documentation website (if relevant)
  • Add dev blog post (if relevant)

[github-actions]

Test Results

    3 files  ±0    17 suites  ±0   6m 45s ⏱️ + 1m 35s
714 tests ±0  714 ✔️ ±0  0 💤 ±0  0 ❌ ±0 
788 runs  ±0  788 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 6aff3ff. ± Comparison against base commit 13bf6d2.

♻️ This comment has been updated with latest results.

[jpraynaud]

LGTM 👍

[jpraynaud]

@scarmuega, @falcucci this is an update on the documentation that explains how to run the devnet with MacOS 👍

[falcucci]

@jpraynaud @dlachaumepalo thank you for the new instructions, really helpful. I'll bring a report from my perspective to run the tests on macOS.

I got some issues that I would like to know if one of you could reproduce aswell or are already aware of?

In the first attempt to run I got a sed compatibility error when mithril tries to setup the cardano-node config file in the mithril-devnet/devnet-mkfiles.sh

sed: 1: "/ByronGenesisFile/ aCon ...": command a expects \ followed by text

the code snippet that throws the error

sed -i ${ROOT}/configuration.yaml \
    -e 's/Protocol: RealPBFT/Protocol: Cardano\nPBftSignatureThreshold: 0.6/' \
    -e 's/minSeverity: Info/minSeverity: Info/' \
    -e 's/TracingVerbosity: NormalVerbosity/TracingVerbosity: MinimalVerbosity/' \
    -e 's/TurnOnLogMetrics: True/TurnOnLogMetrics: False/' \
    -e 's|GenesisFile: genesis.json|ByronGenesisFile: byron/genesis.json|' \
    -e '/ByronGenesisFile/ aConwayGenesisFile: shelley/genesis.conway.json' \
    -e '/ByronGenesisFile/ aAlonzoGenesisFile: shelley/genesis.alonzo.json' \
    -e '/ByronGenesisFile/ aShelleyGenesisFile: shelley/genesis.json' \
    -e 's/RequiresNoMagic/RequiresMagic/' \
    -e 's/LastKnownBlockVersion-Major: 0/LastKnownBlockVersion-Major: 8/' \
    -e 's/LastKnownBlockVersion-Minor: 2/LastKnownBlockVersion-Minor: 0/' \
    -e 's/LastKnownBlockVersion-Alt: 0/LastKnownBlockVersion-Alt: 0/'

the workaround I did to proceed:

sed -i backup.yaml  \
    -e 's/Protocol: RealPBFT/Protocol: Cardano\nPBftSignatureThreshold: 0.6/' \
    -e 's/minSeverity: Info/minSeverity: Info/' \
    -e 's/TracingVerbosity: NormalVerbosity/TracingVerbosity: MinimalVerbosity/' \
    -e 's/TurnOnLogMetrics: True/TurnOnLogMetrics: False/' \
    -e 's|GenesisFile: genesis.json|ByronGenesisFile: byron/genesis.json|' \
    -e '/ByronGenesisFile/ a\
ConwayGenesisFile: shelley/genesis.conway.json' \
    -e '/ByronGenesisFile/ a\
AlonzoGenesisFile: shelley/genesis.alonzo.json' \
    -e '/ByronGenesisFile/ a\
ShelleyGenesisFile: shelley/genesis.json' \
    -e 's/RequiresNoMagic/RequiresMagic/' \
    -e 's/LastKnownBlockVersion-Major: 0/LastKnownBlockVersion-Major: 8/' \
    -e 's/LastKnownBlockVersion-Minor: 2/LastKnownBlockVersion-Minor: 0/' \
    -e 's/LastKnownBlockVersion-Alt: 0/LastKnownBlockVersion-Alt: 0/' \
    configuration.yaml

# copy and tweak the configuration
cp configuration.yaml ${ROOT}/configuration.yaml

---

another sed compatibility error that I got

sed: -I or -i may not be used with stdin

due to the following code snippet also at mithril-devnet/devnet-mkfiles.sh

sed -i shelley/genesis.spec.json \
    -e 's/"slotLength": 1/"slotLength": '${SLOT_LENGTH}'/' \
    -e 's/"activeSlotsCoeff": 5.0e-2/"activeSlotsCoeff": 0.50/' \
    -e 's/"securityParam": 2160/"securityParam": '${SECURITY_PARAM}'/' \
    -e 's/"epochLength": 432000/"epochLength": '${EPOCH_LENGTH}'/' \
    -e 's/"maxLovelaceSupply": 0/"maxLovelaceSupply": 1000000000/' \
    -e 's/"decentralisationParam": 1.0/"decentralisationParam": 0.7/' \
    -e 's/"major": 0/"major": 4/' \
    -e 's/"updateQuorum": 5/"updateQuorum": 2/'

how I made it work?

sed -i backup \
    -e 's/"slotLength": 1/"slotLength": '${SLOT_LENGTH}'/' \
    -e 's/"activeSlotsCoeff": 5.0e-2/"activeSlotsCoeff": 0.50/' \
    -e 's/"securityParam": 2160/"securityParam": '${SECURITY_PARAM}'/' \
    -e 's/"epochLength": 432000/"epochLength": '${EPOCH_LENGTH}'/' \
    -e 's/"maxLovelaceSupply": 0/"maxLovelaceSupply": 1000000000/' \
    -e 's/"decentralisationParam": 1.0/"decentralisationParam": 0.7/' \
    -e 's/"major": 0/"major": 4/' \
    -e 's/"updateQuorum": 5/"updateQuorum": 2/' \
    shelley/genesis.spec.json

another solution is using awk which hasnt those compatibility errors trought the OS

awk '{
    gsub(/"slotLength": 1/, "\"slotLength\": '${SLOT_LENGTH}'");
    gsub(/"activeSlotsCoeff": 5.0e-2/, "\"activeSlotsCoeff\": 0.50");
    gsub(/"securityParam": 2160/, "\"securityParam\": '${SECURITY_PARAM}'");
    gsub(/"epochLength": 432000/, "\"epochLength\": '${EPOCH_LENGTH}'");
    gsub(/"maxLovelaceSupply": 0/, "\"maxLovelaceSupply\": 1000000000");
    gsub(/"decentralisationParam": 1.0/, "\"decentralisationParam\": 0.7");
    gsub(/"major": 0/, "\"major\": 4");
    gsub(/"updateQuorum": 5/, "\"updateQuorum\": 2");
    print $0;
}' shelley/genesis.spec.json > temp.json && mv temp.json shelley/genesis.spec.json

---

TBD

maybe consider to write down that having some mithril nodes compiled is a requirement here?

even if those dependencies it's obvious to run the end-to-end tests, maybe would be a good reminder.

---

the last error I got:

poke SockAddr: path is too long in SockAddr Unix

As PathBuf generates a long tmp path it overpass the socket path length limit, at least on macOS

adding an absolute working-dir path fixed the issue.

after all, the tests worked as expected!

[jpraynaud]

@falcucci thank you for this feedback!

I think that @dlachaumepalo knows a fix to avoid the modification of the sed commands, he will add it in the README for a better DevX 🙂

He will also add a mention to the fact that the Mithril nodes need to be built prior to launching the ent to end test.

[dlachaume]

Thank you @falcucci for your detailed feedback!

Regarding the sed compatibility issue:
Indeed, I had to install and use gnu-sed. This avoid the modification of the existing command.
I'm gonna write an additional note for this.

For your remark on compile Mithril nodes:
I will add this reminder in the "Build and run the test suite" section.

Concerning the third point:
I'm not sure to get it, could you add a few details to this issue so that I can reproduce it?

[falcucci]

@jpraynaud @dlachaumepalo amazing, thanks!

Concerning the third point:
I'm not sure to get it, could you add a few details to this issue so that I can reproduce it?

the error basically throws when the --work-directory <your-path> is not being specified.

due to that the default tmp path generated to the node.sock file seems to be too big.

/private/var/folders/9x/wkxqhs851xbcrjqbbm80t9980000gn/T/mithril_end_to_end/devnet/node-bft1/ipc/node.sock

adding a short absolute path for --work-directory <your-path> fixes it.

./mithril-end-to-end --work-directory <your-path> --db-directory db/ --bin-directory ../../target/release --skip-cardano-bin-download