cardano-testnet: fix that genesis files generated by create-testnet-data were ignored #5732

smelc · 2024-03-28T15:49:02Z

Description

Fixes that genesis files generated by create-testnet-data were not passed to the nodes spawned in clusters
As a consequence, become able to benefit of create-testnet-data new features of creating DReps; which is used in this PR to remove custom inialization

⚠️ This PR changes the shape of directories generates by cardano-testnet. It changes from:

.
├── alonzo-genesis.json
├── byron
│   └── genesis.json
├── byron.genesis.spec.json
├── configuration.yaml
├── conway-genesis.json
├── genesis.alonzo.spec.json
├── genesis.conway.spec.json
└── shelley
    ├── genesis.alonzo.json
    ├── genesis.conway.json
    └── genesis.shelley.json

to

.
├── alonzo-genesis.json
├── byron
│   └── genesis.json
├── byron.genesis.spec.json
├── configuration.yaml
├── conway-genesis.json
├── genesis.alonzo.spec.json
├── genesis.conway.spec.json
└── shelley-genesis.json

Checklist

Commit sequence broadly makes sense and commits have useful messages
Self-reviewed the diff

Note on CI

If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges. Please contact IOG node developers to do this
for you.

smelc · 2024-03-29T14:46:35Z

cardano-testnet/src/Testnet/Components/Configuration.hs

+  H.rewriteJsonFile @Value (tempAbsPath </> defaultGenesisFilepath ShelleyEra) $ \o -> o
    & L.key "maxLovelaceSupply" . L._Integer .~ 10_000_000_000_000_000


I tried to get rid of this rewrite, but it makes the Treasury Growth fail, so something is still wrong somewhere. I will look into it in another PR. I don't want to block this one.

smelc · 2024-03-29T14:48:28Z

cardano-testnet/src/Testnet/Components/Configuration.hs

  genesisByronDir <- H.createDirectoryIfMissing $ tempAbsPath </> "byron"

  files <- H.listDirectory tempAbsPath
-  forM_ files $ \file -> do
-    H.note file
+  forM_ files H.note

  H.renameFile (tempAbsPath </> "byron-gen-command" </> "genesis.json") (genesisByronDir </> "genesis.json")


I tried to make the byron genesis handled like the others (i.e. to end up at byron-genesis.json), but it makes the node refuse to start, because of a hash mismatch. I will solve this later on, because I didn't want to block this PR.

…ata were ignored Simplify structure of directory generated in cardano-testnet: put genesis files at the top-level instead of within shelley/

…estnet-data

cardano-testnet/src/Testnet/Components/Configuration.hs

Jimbo4350

Nice work 💪

Jimbo4350 · 2024-04-02T13:50:05Z

cardano-testnet/src/Testnet/Components/Configuration.hs

+      inputGenesisAlonzoFp  = tempAbsPath </> genesisInputFilepath AlonzoEra
+      inputGenesisConwayFp  = tempAbsPath </> genesisInputFilepath ConwayEra
+
+  -- We write the genesis files to disk, to pass them to create-testnet-data.


These files shouldn't be modified after they are created. create-testnet-data should create the final genesis files we want to use and then we create a configuration file based on those genesis files.

Yep agreed, we should remove the last rewrites that remain later on 👍

Can you update the comment so it's obvious this isn't the behaviour we want?

Jimbo4350 · 2024-04-02T13:56:10Z

cardano-testnet/src/Testnet/Defaults.hs

-defaultGenesisFilepath era = "shelley" </> ("genesis." <> eraToString era <> ".json")
+defaultGenesisFilepath era =
+  -- This path is actually generated by create-testnet-data. Don't change it.
+  eraToString era <> "-genesis.json"


Jimbo4350 · 2024-04-02T13:57:51Z

cardano-testnet/src/Testnet/Defaults.hs

+defaultDRepVkeyFp
+  :: Int -- ^ The DRep's index (starts at 0)
+  -> FilePath
+defaultDRepVkeyFp n = "drep-keys" </> ("drep" <> show n) </> "drep.vkey"


Jimbo4350 · 2024-04-02T13:58:33Z

cardano-testnet/src/Testnet/Components/Configuration.hs

  => (MonadTest m, MonadIO m, HasCallStack)
  => TmpAbsolutePath
  -> AnyCardanoEra -- ^ The era used for generating the hard fork configuration toggle
  -> m LBS.ByteString
-createConfigYaml (TmpAbsolutePath tempAbsPath) era = GHC.withFrozenCallStack $ do
+createConfigJson (TmpAbsolutePath tempAbsPath) era = GHC.withFrozenCallStack $ do


Let's use pretty JSON here. Makes debugging a little easier.

Good point, I'm myself always reformatting those JSON files before inspecting them 😆

Jimbo4350 · 2024-04-02T13:58:52Z

cardano-testnet/src/Testnet/Components/Configuration.hs

 -- 50 second epochs
 -- Epoch length should be "10 * k / f" where "k = securityParam, f = activeSlotsCoeff"
  H.rewriteJsonFile @Value inputGenesisShelleyFp $ \o -> o
-    -- TODO: remove rho and tau adjustment after https://github.com/IntersectMBO/cardano-api/pull/425 gets
-    -- integrated with newer cardano-api into node
-    & L.key "protocolParams" .  L.key "rho" . L._Number  .~ 0.1


Jimbo4350 · 2024-04-02T13:59:25Z

cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/Cli/Conway/DRepRetirement.hs

@@ -93,49 +93,6 @@ hprop_drep_retirement = H.integrationRetryWorkspace 2 "drep-retirement" $ \tempA
      drepSKeyFp :: Int -> FilePath
      drepSKeyFp n = tempAbsPath' </> "drep-keys" </> ("drep" <> show n) </> "drep.skey"

-  -- Create Drep registration certificates


Do we still test this work flow?

No we don't, but I'll readd such a test shortly afterwards.

Follow-up tracked here: #5742

Jimbo4350

LGTM! Only one concern: #5732 (comment)

…r easier debugging

This was referenced Mar 29, 2024

Test correct recording of DRep votes in ledger #5731

Merged

Refactor testnet SPO certificates functions #5733

Merged

smelc force-pushed the smelc/debug-native-testnet-dreps branch 2 times, most recently from 70b39b9 to 980b943 Compare March 29, 2024 14:41

smelc marked this pull request as ready for review March 29, 2024 14:45

smelc requested a review from a team as a code owner March 29, 2024 14:45

smelc commented Mar 29, 2024

View reviewed changes

smelc force-pushed the smelc/debug-native-testnet-dreps branch 2 times, most recently from 70e541e to 9c8cade Compare March 29, 2024 15:24

smelc added 2 commits April 2, 2024 12:43

cardano-testnet: fix that genesis files generated by create-testnet-d…

873f9d9

…ata were ignored Simplify structure of directory generated in cardano-testnet: put genesis files at the top-level instead of within shelley/

cardano-testnet: simplify tests using DReps, thanks to newer create-t…

544120c

…estnet-data

smelc force-pushed the smelc/debug-native-testnet-dreps branch from 9c8cade to 6df5731 Compare April 2, 2024 10:47

carbolymer approved these changes Apr 2, 2024

View reviewed changes

cardano-testnet/src/Testnet/Components/Configuration.hs Outdated Show resolved Hide resolved

smelc force-pushed the smelc/debug-native-testnet-dreps branch from 6df5731 to 9bea1a3 Compare April 2, 2024 13:19

smelc enabled auto-merge April 2, 2024 13:30

Jimbo4350 requested changes Apr 2, 2024

View reviewed changes

Jimbo4350 self-requested a review April 2, 2024 13:59

Jimbo4350 approved these changes Apr 2, 2024

View reviewed changes

smelc disabled auto-merge April 2, 2024 14:18

cardano-testnet: Share definition of path to DRep keys

cd3c037

smelc mentioned this pull request Apr 2, 2024

Test DRep registration workflow #5742

Closed

smelc added 4 commits April 2, 2024 16:29

cardano-testnet: rename a function

bf451fe

cardano-testnet: remove rewrite that became obsolete

1561146

cardano-testnet: Use encodePretty when writing some files to disk, fo…

38c754b

…r easier debugging

cardano-testnet: Update golden files

45d1b8c

smelc force-pushed the smelc/debug-native-testnet-dreps branch from 9bea1a3 to 45d1b8c Compare April 2, 2024 14:32

smelc enabled auto-merge April 2, 2024 14:32

smelc added this pull request to the merge queue Apr 3, 2024

Merged via the queue into master with commit 6c606d3 Apr 3, 2024
27 of 28 checks passed

smelc deleted the smelc/debug-native-testnet-dreps branch April 3, 2024 09:13

palas mentioned this pull request Apr 4, 2024

Remove manual DRep registration from InfoAction test #5749

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cardano-testnet: fix that genesis files generated by create-testnet-data were ignored #5732

cardano-testnet: fix that genesis files generated by create-testnet-data were ignored #5732

smelc commented Mar 28, 2024

smelc Mar 29, 2024 •

edited

Loading

smelc Mar 29, 2024

Jimbo4350 left a comment

Jimbo4350 Apr 2, 2024

smelc Apr 2, 2024

Jimbo4350 Apr 2, 2024

Jimbo4350 Apr 2, 2024 •

edited

Loading

Jimbo4350 Apr 2, 2024

Jimbo4350 Apr 2, 2024

smelc Apr 2, 2024

Jimbo4350 Apr 2, 2024

Jimbo4350 Apr 2, 2024

smelc Apr 2, 2024

Jimbo4350 left a comment •

edited

Loading

		H.rewriteJsonFile @Value (tempAbsPath </> defaultGenesisFilepath ShelleyEra) $ \o -> o
		& L.key "maxLovelaceSupply" . L._Integer .~ 10_000_000_000_000_000

cardano-testnet: fix that genesis files generated by create-testnet-data were ignored #5732

cardano-testnet: fix that genesis files generated by create-testnet-data were ignored #5732

Conversation

smelc commented Mar 28, 2024

Description

Checklist

Note on CI

smelc Mar 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jimbo4350 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jimbo4350 Apr 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jimbo4350 left a comment • edited Loading

Choose a reason for hiding this comment

smelc Mar 29, 2024 •

edited

Loading

Jimbo4350 Apr 2, 2024 •

edited

Loading

Jimbo4350 left a comment •

edited

Loading