plutus: 1.9 -> 1.15.0.1, cardano-api: 8.20 -> 8.29.1 #1168

locallycompact · 2023-11-17T12:02:33Z

No description provided.

ch1bo

Let's try to only tie hydra-cardano-api to cardano-api and insulate the remaining code base as much as possible. See my comments for some ideas.

hydra-cardano-api/src/Hydra/Cardano/Api/MultiAssetSupportedInEra.hs

hydra-cardano-api/src/Hydra/Cardano/Api/ReferenceTxInsScriptsInlineDatumsSupportedInEra.hs

hydra-cardano-api/src/Hydra/Cardano/Api/ScriptDataSupportedInEra.hs

hydra-cardano-api/src/Hydra/Cardano/Api/Value.hs

ch1bo · 2023-11-20T10:10:22Z

hydra-node/src/Hydra/Ledger/Cardano.hs

-  body <- createAndValidateTransactionBody bodyContent
-  let witnesses = [makeShelleyKeyWitness body (WitnessPaymentKey sk)]
+  body <- createAndValidateTransactionBody BabbageEra bodyContent
+  let witnesses = [makeShelleyKeyWitness ShelleyBasedEraBabbage body (WitnessPaymentKey sk)]


Commenting once, but applies to multiple places:

Avoid mentioning Babbage in packages outside of hydra-cardano-api. Currently, that package serves as an "anti-corruption" layer and hard-codes the era we target. Even if we never make it configurable, we want to limit the amount of code that needs changing. That's also the reason why we have all those patterns and alias functions in Hydra.Cardano.Api, which remove the need for call sites to pick an era.

Must: try to avoid "witness" values outside of hydra-cardano-api.

Concretely telling from this call site, I would propose to provide a fixed-era createAndValidateTransactionBody :: TxBodyContent Era -> Either TxBodyError (TxBody Era) in hydra-cardano-api and the same for makeShelleyKeyWitness and/or using type class constraints. Note the Tx return value of mkSimpleTx is a hard-coded Tx Era = Tx BabbageEra right now.

ghost · 2023-11-30T11:08:02Z

Seems like there is some "funny" test errors which are hard to troubleshoot: https://github.com/input-output-hk/hydra/actions/runs/7045153258/job/19174290641?pr=1168#step:5:1856

github-actions · 2023-12-04T11:51:10Z

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2023-12-06 16:48:46.121134602 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	985245919fcc6c0c5cd116023cd2c947c43e80dcbb5075fe12433fbb	4072
νCommit	7cb20fa71eb4c563ca283566ebe0aa65859d96c3f8cba35c52c181fd	2043
νHead	7a36661f5c15e9f1783aeaab890812c59b7286cbbc6de762d3110772	8816
μHead	8b111ac12274e46314769295a1c5dcab1d260096fc469fd698065463*	3851

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4374	10.46	4.04	0.46
2	4576	12.85	4.95	0.49
3	4776	14.79	5.67	0.52
5	5179	19.33	7.40	0.59
10	6184	30.58	11.67	0.75
41	12417	99.02	37.59	1.77

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	534	11.37	4.44	0.30
2	723	15.04	6.07	0.35
3	908	18.85	7.75	0.40
5	1282	26.90	11.27	0.51
10	2216	49.55	20.97	0.80
19	3899	99.43	41.75	1.43

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	484	21.07	8.25	0.40
2	113	590	32.34	12.79	0.53
3	171	700	48.40	19.19	0.71
4	228	814	61.62	24.67	0.86
5	283	920	79.03	31.77	1.06

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	552	16.72	7.70	0.37
2	765	18.54	9.37	0.40
3	944	20.35	11.02	0.44
5	1235	23.36	13.87	0.49
10	2170	32.75	22.33	0.67
50	8960	96.84	82.87	1.92

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	516	19.82	8.62	0.40
2	763	22.26	10.69	0.44
3	925	24.19	12.36	0.48
5	1330	28.07	15.76	0.55
10	2144	37.65	23.95	0.72
43	7807	99.81	77.63	1.86

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4334	19.02	8.19	0.55
2	4475	31.96	13.92	0.71
3	4650	47.97	21.03	0.89
4	4819	66.37	29.21	1.11
5	4909	80.55	35.33	1.27

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4205	8.26	3.46	0.43
5	1	57	4240	9.49	4.22	0.44
5	5	285	4376	14.19	7.18	0.51
5	10	572	4547	20.55	11.08	0.60
5	20	1138	4884	32.64	18.62	0.76
5	30	1711	5228	44.94	26.25	0.93
5	40	2275	5562	57.25	33.88	1.10
5	50	2847	5905	69.77	41.61	1.27
5	74	4212	6717	99.34	59.96	1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2023-12-06 16:42:58.932034475 UTC

Baseline Scenario

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	23.578696998
P99	102.13375788000026ms
P95	33.44953534999999ms
P50	21.082649ms
Number of Invalid txs	0

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	4.290704354
P99	6.029217359999999ms
P95	5.069519049999999ms
P50	4.186731999999999ms
Number of Invalid txs	0

github-actions · 2023-12-04T12:50:04Z

Test Results

372 tests ±0 367 ✔️ ±0 21m 42s ⏱️ -11s
126 suites ±0     5 💤 ±0
    5 files ±0     0 ❌ ±0

Results for commit 03c185b. ± Comparison against base commit b2dc5a0.

♻️ This comment has been updated with latest results.

ch1bo

This is looking good now! Only a few suggestions and one commit to drop before we merge this.

Please update the PR title and provide a description why this is needed / how it is related to some issue (I think it is needed, but PRs or commits should be documenting this)

Also, please honor the PR template and update at least the CHANGELOG.

ch1bo · 2023-12-04T17:14:56Z

cabal.project

-  , hackage.haskell.org 2023-10-20T06:54:18Z
-  , cardano-haskell-packages 2023-10-20T16:16:59Z
+  , hackage.haskell.org 2023-12-27T06:54:18Z
+  , cardano-haskell-packages 2023-12-27T16:16:59Z


Should: use actual index dates to remove warnings printed

hydra-cluster/datasets/1-node.cbor

ch1bo · 2023-12-04T17:19:14Z

hydra-cluster/test/Test/LogFilterSpec.hs

This is the same issue as it was with the benchmarks. The fact that we are persisting JSON serialized transactions here which can change their byte representation (which hashed gives the transaction id) is fragile.

No need to change something in this PR, but this is yet another argument against having JSON of transactions and maybe only allow CBOR encoding of them (+ prepare / select tooling to easily introspect transaction CBOR)

hydra-node/src/Hydra/Chain/Direct/Wallet.hs

hydra-node/src/Hydra/Chain/CardanoClient.hs

hydra-chain-observer/src/Hydra/ChainObserver.hs

hydra-cluster/src/Hydra/Cluster/Faucet.hs

hydra-cluster/src/CardanoClient.hs

hydra-cardano-api/src/Hydra/Cardano/Api/Tx.hs

cardano-api: 8.20 -> 8.29.1

It seems like ledger changed some metada encoding between versions. We had some specific auxiliaryData re-encoding fail, while most of the golden data was roundtripping just fine. We suspect a slight difference in the "canonical" serialization produced by the ledger **iff** a data type was not serialized before. Which is the case in our golden test suite where we generated random values and serialize them. TLDR; our golden tests are quite strict and this should be fine

This ensures that the byte-for-byte decoded data set is ingested into a Hydra head correctly. The JSON instance of transactions might not preserve the body byte-for-byte and hence invalidate witnesses leading to invalid transaction.

Seems like we are having issues with inconsistent To/FromJSON serialisation following upgrade, so perhaps we should move our logs to use the "canonical" CBOR represenation of a Tx instead of semi-custom JSON?

This was renamed upstream and we should capture the meaning the same way. It is to provide witnesses for features which are only available from mary era onwards.

ch1bo

Addressed all my concerns and this can be merged now IMO

Same rationale as previous commit, got renamed upstream.

Same rationale as previous two commits

These re-definitions are to avoid additional witness passing as long as we are just fine with the one specific / latest era (which we are by using Hydra.Cardano.Api). Consequently and according to the package README, these belong to the Api.hs module.

This avoids mentioning "Babbage" in downstream packages like hydra-cluster and will reduce the number of places to switch.

This reduces the number of times Babbage is mentioned in the hydra-node package and will help switching eras.

This removes "Requested index-state .. is newer than" warnings when building with cabal.

ch1bo requested changes Nov 20, 2023

View reviewed changes

locallycompact force-pushed the lc/plutus-1.15 branch 3 times, most recently from 036195e to d21fa60 Compare November 27, 2023 13:53

locallycompact force-pushed the lc/plutus-1.15 branch from d21fa60 to d1e51a7 Compare November 27, 2023 14:32

ch1bo linked an issue Nov 28, 2023 that may be closed by this pull request

Update to cardano-api 8.29 + plutus 1.15 #1114

Closed

locallycompact force-pushed the lc/plutus-1.15 branch from eb28a6b to fb5378d Compare November 30, 2023 10:26

locallycompact force-pushed the lc/plutus-1.15 branch from ee567e9 to 76946a2 Compare November 30, 2023 21:42

locallycompact changed the title ~~WIP: plutus: 1.9 -> 1.15, cardano-api: 8.20 -> 8.32~~ plutus: 1.9 -> 1.15.0.1, cardano-api: 8.20 -> 8.29.1 Nov 30, 2023

locallycompact force-pushed the lc/plutus-1.15 branch 3 times, most recently from 6f4866f to cbb1ab0 Compare December 4, 2023 11:39

locallycompact force-pushed the lc/plutus-1.15 branch 2 times, most recently from c1de8d0 to 7d6548e Compare December 4, 2023 12:36

locallycompact force-pushed the lc/plutus-1.15 branch from 7d6548e to 8fd95c7 Compare December 4, 2023 13:56

ch1bo force-pushed the lc/plutus-1.15 branch from 8fd95c7 to 211056c Compare December 4, 2023 17:11

ch1bo requested changes Dec 4, 2023

View reviewed changes

locallycompact force-pushed the lc/plutus-1.15 branch from 211056c to b21aae6 Compare December 5, 2023 11:00

ch1bo mentioned this pull request Dec 6, 2023

Crashing hydra-node #1193

Open

locallycompact and others added 8 commits December 6, 2023 11:36

plutus: 1.9 -> 1.15.0.1

b648a88

cardano-api: 8.20 -> 8.29.1

hydra-cardano-api: reexport createAndValidateTransactionBody

64f0351

hydra-cardano-api: reexport makeShelleyKeyWitness

9138ca7

Regenerate plutus scripts

699805a

hydra-plutus: split getHeadAddress into getHeadInput and getHeadAddress

5175d90

getHeadInput: Use case split instead of fromMaybe

cc9bc14

Regenerate plutus scripts

13d2ba1

locallycompact and others added 4 commits December 6, 2023 11:36

nix flake update

29015a3

LogFilterSpec: update txids

dce65b8

Only extract txId from JSON-serialised Tx in benchmarks

0e23fc7

Seems like we are having issues with inconsistent To/FromJSON serialisation following upgrade, so perhaps we should move our logs to use the "canonical" CBOR represenation of a Tx instead of semi-custom JSON?

ghost force-pushed the lc/plutus-1.15 branch from 9f07e92 to 0e23fc7 Compare December 6, 2023 10:36

Rename MultiAssetSupportedInEra -> MaryEraOnwards

6453095

This was renamed upstream and we should capture the meaning the same way. It is to provide witnesses for features which are only available from mary era onwards.

ch1bo force-pushed the lc/plutus-1.15 branch from 12d9604 to ccbb69a Compare December 6, 2023 13:58

ch1bo approved these changes Dec 6, 2023

View reviewed changes

ch1bo force-pushed the lc/plutus-1.15 branch from d65c619 to d5093d5 Compare December 6, 2023 15:42

ch1bo requested a review from a team December 6, 2023 15:43

ch1bo and others added 7 commits December 6, 2023 17:33

Rename HasInlineDatums -> BabbageEraOnwards

3acab95

Same rationale as previous commit, got renamed upstream.

Rename HasScriptData -> IsAlonzoEraOnwards

5da1e75

Same rationale as previous two commits

Create an Era-specific wrapper for defaultTxBodyContent

0dd4467

This avoids mentioning "Babbage" in downstream packages like hydra-cluster and will reduce the number of places to switch.

Create an Era-specific wrapper for signShelleyTransaction

427c115

This avoids mentioning "Babbage" in downstream packages like hydra-cluster and will reduce the number of places to switch.

Do not mention Babbage outside of hydra-cardano-api

f1e42cb

This reduces the number of times Babbage is mentioned in the hydra-node package and will help switching eras.

Use recent cabal index-states

03c185b

This removes "Requested index-state .. is newer than" warnings when building with cabal.

ch1bo force-pushed the lc/plutus-1.15 branch from d5093d5 to 03c185b Compare December 6, 2023 16:33

v0d1ch approved these changes Dec 8, 2023

View reviewed changes

locallycompact merged commit 06feee9 into master Dec 8, 2023
21 checks passed

locallycompact deleted the lc/plutus-1.15 branch December 8, 2023 10:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

plutus: 1.9 -> 1.15.0.1, cardano-api: 8.20 -> 8.29.1 #1168

plutus: 1.9 -> 1.15.0.1, cardano-api: 8.20 -> 8.29.1 #1168

locallycompact commented Nov 17, 2023

ch1bo left a comment

ch1bo Nov 20, 2023

ghost commented Nov 30, 2023

github-actions bot commented Dec 4, 2023 •

edited

Loading

github-actions bot commented Dec 4, 2023 •

edited

Loading

ch1bo left a comment

ch1bo Dec 4, 2023

ch1bo Dec 4, 2023

ch1bo left a comment

plutus: 1.9 -> 1.15.0.1, cardano-api: 8.20 -> 8.29.1 #1168

plutus: 1.9 -> 1.15.0.1, cardano-api: 8.20 -> 8.29.1 #1168

Conversation

locallycompact commented Nov 17, 2023

ch1bo left a comment

Choose a reason for hiding this comment

ch1bo Nov 20, 2023

Choose a reason for hiding this comment

ghost commented Nov 30, 2023

github-actions bot commented Dec 4, 2023 • edited Loading

Transactions Costs

Script summary

Cost of Init Transaction

Cost of Commit Transaction

Cost of CollectCom Transaction

Cost of Close Transaction

Cost of Contest Transaction

Cost of Abort Transaction

Cost of FanOut Transaction

End-To-End Benchmark Results

Baseline Scenario

Baseline Scenario

github-actions bot commented Dec 4, 2023 • edited Loading

Test Results

ch1bo left a comment

Choose a reason for hiding this comment

ch1bo Dec 4, 2023

Choose a reason for hiding this comment

ch1bo Dec 4, 2023

Choose a reason for hiding this comment

ch1bo left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 4, 2023 •

edited

Loading

github-actions bot commented Dec 4, 2023 •

edited

Loading