Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix logs schema #1244

Merged
merged 3 commits into from Jan 11, 2024
Merged

Fix logs schema #1244

merged 3 commits into from Jan 11, 2024

Conversation

ch1bo
Copy link
Member

@ch1bo ch1bo commented Jan 10, 2024

🐞 Fixes several broken JSON schemas from stateless observation changes (have not been released, hence no CHANGELOG update here)

🐞 Fixes and improves output of prop_validateJSONSchema such that it allowed me to find those schema mismatches each with 1-2 runs of --match "/Hydra.Logging/Validates logs.yaml schema/". The 5 successes and 100 sized test vectors seem to be a good trade-off.

  • CHANGELOG updated or not needed
  • Documentation updated or not needed
  • Haddocks updated or not needed
  • No new TODOs introduced or explained herafter

@ch1bo ch1bo added this to the 0.15.0 milestone Jan 10, 2024
@ch1bo ch1bo requested a review from a team January 10, 2024 10:28
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 10, 2024 •
edited

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-01-11 09:55:49.243020446 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial 985245919fcc6c0c5cd116023cd2c947c43e80dcbb5075fe12433fbb 4072
νCommit 7cb20fa71eb4c563ca283566ebe0aa65859d96c3f8cba35c52c181fd 2043
νHead 7a36661f5c15e9f1783aeaab890812c59b7286cbbc6de762d3110772 8816
μHead 8b111ac12274e46314769295a1c5dcab1d260096fc469fd698065463* 3851
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 4372 10.46 4.04 0.46
2 4576 12.57 4.83 0.49
3 4781 14.94 5.74 0.52
5 5177 19.22 7.35 0.59
10 6189 30.30 11.55 0.75
41 12417 99.02 37.59 1.77

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 534 11.37 4.44 0.30
2 718 15.04 6.07 0.35
3 906 18.85 7.75 0.40
5 1282 26.90 11.27 0.51
10 2225 49.55 20.97 0.80
19 3909 99.43 41.75 1.43

Cost of CollectCom Transaction

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 484 21.78 8.51 0.41
2 113 590 34.21 13.47 0.55
3 170 700 45.75 18.21 0.68
4 226 810 63.78 25.44 0.88
5 283 920 82.73 33.14 1.10
6 339 1031 93.58 37.96 1.23

Cost of Close Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 567 16.64 7.68 0.37
2 739 18.45 9.31 0.40
3 917 20.27 10.96 0.43
5 1208 23.18 13.77 0.49
10 2123 32.32 22.04 0.66
50 8685 97.47 82.72 1.92

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 564 20.18 8.94 0.40
2 774 22.30 10.71 0.44
3 927 24.23 12.38 0.48
5 1233 27.58 15.36 0.54
10 2101 37.58 23.82 0.72
43 7537 97.88 76.11 1.82

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4339 19.02 8.19 0.55
2 4506 31.87 13.90 0.71
3 4545 41.85 18.14 0.82
4 4765 59.73 26.10 1.03
5 5005 88.01 38.80 1.36

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
5 0 0 4206 8.05 3.37 0.42
5 1 57 4240 9.28 4.13 0.44
5 5 285 4376 14.40 7.27 0.51
5 10 567 4542 20.55 11.08 0.60
5 20 1138 4883 32.85 18.71 0.77
5 30 1708 5225 44.94 26.25 0.93
5 40 2276 5563 57.25 33.88 1.10
5 50 2845 5902 69.35 41.43 1.27
5 74 4215 6721 99.13 59.87 1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-01-11 09:58:40.037315429 UTC

Baseline Scenario

Number of nodes 3
Number of txs 9000
Avg. Confirmation Time (ms) 23.345994873
P99 113.38022407000005ms
P95 32.9747366ms
P50 20.7097945ms
Number of Invalid txs 0

Baseline Scenario

Number of nodes 1
Number of txs 3000
Avg. Confirmation Time (ms) 4.053925358
P99 7.303820499999983ms
P95 4.8601696ms
P50 3.9001314999999996ms
Number of Invalid txs 0

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 10, 2024 •
edited

Test Results

401 tests  ±0   394 ✅ ±0   16m 26s ⏱️ +18s
134 suites ±0     7 💤 ±0 
  5 files   ±0     0 ❌ ±0 

Results for commit 516f761. ± Comparison against base commit ff2182a.

♻️ This comment has been updated with latest results.

@ch1bo ch1bo force-pushed the fix-logs-schema branch 3 times, most recently from a0adf26 to 84251ae Compare January 10, 2024 14:18
abailly-iohk
abailly-iohk approved these changes Jan 11, 2024
View reviewed changes
Copy link
Contributor

@abailly-iohk abailly-iohk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mention 5 successes and lists of size 100 but it seems this comment is not true anymore? It's one success with a list of size 1000.

hydra-node/json-schemas/api.yaml
@@ -1555,7 +1555,7 @@ components:
tag:
type: string
enum: ["InvalidHeadId"]
headSeed:
headId:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@ch1bo @abailly-iohk
Fix shrinking in prop_validateJSONSchema and enable verbos output
26420d6 
Using mapSize was preventing shrinking and produced very big counter
examples. Adding -v allows to debug non-matching 'oneOf' schemas better.
@ch1bo @abailly-iohk
Fix several schemas in api.yaml and logs.yaml
66c770e
@ch1bo @abailly-iohk
Make prop_validateJSONSchema run with a fixed maxSuccess=1
516f761 
This also bumps up the number of generated values to only invoke the
external program once (if it passes). Shrinking will pick apart the big
list of values again.
@ch1bo ch1bo merged commit b0b8563 into master Jan 11, 2024
21 checks passed
@ch1bo ch1bo deleted the fix-logs-schema branch January 11, 2024 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@abailly-iohk abailly-iohk abailly-iohk approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.15.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ch1bo @abailly-iohk