-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Tooling] Add protocheck
CLI tool
#770
Conversation
8513f16
to
cc86b54
Compare
24b52bb
to
c8e195e
Compare
c8e195e
to
07f17cb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, one more thing I'd love this script to do though is to return non-zero exit code on make check_proto_unstable_marshalers
.
Currently:
❯ make check_proto_unstable_marshalers
INF Recursively checking for files matching "*.proto" in "./proto"
INF Found 52 unstable marshaler proto files:
INF proto/poktroll/gateway/query.proto
... more
❯ echo $?
0
Why: that will allow us to add make check_proto_unstable_marshalers
as a CI step which will act as a gate before we merge into the main
branch.
Nice observation! 😎 That was an oversight on my part. 🙇 Will fix. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bryanchriswhite Thanks for building the tool!
Just a couple things. Per my comment in #772, I'm thinking we could:
- Make it part of CI (or at least a TODO)
- Document it here: https://dev.poktroll.com/develop/developer_guide/debug_tips
…ke-transfer * pokt/main: [Relay Mining] Use per-service difficulty in the relayer (#745) [Tilt] Enable rest service by default in Tilt (#793) [Docs] Minor doc updates across the board (#792) build(deps): bump webpack from 5.89.0 to 5.94.0 in /docusaurus (#775) [Docs] Tokenomics Documentation (#750) fix: ensure stable_marshaler_all option on all proto files (#772) build(deps): bump micromatch from 4.0.5 to 4.0.8 in /docusaurus (#774) [Tooling] Add `protocheck` CLI tool (#770) Update `adding_params.md` to use `ignite` (#764) [Docs] Cleanup & Deprecation (#766)
…actor/transfer-msg_period-param * issues/657/feat/app-stake-transfer: chore: regenerate protobufs [Relay Mining] Use per-service difficulty in the relayer (#745) [Tilt] Enable rest service by default in Tilt (#793) [Docs] Minor doc updates across the board (#792) build(deps): bump webpack from 5.89.0 to 5.94.0 in /docusaurus (#775) [Docs] Tokenomics Documentation (#750) fix: ensure stable_marshaler_all option on all proto files (#772) build(deps): bump micromatch from 4.0.5 to 4.0.8 in /docusaurus (#774) [Tooling] Add `protocheck` CLI tool (#770) Update `adding_params.md` to use `ignite` (#764) [Docs] Cleanup & Deprecation (#766)
…ues/657/chore/app-transfer-period * issues/657/refactor/transfer-msg_period-param: chore: regenerate protobufs chore: regenerate protobufs fix: failing test chore: regenerate protobufs Revert "refactor: rename param to application_transfer_and_unboding_period_sessions" [Relay Mining] Use per-service difficulty in the relayer (#745) [Tilt] Enable rest service by default in Tilt (#793) [Docs] Minor doc updates across the board (#792) build(deps): bump webpack from 5.89.0 to 5.94.0 in /docusaurus (#775) [Docs] Tokenomics Documentation (#750) fix: ensure stable_marshaler_all option on all proto files (#772) build(deps): bump micromatch from 4.0.5 to 4.0.8 in /docusaurus (#774) [Tooling] Add `protocheck` CLI tool (#770) Update `adding_params.md` to use `ignite` (#764) [Docs] Cleanup & Deprecation (#766)
## Summary Add `protocheck` CLI tool and `unstable` subcommand to resolve stable serialization issue related to map protobuf message fields: `go run ./tools/scripts/protocheck/cmd unstable -h`: ``` Recursively list or fix all protobuf files which omit the 'stable_marshaler_all' option. Usage: protocheck unstable [flags] Flags: -f, --fix If present, protocheck will add the 'gogoproto.stable_marshaler_all' option to files which were discovered to be unstable. -h, --help help for unstable Global Flags: -p, --file-pattern string Set the pattern passed to filepath.Match(), used to include file names which match. (default "*.proto") -r, --root string Set the path of the directory from which to start walking the filesystem tree in search of files matching --file-pattern. (default "./proto") ``` ## Issue This mitigates non-deterministic serialization due to usage of map fields in protobuf message fields. It **does not** address any non-deterministic serialization due to mis-usage of `repeated` protobuf message fields, which is a much more complex concern. - #761 ## Type of change Select one or more: - [x] New feature, functionality or library - [ ] Bug fix - [ ] Code health or cleanup - [ ] Documentation - [ ] Other (specify) ## Testing - [ ] **Documentation**: `make docusaurus_start`; only needed if you make doc changes - [ ] **Unit Tests**: `make go_develop_and_test` - [ ] **LocalNet E2E Tests**: `make test_e2e` - [ ] **DevNet E2E Tests**: Add the `devnet-test-e2e` label to the PR. ## Sanity Checklist - [x] I have tested my changes using the available tooling - [x] I have commented my code - [x] I have performed a self-review of my own code; both comments & source code - [ ] I create and reference any new tickets, if applicable - [ ] I have left TODOs throughout the codebase, if applicable
Summary
Add
protocheck
CLI tool andunstable
subcommand to resolve stable serialization issue related to map protobuf message fields:go run ./tools/scripts/protocheck/cmd unstable -h
:Issue
This mitigates non-deterministic serialization due to usage of map fields in protobuf message fields.
It does not address any non-deterministic serialization due to mis-usage of
repeated
protobuf message fields, which is a much more complex concern.Type of change
Select one or more:
Testing
make docusaurus_start
; only needed if you make doc changesmake go_develop_and_test
make test_e2e
devnet-test-e2e
label to the PR.Sanity Checklist