feat(storage/metrics): Metrics to measure storage #1974

jmalicevic · 2024-01-05T18:11:17Z

This PR superceeds #79 with some adjustments to the segments of code timed as well as bucket sizes. The majority of the code was done by William in #79. I tried to fine tune the measurements to exclude proto marshalling/unmarshalling where I think it made sense.

Closes #46

Blocked on benchmarking to confirm it is measuring what we want.
(follow up) The metrics gave us nice and realistic measurements in our benchmarks.

PR checklist

Tests written/updated
Changelog entry added in .changelog (we use unclog to manage our changelog)
Updated relevant documentation (docs/ or spec/) and code comments

This reverts commit 68c48e8.

melekes

nice 👍

cmd/cometbft/commands/inspect.go

internal/state/store.go

internal/store/store.go

github-actions · 2024-01-22T00:14:44Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

…metrics

…ible

jmalicevic · 2024-01-26T15:07:56Z

Do we maybe want documentation on the metrics, the methods they cover? At the moment there are two metrics :

block_store_access_time
state_store_access_time

But the time is measured individually per method accessing the storage. And some methods include others (save_blocks includes the time tow rite and marshall all block parts). So maybe documenting this for users might be benefitial?

greg-szabo

Great work here! I made a few comments in the StateStore. I kind of lost track in the middle of BlockStore so I'll have another go at it. Until then, I think some of the comments are worth a bit of discussion.

internal/state/store.go

greg-szabo · 2024-01-29T18:48:01Z

internal/state/store.go

 	if from <= 0 || to <= 0 {
 		return fmt.Errorf("from height %v and to height %v must be greater than 0", from, to)
 	}
 	if from >= to {
 		return fmt.Errorf("from height %v must be lower than to height %v", from, to)
 	}

-	valInfo, err := loadValidatorsInfo(store.db, min(to, evidenceThresholdHeight))
+	valInfo, elapsedTime, err := loadValidatorsInfo(store.db, min(to, evidenceThresholdHeight))


I'm not sure about the patterns used in our codebase, but it looks to me that loadValidatorsInfo doesn't benefit from elapsedTime at all and maybe the measurement should happen in this (PruneStates) function, not in the loadValidatorsInfo.

I can see that the unmarshaling of the data from the database is not measured, which gives us a more precise measurement for database activity. However, this is not indicated in loadValidatorsInfo. Maybe a solution is to document that the result is the elapsed time for database access or something. But maybe we should move this measurement to PruneStates and accept that the unmarshaling is also part of the measurement.

The point was to avoid measuring marshalling/unmarshalling to make sure the measurements include only DB access time. You are however right that the function loadValidatorsInfo itself will not report this. The function has no object that has metrics associated with it to enter the value. That is why I report the time spent in this function only within functions that call it.

greg-szabo · 2024-01-29T18:49:57Z

internal/state/store.go

@@ -333,7 +365,8 @@ func (store dbStore) PruneStates(from int64, to int64, evidenceThresholdHeight i
 		// params, otherwise they will panic if they're retrieved directly (instead of
 		// indirectly via a LastHeightChanged pointer).
 		if keepVals[h] {
-			v, err := loadValidatorsInfo(store.db, h)
+			v, tmpTime, err := loadValidatorsInfo(store.db, h)


Again, I lack some context, but do we allow variable shadowing? err is previously defined in the function.

So this was here from before, I don't think we have explicit rules but given that on any error before or after we return immediately it can pass here.. It would be messier to declare the vars above just to avoid overwriting this one.

greg-szabo · 2024-01-29T18:54:52Z

internal/store/store.go

+type BlockStoreOption func(*BlockStore)
+
+// WithMetrics sets the metrics.
+func WithMetrics(metrics *Metrics) BlockStoreOption {


I like this way better than the NewStore function in StateStore. Maybe I'm too Rust-oriented.

Not Rust oriented, we have this Builder pattern in several places in the codebase including the RPC APIs, I think it's a good pattern too and it helps with unit testing.

andynog

lgtm, I've added a few comments but more like suggestions or questions to ensure nothing is breaking.

.changelog/unreleased/features/1974-statestore-metrics.md

.changelog/unreleased/features/1974-store-metrics.md

internal/state/store.go

internal/store/store.go

.changelog/unreleased/features/1974-statestore-metrics.md

internal/store/store.go

internal/state/store.go

…metrics

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

melekes · 2024-02-22T16:31:48Z

@mergify backport v1.x

mergify · 2024-02-22T16:31:52Z

backport v1.x

✅ Backports have been created

#2411 feat(storage/metrics): Metrics to measure storage (backport #1974) has been created for branch v1.x but encountered conflicts

This PR superceeds #79 with some adjustments to the segments of code timed as well as bucket sizes. The majority of the code was done by William in #79. I tried to fine tune the measurements to exclude proto marshalling/unmarshalling where I think it made sense. Closes #46 Blocked on benchmarking to confirm it is measuring what we want. (follow up) The metrics gave us nice and realistic measurements in our benchmarks.  --- #### PR checklist - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments --------- Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> (cherry picked from commit dfd3f6c) # Conflicts: # internal/store/store.go

…2411) This is an automatic backport of pull request #1974 done by [Mergify](https://mergify.com). Cherry-pick of dfd3f6c has failed: ``` On branch mergify/bp/v1.x/pr-1974 Your branch is up to date with 'origin/v1.x'. You are currently cherry-picking commit dfd3f6c. (fix conflicts and run "git cherry-pick --continue") (use "git cherry-pick --skip" to skip this patch) (use "git cherry-pick --abort" to cancel the cherry-pick operation) Changes to be committed: new file: .changelog/unreleased/features/1974-statestore-metrics.md new file: .changelog/unreleased/features/1974-store-metrics.md modified: internal/state/export_test.go modified: internal/state/metrics.gen.go modified: internal/state/metrics.go modified: internal/state/store.go new file: internal/store/metrics.gen.go new file: internal/store/metrics.go modified: node/node.go modified: node/setup.go modified: test/loadtime/basic.sh Unmerged paths: (use "git add <file>..." to mark resolution) both modified: internal/store/store.go ``` To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally --- <details> <summary>Mergify commands and options</summary> <br /> More conditions and actions can be found in the [documentation](https://docs.mergify.com/). You can also trigger Mergify actions by commenting on this pull request: - `@Mergifyio refresh` will re-evaluate the rules - `@Mergifyio rebase` will rebase this PR on its base branch - `@Mergifyio update` will merge the base branch into this PR - `@Mergifyio backport <destination>` will backport this PR on `<destination>` branch Additionally, on Mergify [dashboard](https://dashboard.mergify.com) you can: - look at your merge queues - generate the Mergify configuration with the config editor. Finally, you can contact us on https://mergify.com </details> --------- Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

- **Versioned protobuf files (#495)** - **Restore rpc/grpc/v1/types.pb.go** - **proto: restore deprecation notice on BroadcastAPI** - **Struct `Client` exposes sensitive data (#784)** - **Unsafe int cast in `kill` command (#783)** - **ADR-100: Data Companion Push API (#73)** - **rfc: incoming txs while catching up (#735)** - **build(deps): Bump bufbuild/buf-setup-action from 1.17.0 to 1.18.0 (#804)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.8 to 1.4.9 (#806)** - **build(deps): Bump golang.org/x/sync from 0.1.0 to 0.2.0 (#807)** - **build(deps): Bump google.golang.org/grpc from 1.54.0 to 1.55.0 (#813)** - **build(deps): Bump github.com/prometheus/client_golang from 1.15.0 to 1.15.1 (#810)** - **build(deps): Bump github.com/bufbuild/buf from 1.17.0 to 1.18.0 (#809)** - **build(deps): Bump github.com/bufbuild/buf from 1.17.0 to 1.18.0 (#809)** - **build(deps): Bump github.com/prometheus/client_model from 0.3.0 to 0.4.0 (#812)** - **build(deps): Bump github.com/prometheus/common from 0.42.0 to 0.43.0 (#811)** - **ADR-103: Protobuf definition versioning (#772)** - **spec/p2p: Specify the operation of a Reactor (#714)** - **build(deps): Bump github.com/docker/distribution (#827)** - **build(deps): Bump github.com/cloudflare/circl from 1.3.1 to 1.3.3 (#828)** - **ADR-103: Protobuf definition versioning (#772) (#817)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.9 to 1.4.10 (#830)** - **build(deps): Bump golang.org/x/net from 0.9.0 to 0.10.0 (#831)** - **build(deps): Bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#832)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.26.1 to 2.27.1 (#833)** - **rpc: Remove response data from response failure logs (#829)** - **pubsub/kvindexer:support for big numbers - v2 (#797)** - **Extend the infrastructure provider with `StartNodes` and `StopTestnet` (#796)** - **grpc: Add base gRPC server with version service (#818)** - **Digital Ocean implementation of `StartNodes` and `StopTestnet` (#846)** - **RFC 102: Improve forward compatibility of proto-generated Rust (#724)** - **mempool: Add metric to measure how many times a tx was received (#637)** - **`e2e` provider is extended with `Disconnect`, `Reconnect` and `CheckUpgraded` (#852)** - **Sort `loadtime` tool's report (#854)** - **feat: make handshake cancelable (#857)** - **proto: Restore snake_case JSON for `ExecTxResult` (#856)** - **qa: 200-nodes test on v0.38 (#877)** - **qa: rotating node test on `v0.38.x` (#883)** - **Changes to QA related files (e.g., `method.md`) (#878)** - **build(deps): Bump bufbuild/buf-setup-action from 1.18.0 to 1.19.0 (#868)** - **build(deps): Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#873)** - **build(deps): Bump github.com/bufbuild/buf from 1.18.0 to 1.19.0 (#872)** - **Adds `vote_extension_size` to manifests (#858)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.27.1 to 2.28.1 (#893)** - **build(deps): Bump github.com/prometheus/common from 0.43.0 to 0.44.0 (#892)** - **build(deps): Bump slackapi/slack-github-action from 1.23.0 to 1.24.0 (#869)** - **ADR 104: State sync from local application snapshot (#801)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 (#891)** - **mempool: slight refactor for improving readability (#894)** - **Add Vote Extension varying size testnet (#888)** - **fix: lint fail when golangci-lint bump from v1.52.2 to v1.53.x (#908)** - **build(deps): Bump github.com/bufbuild/buf from 1.19.0 to 1.20.0 (#912)** - **build(deps): Bump github.com/golangci/golangci-lint (#917)** - **build(deps): Bump github.com/spf13/viper from 1.15.0 to 1.16.0 (#913)** - **build(deps): Bump bufbuild/buf-setup-action from 1.19.0 to 1.20.0 (#910)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.28.1 to 2.28.2 (#914)** - **Minor fixes to `RELEASES.md` (#923)** - **Prevent a transaction to appear twice in the mempool (#890)** - **New metrics to track duplicate votes and block parts (#896) (#905)** - **e2e tests are being skipped; `INFRASTRUCTURE-*` bug; portGen inconsistency bug (#933)** - **build(deps): Bump github.com/bufbuild/buf from 1.20.0 to 1.21.0 (#948)** - **build(deps): Bump bufbuild/buf-setup-action from 1.20.0 to 1.21.0 (#946)** - **spec/p2p: document the p2p API used by Reactors (#851)** - **build(deps): Bump docker/login-action from 2.1.0 to 2.2.0 (#944)** - **e2e: Generate prometheus.yaml on setup (#954)** - **Update the type annotations in the light client spec (#955)** - **build(deps): Bump docker/build-push-action from 4.0.0 to 4.1.0 (#947)** - **build(deps): Bump docker/setup-buildx-action from 2.5.0 to 2.7.0 (#962)** - **Add requirement for `CheckTx` in ABCI spec (#928)** - **fix: avoid recursive call after rename to (*PeerState).MarshalJSON (#865)** - **Port upstream proto changes to cometbft package** - **changelog: Import entries for v0.34.28, v0.34.29, v0.37.1 and v0.37.2 to `main` (#974)** - **test/e2e: Fix Docker image build (#984)** - **build(deps): Bump github.com/BurntSushi/toml from 1.3.0 to 1.3.2 (#952)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.28.2 to 2.29.0 (#953)** - **Remove duplicate function `deterministicExecTxResult` (#986)** - **mempool: Fix the benchmarks (#934)** - **Add `CMT_HOME` (or remove it?) (#983)** - **Remove buf.yaml at the root of the source tree (#979)** - **build(deps): Bump docker/build-push-action from 4.1.0 to 4.1.1 (#989)** - **build(deps): Bump golang.org/x/net from 0.10.0 to 0.11.0 (#994)** - **build(deps): Bump golang.org/x/sync from 0.2.0 to 0.3.0 (#990)** - **build(deps): Bump google.golang.org/grpc from 1.55.0 to 1.56.0 (#992)** - **build(deps): Bump github.com/golangci/golangci-lint from 1.53.2 to 1.53.3 (#996)** - **build(deps): Bump github.com/prometheus/client_golang from 1.15.1 to 1.16.0 (#991)** - **spec/p2p: new structure for the p2p specification (#966)** - **consensus: optimize vote and block part gossip with HasProposalBlockPartMessage and random sleeps (#904)** - **Extend ABCI `max_block_size` parameter to give extended control to the app (#1003)** - **build(deps): Bump bufbuild/buf-setup-action from 1.21.0 to 1.22.0 (#1026)** - **Bump logs on non-fatal vote validation failures to Info level (#1022)** - **Update codeowners to include DevRel team (#1023)** - **deps: Manually bump github.com/vektra/mockery/v2 from 2.29.0 to 2.30.1 (#1009)** - **Update releasing.md to sign tags (#1034)** - **build(deps): Bump github.com/bufbuild/buf from 1.21.0 to 1.22.0 (#1027)** - **Rename ABCI enums and values to satisfy buf guidelines (#975)** - **build(deps): Bump google.golang.org/grpc from 1.56.0 to 1.56.1 (#1028)** - **docs: Added double quotes to /abci_query path param (#1015)** - **Update Docs with Finalize Block (#760)** - **Clarifies that processProposal may be called for set of transactions different from the one returned in the preceding prepareProposal (#1033)** - **Update the annotation of part_set.go (#1056)** - **build(deps): Bump docker/setup-buildx-action from 2.7.0 to 2.8.0 (#1071)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.30.1 to 2.30.16 (#1064)** - **build(deps): Bump bufbuild/buf-setup-action from 1.22.0 to 1.23.1 (#1072)** - **build(deps): Bump github.com/bufbuild/buf from 1.22.0 to 1.23.1 (#1066)** - **build(deps): Bump google.golang.org/protobuf from 1.30.0 to 1.31.0 (#1065)** - **build(deps): Bump google.golang.org/grpc from 1.56.1 to 1.56.2 (#1101)** - **build(deps): Bump golang.org/x/crypto from 0.10.0 to 0.11.0 (#1103)** - **build(deps): Bump docker/setup-buildx-action from 2.8.0 to 2.9.0 (#1105)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.30.16 to 2.31.1 (#1100)** - **build(deps): Bump golang.org/x/net from 0.11.0 to 0.12.0 (#1102)** - **ci: Trigger workflows on merge group (#1118)** - **Revert "config: add bootstrap peers (#9680)" (#1109)** - **node: Revert removal of public reactor accessors (#1120)** - **ci: Disable CodeQL check in merge queues (#1123)** - **p2p: Remove UPnP functionality (#1114)** - **ADR 107: Rename proto versions to pre-v1 betas (#1110)** - **RFC 104: Internal messaging using the actor model (#1092)** - **build(deps): Bump github.com/bufbuild/buf from 1.23.1 to 1.24.0 (#1131)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.31.1 to 2.32.0 (#1132)** - **build(deps): Bump docker/setup-buildx-action from 2.9.0 to 2.9.1 (#1133)** - **build(deps): Bump bufbuild/buf-setup-action from 1.23.1 to 1.24.0 (#1134)** - **spec: Add mempool specification in English and Quint (#997)** - **mempool: ADR for refactoring list of senders (#1032)** - **build(deps): Bump bufbuild/buf-setup-action from 1.24.0 to 1.25.0 (#1157)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.7.0 to 5.8.0 (#1159)** - **build(deps): Bump github.com/bufbuild/buf from 1.24.0 to 1.25.0 (#1160)** - **proxy: Rename "unsynchronized" to "connection-synchronized" local client creator (#1145)** - **mempool: Keep track of senders in reactor instead of implementation (#1010)** - **cmd: Remove `replay` and `replay-console` subcommands (#1170)** - **build(deps): Bump google.golang.org/grpc from 1.56.2 to 1.57.0 (#1181)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.8.0 to 5.8.1 (#1180)** - **ci: Remove Mergify automerge (#1182)** - **Add gRPC block service (#1142)** - **blocksync: export errors (#1186)** - **Porting changes related to creating a lean docker image from `pierre/fast-prototyping-1059` branch. (#1192)** - **chore: Format repo (#1193)** - **Add BlockResults gRPC service (#1168)** - **consensus: remove logic to unlock block on +2/3 prevote for nil (#1175)** - **ADR 101: Add GetLatest method to block service (#1209)** - **ADR 101: Implement pruning mechanism (#1150)** - **config: export errors (#1190)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.0 to 2.32.3 (#1218)** - **Add check for non-`nil` in `enterCommit` (#1208)** - **Log proposer's address when correctly accepting a proposal (#1079)** - **Close evidence.db OnStop (#1210)** - **build(deps): Bump bufbuild/buf-setup-action from 1.25.0 to 1.25.1 (#1213)** - **build(deps): Bump golang.org/x/crypto from 0.11.0 to 0.12.0 (#1215)** - **build(deps): Bump github.com/bufbuild/buf from 1.25.0 to 1.25.1 (#1216)** - **build(deps): Bump golang.org/x/net from 0.12.0 to 0.14.0 (#1217)** - **Forward-port: update state to prevote `nil` when proposal block does not match locked block (#1203)** - **mempool: Store peer ids as p2p.ID instead of uint16 (remove mempoolIDs) (#1191)** - **ADR-101: implement gRPC `PruningService` (#1154)** - **consensus: Avoid process proposal when we know correct nodes validated it (#1231)** - **ADR-101: Metrics to monitor the pruning (#1234)** - **build(deps): Bump bufbuild/buf-setup-action from 1.25.1 to 1.26.0 (#1241)** - **build(deps): Bump github.com/bufbuild/buf from 1.25.1 to 1.26.1 (#1239)** - **build(deps): Bump github.com/golangci/golangci-lint (#1240)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.3 to 2.32.4 (#1242)** - **abci: export errors (#1185)** - **consensus: export errors (#1211)** - **ADR-101: Pruning mechanism minor fixes (#1246)** - **chore: Bump minimum Go version on `main` to v1.21 (#1244)** - **ADR 101: Add `Close` method to gRPC client (#1251)** - **ADR 108: ADR for extending E2E infrastructure so that we can check CometBFT's behaviour with respect to ABCI++ grammar. (#902)** - **docs: Add logging guide to contributing guidelines (#1250)** - **chore: log `app_hash` as hex (#1264)** - **update language (#1263)** - **build(deps): Bump bufbuild/buf-setup-action from 1.26.0 to 1.26.1 (#1272)** - **build(deps): Bump gonum.org/v1/gonum from 0.13.0 to 0.14.0 (#1275)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.10 to 1.4.11 (#1276)** - **ADR 101: Refactor height check-related logic and tests (#1271)** - **Fixed linter** - **Revert "Fixed linter"** - **Fixed mem aliasing in for loop (#1280)** - **Remove genesis persistence in state db (#1017)** - **Revert "Remove genesis persistence in state db (#1017)" (#1294)** - **evidence: export errors (#1284)** - **Re-implement removal of genesis persistence in state db (#1295)** - **build(deps): Bump docker/setup-buildx-action from 2.9.1 to 2.10.0 (#1298)** - **node: verify genesis doc hash against the file contents rather than remarshalled JSON (#1293)** - **indexer: Implement pruning mechanism (#1176)** - **Provide relevant block data in `ExtendVote` (#1270)** - **node/state:bootstrap state api (#1057)** - **build(deps): Bump actions/checkout from 3 to 4 (#1319)** - **build(deps): Bump docker/build-push-action from 4.1.1 to 4.2.1 (#1318)** - **Fix (#1323)** - **state: remove genesis file from database (#1297)** - **Added changelog for pruning metrics (#1335)** - **Removed begin_block_events and end_block_events from rpc doc (#1338)** - **consensus: test for precommit/locking corner case (#1257)** - **TxIndexer and BlockIndexer pruning metrics (#1334)** - **Forward port changes to changelog and `UPGRADING.md` for release `v0.38.0` (#1341)** - **Update Go version specified in root README on main for v0.34.x (#1352)** - **Add test/e2e/data to gitignore (#1349)** - **docs: gRPC services and data companion pruning services (#1307)** - **Add genesis_hash flag check on node startup (#1324)** - **gRPC for Tx and Block indexer pruning (#1327)** - **build(deps): Bump docker/build-push-action from 4.2.1 to 5.0.0 (#1371)** - **build(deps): Bump docker/setup-buildx-action from 2.10.0 to 3.0.0 (#1370)** - **build(deps): Bump docker/login-action from 2.2.0 to 3.0.0 (#1368)** - **build(deps): Bump goreleaser/goreleaser-action from 4 to 5 (#1369)** - **ADR101: Moved changelog entries (#1377)** - **Max byte check (#1384)** - **Update to string (#1385)** - **mempool: Remove unused peerID constants (#1379)** - **build(deps): Bump github.com/google/uuid from 1.3.0 to 1.3.1 (#1392)** - **build(deps): Bump github.com/rs/cors from 1.9.0 to 1.10.0 (#1393)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.8.1 to 5.9.0 (#1396)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.4 to 2.34.0 (#1400)** - **build(deps): Bump github.com/go-kit/kit from 0.12.0 to 0.13.0 (#1397)** - **build(deps): Bump github.com/golangci/golangci-lint (#1398)** - **build(deps): Bump google.golang.org/grpc from 1.57.0 to 1.58.2 (#1399)** - **doc: improve documentation of BlockParams.MaxBytes (#1405)** - **rpc: Improve abci_query parameter documentation (#1184)** - **Bump p2p version following introduction of `HasProposalBlockPartMessage` (#1411)** - **RFC: Allowing Non-Determinism in `ProcessProposal` (#1391)** - **ADR-101: Data Companion Pull API (#82)** - **build(deps): Bump github.com/rs/cors from 1.10.0 to 1.10.1 (#1418)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.34.0 to 2.34.2 (#1416)** - **build(deps): Bump github.com/prometheus/client_golang (#1417)** - **fix typos (#1413)** - **Renames Tendermint to CometBFT in the RPC related test code (#1428)** - **Renames Tendermint to CometBFT in the RPC related test code. This file was missing and it was not caught in the branch testing. (#1433)** - **Reject incoming txs while node is catching up (#1119)** - **build(deps): Bump golang.org/x/crypto from 0.13.0 to 0.14.0 (#1458)** - **build(deps): Bump golang.org/x/sync from 0.3.0 to 0.4.0 (#1454)** - **build(deps): Bump golang.org/x/net from 0.15.0 to 0.16.0 (#1453)** - **build(deps): Bump bufbuild/buf-setup-action from 1.26.1 to 1.27.0 (#1449)** - **build(deps): Bump styfle/cancel-workflow-action from 0.11.0 to 0.12.0 (#1450)** - **build(deps): Bump github.com/prometheus/client_model (#1455)** - **build(deps): Bump github.com/bufbuild/buf from 1.26.1 to 1.27.0 (#1459)** - **build(deps): Bump github.com/spf13/viper from 1.16.0 to 1.17.0 (#1456)** - **test: respect P2PConfig fuzzing configuration in MultiplexTransport (#1414)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.34.2 to 2.35.3 (#1462)** - **crypto: export errors (#1463)** - **Do not race with pruning to validate headers and validators (#1467)** - **fix typos (#1487)** - **Call observer only when retain height changes (#1490)** - **build(deps): Bump pillow from 9.3.0 to 10.0.1 in /scripts/qa/reporting (#1493)** - **abci: Add relaxed local client synchronization models (#1141)** - **docs: fix typos (#1500)** - **Post-merge fixes for feature/proto-upgrade** - **Make BuildLastCommitInfo and BuildExtendedCommitInfo public (#1502)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.35.3 to 2.35.4 (#1497)** - **build(deps): Bump google.golang.org/grpc from 1.58.2 to 1.58.3 (#1498)** - **build(deps): Bump golang.org/x/net from 0.16.0 to 0.17.0 (#1499)** - **tiny change: reorder expected and actual values (#1506)** - **Fixed docker port alias (#1507)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.0 to 1.27.1 (#1527)** - **Implement ADR-107: rename proto package version suffixes to v1beta* pattern (#1510)** - **e2e: Fix flakiness in grpc tests due to pruning (#1492)** - **fix spelling of bandwidth (#1534)** - **Fix linting on `main` (#1531)** - **build(deps): Bump github.com/prometheus/common from 0.44.0 to 0.45.0 (#1524)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.0 to 1.27.1 (#1522)** - **build(deps): Bump github.com/golangci/golangci-lint (#1540)** - **fix some typos (#1541)** - **indexer-respect-height-params-on-query (#1529)** - **Correct go_package paths in service proto files (#1538)** - **build(deps): Bump google.golang.org/grpc from 1.58.3 to 1.59.0 (#1521)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.35.4 to 2.36.0 (#1523)** - **feat(mempool): export error (#1427)** - **Rename proto messages and services to satisfy default buf lints (#1533)** - **RFC 106: non-idempotent methods in data companion (#1545)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.9.0 to 5.10.0 (#1550)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.1 to 1.27.2 (#1551)** - **build(deps): Bump github.com/google/uuid from 1.3.1 to 1.4.0 (#1552)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.1 to 1.27.2 (#1553)** - **use assert.Greater instead of Equal for better error reporting (#1547)** - **rpc: Version the RPC APIs (#1412)** - **mempool: Add metric size of pool in bytes (#1512)** - **build(deps): Bump github.com/docker/docker (#1570)** - **build(deps): Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 (#1572)** - **build(deps): Bump golang.org/x/sync from 0.4.0 to 0.5.0 (#1574)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.36.0 to 2.36.1 (#1576)** - **build(deps): Bump github.com/gorilla/websocket from 1.5.0 to 1.5.1 (#1575)** - **build(deps): Bump github.com/golangci/golangci-lint (#1573)** - **chore: consolidate changelog improvements section (#1578)** - **Implementation of ADR-108 (#930)** - **Reduce the default MaxBytes to 4mb and increase MaxGas to 10 million (#1518)** - **e2e: Allow disabling the PEX reactor on all nodes in the testnet (#1580)** - **Experimental - Reduce # of connections effectively used to gossip transactions out (#1558)** - **Update proto file references to follow renaming/versioning (#1555)** - **Make `LoadBlock` also return block metadata (#1557)** - **mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584)** - **fix: docs: default db provider moved from node to config (#1588)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.2 to 1.28.0 (#1598)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.2 to 1.28.0 (#1602)** - **build(deps): Bump golang.org/x/crypto from 0.14.0 to 0.15.0 (#1599)** - **build(deps): Bump golang.org/x/net from 0.17.0 to 0.18.0 (#1597)** - **RFC 107: Internal signalling using event observers (#1164)** - **ADR 110: Remote mempool (#1565)** - **chore: aligns function descriptions with godoc standards in the types package (#1593)** - **docs: various improvements (#1603)** - **chore: Merge branch main into feature/proto-upgrade (#1612)** - **Removes wrong assertion and decreases likelyhood of block from the future has actually been already created (#1619)** - **Split /api off to separate go.mod (#1608)** - **feature: Reduce Go API surface area (ADR 109) (#1605)** - **refactor: Export func MakeHTTPDialer (#1594)** - **First draft (#1483)** - **pruning:do not attempt to prune state if no blocks are pruned (#1616)** - **Update SECURITY.md (#1626)** - **Update changelog on `main` with new releases (#1644)** - **proto: Update README (#1648)** - **ADR 111: `nop` Mempool (#1585)** - **ADR-111: Addressed Ethan's comment (#1667)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.36.1 to 2.37.1 (#1662)** - **build(deps): Bump docker/build-push-action from 5.0.0 to 5.1.0 (#1664)** - **build(deps): Bump bufbuild/buf-setup-action from 1.28.0 to 1.28.1 (#1665)** - **ci: Fix v0.34 nightly build (#1655)** - **build(deps): Bump github.com/bufbuild/buf from 1.28.0 to 1.28.1 (#1663)** - **feature/proto-upgrade: Fix generated mock code (#1673)** - **Update link in README.md (#1674)** - **Renames the semantic versioning variable to CMT (#1621)** - **mempool: add `nop` mempool (#1643)** - **e2e:fix-digital-ocean-ports (#1678)** - **Revert modularization of the test infra (#1488) (#1676)** - **docs: various small improvements (part 2) (#1683)** - **Do not block indefinitely on the semaphore (#1654)** - **Adds tests that check for FIFO ordering being broken by gossip (#1628)** - **loadtime: Add parameter for displaying results in one line (#1511)** - **Promote latest versions of Protobuf definitions to v1 packages (#1677)** - **chore: Update CHANGELOG.md (#1701)** - **build(deps): Bump github.com/google/uuid from 1.3.1 to 1.4.0 (#1696)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.37.1 to 2.38.0 (#1697)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.9.0 to 5.10.1 (#1702)** - **proto: document the `cometbft.*` versioned protos (#1694)** - **Clean up changelog entries and README files for versioned proto changes (#1707)** - **chore(proto): add missing docs (#1703)** - **feature/proto-upgrade: Last-minute fixes (#1709)** - **e2e: Support emulated latencies between docker images (#1560)** - **build(deps): Bump golang.org/x/net from 0.14.0 to 0.17.0 in /api (#1713)** - **Prepare `main` for `v1.x` backport branch (#1714)** - **build(deps): Bump golang.org/x/crypto from 0.15.0 to 0.16.0 (#1720)** - **build(deps): Bump golang.org/x/net from 0.18.0 to 0.19.0 (#1719)** - **types: validate Validator#Address field (#1715)** - **Enable Mergify backports for `v1.x` branch (#1724)** - **deps: Bump cometbft-db to v0.9.0 (#1725)** - **buf improvements prior to publishing on BSR (#1726)** - **proto: give our buf module a name (#1734)** - **deps: Bump cometbft-db to v0.9.1 (#1737)** - **chore: Update root docs (#1740)** - **fix: increase abci socket message size limit to 2GB (#1730)** - **proto: make comment in README more Markdowny (#1743)** - **fix: Txs Validate (#1687)** - **Add test missing in #1687 (#1712)** - **perf(state): batch save `State` (#1735)** - **Update CODE_OF_CONDUCT.md (#1708)** - **[e2e] Fixes prepareProposal not to return oversized set of transactions (#1756)** - **perf(store): Batch save `Block` (#1755)** - **Fix: Fix minor typos (#1780)** - **`VerifyCommitLight` and `VerifyCommitLightTrusting` _never_ check all signatures (#1750)** - **Fix typo in encoding.md (#1801)** - **chore(docs): small improvements (#1781)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.10.1 to 5.11.0 (#1797)** - **build(deps): Bump actions/setup-go from 4 to 5 (#1794)** - **build(deps): Bump actions/stale from 8 to 9 (#1795)** - **build(deps): Bump github.com/spf13/viper from 1.17.0 to 1.18.1 (#1796)** - **Chore/fix other typos (#1809)** - **chore(spec): specify which fields must be deterministic (#1804)** - **Small improvements in #1806 not present in #1750 (#1808)** - **Add changelog for #1749 (#1807)** - **e2e: Implement latency emulation for DigitalOcean (#1587)** - **add GH workflow for linting code using codespell (#1824)** - **chore(docs): explain the effect of `timeout_propose` (#1798)** - **build(deps): Bump actions/upload-artifact from 3 to 4 (#1845)** - **build(deps): Bump actions/setup-python from 2 to 5 (#1843)** - **build(deps): Bump github/codeql-action from 2 to 3 (#1847)** - **build(deps): Bump actions/checkout from 2 to 4 (#1846)** - **build(deps): Bump github.com/google/uuid from 1.4.0 to 1.5.0 (#1850)** - **build(deps): Bump google.golang.org/grpc from 1.59.0 to 1.60.0 (#1851)** - **golangci-lint: enable all the linters by default (#1838)** - **Updates go crypto package to v0.17.0 (#1859)** - **Adds checks for nil keys and signatures (#1855)** - **The store DB and the companion `BlockStore` struct are sometimes out of sync (#1856)** - **Allow blocksync to not verify all signatures (#1858)** - **Fixing the ABCI grammar and updating the code for e2e tests to account for the new grammar. (#1829)** - **docs: Fix Discord links in README (#1874)** - **build(deps): Bump github.com/prometheus/client_golang (#1909)** - **build(deps): Bump github.com/btcsuite/btcd/btcutil from 1.1.3 to 1.1.5 (#1908)** - **build(deps): Bump google.golang.org/protobuf (#1885)** - **build(deps): Bump github.com/spf13/viper from 1.18.1 to 1.18.2 (#1884)** - **build(deps): Bump google.golang.org/grpc from 1.60.0 to 1.60.1 (#1883)** - **remove unused linters that produced a warning and ensure that unparam is enabled (#1905)** - **Fix HTTP response body not being closed after reading (#1945)** - **Disable undesired linting code (#1959)** - **Extend kvstore example add with with key types (#1876)** - **Extra check in `VerifyExtension` (#1877)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.38.0 to 2.39.1 (#1882)** - **perf(crypto/merkle, crypto/tmhash): simplify+optimize SHA256 hashing of multiple byteslices (#1921)** - **Update localnet-start Makefile (#1975)** - **blocksync: wait for poolRoutine to stop in (*Reactor).OnStop (#1879)** - **fix TestListenerTimeoutReadWrite (#1947)** - **docs: fix typos (#1988)** - **this is the kindest commit I can think of. Take it that way and read the code. (#1987)** - **there's really no need for this anymore (#1986)** - **feat: DefaultHttpClient support setting proxy from env (#1900)** - **feat: lint tests (#1906)** - **chore: fix typos (#1991)** - **build(deps): Bump golang.org/x/sync from 0.5.0 to 0.6.0 (#1992)** - **feat(proxy): export errors (#1899)** - **doc:fix broken link in DOCKER/README.md (#1996)** - **feat: custom dependencies order (#1994)** - **PBTS: migrating to `main` the new version of the specification (#1973)** - **Fix paths to scripts/metricsgen in go:generate commands (#1998)** - **chore: fix some typos (#2002)** - **docs(light): fix broken link to spec (#2007)** - **chore(test/loadtime): replace tm-load-test w/ cometbft-load-test (#2009)** - **ci: Remove unused `tests` target when compiling e2e (#2022)** - **spec: fix typo in `encoding.md` (#2024)** - **mempool: Fix data races in CListMempool's height and notifiedTxsAvailable (#2021)** - **chore(test/loadtime): cleanup go.sum and golangci.yml (#2025)** - **build(deps): Bump github.com/prometheus/common from 0.45.0 to 0.46.0 (#2040)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.39.1 to 2.40.1 (#2038)** - **feat(rpc): Use default port for HTTP(S) URLs when there is no explicit port (#1903)** - **misc(go.mod): remove unused peg dependency (#2044)** - **config: Remove unused `max_batch_bytes` (#2050)** - **Merge pull request from GHSA-qr8r-m495-7hc4** - **misc(tools)!: remove `tools` package (#2046)** - **docs(guides): add missing import (#2070)** - **deps(localnode): bump alpine version (#2077)** - **misc(Makefile): add `help` target to display the help msg (#2074)** - **`e2e`: test vote extension activation via `InitChain` and `FinalizeBlock` (#2066)** - **scripts: metricsgen parses '// metrics:' pattern (#2090)** - **e2e: Add `load_max_txs` option to manifest (#2094)** - **consensus: Add `chain_size_bytes` metric (#2093)** - **feat: conventional commits (#1995)** - **Forward port of tendermint/tendermint#7605, tendermint/spec#393, and tendermint/tendermint#8142 (#2018)** - **ci: fix permissions for "Conventional PR Title" (#2099)** - **PBTS: additions and fixes on migrated spec (#2013)** - **PBTS: forward port of tendermint/tendermint#7709 (#2089)** - **feat: use go workspace (#1924)** - **fix: `ValidateUpdate`: allow no-change updates regardless of current height (#2112)** - **test(localnet): Add monitoring tools for localnet (#2108)** - **feat: add gofumpt (#2049)** - **feat(consensus): increase log level for corner case when prevoting a proposal (#2042)** - **docs: v1 - diataxis framework (#2105)** - **feat(consensus): additional sanity checks for the size of proposed blocks (#1408)** - **perf(internal/blocksync): avoid double-calling `types.BlockFromProto` (#2016)** - **build(deps): Bump google.golang.org/grpc from 1.60.1 to 1.61.0 (#2178)** - **build(deps): Bump bufbuild/buf-setup-action from 1.28.1 to 1.29.0 (#2161)** - **build(deps): Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 (#2160)** - **build(deps): Bump slackapi/slack-github-action from 1.24.0 to 1.25.0 (#2162)** - **build(deps): Bump github.com/google/uuid from 1.5.0 to 1.6.0 (#2176)** - **refactor(privval): reverse conditional + more idiomatic Go code with early returns (#2156)** - **fix(e2e): path to latency script files in DO (#2148)** - **build(deps): Bump github.com/cometbft/cometbft-db from 0.9.1 to 0.10.0 (#2175)** - **chore(Makefile): add `setup-pre-commit` target to init git hook (#2123)** - **feat: add pebbledb (https://github.com/cockroachdb/pebble) (#2132)** - **perf(internal/blocksync): do not `ValidateBlock` twice (#2026)** - **feat(e2e): Add new targets `fast` and `clean` to Makefile (#2192)** - **ci: add `spec` type in conventional-pr-title (#2191)** - **fix(privval): retry accepting a connection on errors (#2047)** - **fix(.github/workflows): correct parameters to lint_pr_title (#2199)** - **docs: update QA method steps for qa-infra changes (#2198)** - **feat!(pbts): forward port of tendermint/tendermint#7711 (#2149)** - **docs(pbts): forward port of PBTS documentation and fixes (#2124)** - **fix: if-return and early-return (#2215)** - **chore: remove amazon linux Dockerfile and update test/docker/Dockerfile (#2135)** - **feat(ci): pre-commit framework (#2214)** - **ci: add `merge` type for conventional-pr-title (#2219)** - **spec(consensus/pbts): update description for introduced parameters. (#2206)** - **perf: optimize psql indexer (#2142)** - **fix(go.mod): do not use `replace` for api (#2236)** - **fix(Makefile): gofumpt no longer needed (#2235)** - **fix(p2p/pex): gracefully shutdown `Reactor` (#2010)** - **test(mempool): fix TestMempoolUpdateDoesNotPanicWhenApplicationMissedTx (#2242)** - **fix(flowrate): fix non-determinism in flowrate tests (#2147)** - **refactor!: Implement RFC 106: remove `GetLatestBlock*` methods from data companion API (#2240)** - **build(deps): Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 (#2253)** - **chore(config): update DB section in toml.go (#2249)** - **fix(consensus): do not precommit nil if proposal block is received (#2221)** - **build(deps): Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 (#2261)** - **fix(mempool/tests): Reduce tests duration (#2263)** - **ci: run golangci-lint linters for all files (not only ones which were modified) (#2250)** - **refactor(examples): small changes to kvstore app (#2267)** - **refactor(mempool): simplify parameters of resCbFirstTime and resCbRecheck (#2272)** - **perf(mempool/cache): clear map in Reset (#2259)** - **test(abci): update e2e tests to check against ABCI 2.0 grammar (#2201)** - **docs: symbolic references to steps in the qa-infra README (#2280)** - **chore: updated doc link (#2296)** - **fix(consensus): prevote nil upon timeout when Proposal is missing (#2218)** - **build(deps): Bump pre-commit/action from 3.0.0 to 3.0.1 (#2307)** - **build(deps): Bump golang.org/x/net from 0.20.0 to 0.21.0 (#2300)** - **build(deps): Bump github.com/cometbft/cometbft-db from 0.10.0 to 0.11.0 (#2299)** - **docs: ADR-112 Proposer-Based Timestamps (#2223)** - **refactor(internal/statesync): replace ticker with `time.After(X)` (#2294)** - **fix: simultaneous rpc test failures (#2150)** - **chore(docs): alter the section about the effects of `timeout_commit` (#1892)** - **refactor(p2p)!: Refactor PeerSet to eliminate data races & improve performance (#2246)** - **feat(pbts): Update consensus params to include pbts enabled (#2231)** - **feat(storage/metrics): Metrics to measure storage (#1974)** - **feat(e2e): Log number of sent txs (success and failed) (#2328)** - **chore: enable gomoddirectives, unparam (#2290)** - **docs: images not rendering properly in docs (#2331)** - **docs: Fix references about DeliverTx (#2330)** - **docs(qa): Add table of contents to existing QA reports (#2343)** - **chore: enable perfsprint linter (#2291)** - **feat(pruning): trigger explicitly compaction upon pruning (#1972)** - **docs: explain how to skip pre-commit hook (#2358)** - **build(deps): Bump github.com/prometheus/common from 0.46.0 to 0.47.0 (#2373)** - **build(deps): Bump google.golang.org/grpc from 1.61.0 to 1.61.1 (#2372)** - **feat(e2e): Option to set max block size in bytes at genesis (#2362)** - **build(deps): Bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 (#2371)** - **revert(pbts): revert removal of BFT Time related functions (#2205)** - **fix(e2e): Reduce flakiness of TestGRPC_GetBlockResults (#2367)** - **spec(consensus): update Block Time documentation in spec (#2316)** - **spec: adaptive MSGDELAY parameter included in PBTS spec (#2318)** - **feat(pbts): Moving VoteExtensionsEnableHeight from ABCIParams to FeatureParams. (#2335)** - **fix: remove TestMempoolFIFOWithParallelCheckTx (#2364)** - **fix(docker-compose): fix subnet (#2383)** - **feat(pbts): enable `e2e` manifest config to set `PbtsEnableHeight` (#2284)** - **perf(internal/state): avoid double-saving FinalizeBlockResponse (#2017)** - **build(deps): Bump fonttools from 4.37.4 to 4.43.0 in /scripts/qa/reporting (#2407)** - **build(deps): Bump pillow from 10.0.1 to 10.2.0 in /scripts/qa/reporting (#2408)** - **fix(node): Forward compaction config params to blockstore constructor (#2418)** - **refactor!: moved `MedianTime` out of `internal` folder (#2397)** - **chore: typo fixes (#2403)** - **chore(pbts): Removes useless code (#2420)** - **build(deps): Bump github.com/prometheus/common from 0.47.0 to 0.48.0 (#2430)** - **build(deps): Bump google.golang.org/grpc from 1.61.1 to 1.62.0 (#2429)** - **test(consensus): PBTS should be enabled by default in test units (#2329)** - **test(consensus): test enabling PBTS at a given height (#2404)** - **refactor(protobuf): switch from gogofaster to gocosmos generator (#2425)** - **docs(tutorials/install): add `From Go package` (#2414)** - **docs(pbts): documentation for PBTS-related consensus parameters (#2376)** - **refactor(consensus): print err from SignAndCheckVote (#2346)** - **feat(e2e): add parameter to set a custom output directory for testnet files (#2433)** - **spec(consensus): update description of consensus parameters for PBTS (#2415)** - **fix(localnet): Grafana dashboards for storage tests - removed experimental metrics (#2448)** - **fix(jsonrpc): enable HTTP basic auth in WS client (#2434)** - **feat(pbts): Adds timestamp to the `msgInfo` written into the WAL (#2388)** - **fix: `Rollback`: wrong modification of `state.LastHeightValidatorsChanged` while rollback at a special height (#2136)** - **docs(changelog): add missing entry for #2136 (#2459)** - **fix(pbts): legacy `ABCIParams` values are properly translated to `FeatureParams` values (#2462)** - **fix(e2e): Fixing the bug in ABCI e2e tests (#2468)** - **feat(localnet): Grafana PNG rendering for easier graph exports (#2472)** - **docs(README): remove Terra (#2469)** - **docs: document PBTS adaptive delays mechanism (#2452)** - **fix(config): Moved compact variables to the right section (#2477)** - **feat(metrics): more buckets for ProposalTimestampDifference (#2479)** - **fix(metrics): more buckets for ProposalTimestampDifference (#2481)** - **feat(pbts): Make synchrony params adaptive (#2431)** - **ci: check metrics generation in CI checks (#2483)** - **feat(consensus): improve logging for timely and untimely messages (#2321)** - **feat(types): improve Proposal and SynchronyParams validation tests (#2489)** - **test(consensus): refactor TestStateLock_POLSafety tests (#2492)** - **feat(types): refactor types.AdaptiveSynchronyParams method (#2490)** - **fix: lint errors introduced with the latest merge (#2495)** - **build(deps): Bump docker/setup-buildx-action from 3.0.0 to 3.1.0 (#2500)** - **build(deps): Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 (#2508)** - **build(deps): Bump golang.org/x/crypto from 0.19.0 to 0.20.0 (#2507)** - **build(deps): Bump github.com/prometheus/common from 0.48.0 to 0.49.0 (#2506)** - **build(deps): Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2505)** - **fix(docs): include `timesync` daemon instructions (#2491)** - **feat(privval)!: DO NOT require extension signature (#2496)** - **refactor(privval): rename sign_extension to skip_sign_extension (#2519)** - **spec(abci): fixes the spec to inform about the presence of invalid extensions in `last_commit` (#2423)** - **fix(pbts): block Time must be Canonical (#2493)** - **feat(e2e): Add manifest option `clock_skew` and corresponding backend functionality (#2454)** - **fix(mempool): converting to uint64 before additions to avoid overflows (#2498)** - **refactor!: Rename `skip_sign_extension` to `skip_extension_signing` (#2522)** - **docs(ADR): ADR-108 updated to account for the new changes in the grammar and code. (#2528)** - **feat(e2e): add generator support for clock skew + env variable rename (#2488)** - **spec(proto): add `cometbft.privval.v1beta2` proto package (#2529)** - **chore: improving error handling in the PendingEvidence (#2550)** - **feat(blocksync)!: set the max number of (concurrently) downloaded blocks to {peersCount * 20} (#2467)** - **build(deps): Bump bufbuild/buf-setup-action from 1.29.0 to 1.30.0 (#2557)** - **build(deps): Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#2560)** - **build(deps): Bump golang.org/x/net from 0.21.0 to 0.22.0 (#2562)** - **fix(cmd/cometbft/commands/version): update the output for v1 (#2546)** - **build(deps): Bump docker/build-push-action from 5.1.0 to 5.2.0 (#2558)** - **build(deps): Bump github.com/prometheus/common from 0.49.0 to 0.50.0 (#2559)** - **build(deps): Bump google.golang.org/grpc from 1.62.0 to 1.62.1 (#2563)** - **build(deps): Bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#2561)** - **feat(blocksync): sort peers by download rate & multiple requests for closer blocks (#2475)** - **feat(pbts): Adjusts PBTS metrics buckets (#2578)** - **fix(blocksync): use timer instead of time.After (#2584)** - **fix: temporary fix for `api` dependency (#2589)** --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Mikhail Zabaluev <mikhail@informal.systems> Co-authored-by: Troy Kessler <43882936+troykessler@users.noreply.github.com> Co-authored-by: leven <112051166+lx-xiang@users.noreply.github.com> Co-authored-by: werty144 <anton-paramonov2000@yandex.ru> Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Sukey <35202440+sukey2008@users.noreply.github.com> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> Co-authored-by: moriluka <85185077+moriluka@users.noreply.github.com> Co-authored-by: Đỗ Việt Hoàng <hoangdv2429@gmail.com> Co-authored-by: alex <152680487+azukiboy@users.noreply.github.com> Co-authored-by: Hernán Vanzetto <15466498+hvanz@users.noreply.github.com> Co-authored-by: Daniel <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: nenadmilosevic95 <50905385+nenadmilosevic95@users.noreply.github.com> Co-authored-by: nenadmilosevic95 <nenad@informal.systems> Co-authored-by: Jacob Gadikian <jacobgadikian@gmail.com> Co-authored-by: Greg Szabo <16846635+greg-szabo@users.noreply.github.com> Co-authored-by: Emmanuel T Odeke <emm.odeke@gmail.com> Co-authored-by: jchappelow <140431406+jchappelow@users.noreply.github.com> Co-authored-by: shuoer86 <129674997+shuoer86@users.noreply.github.com> Co-authored-by: levisyin <150114626+levisyin@users.noreply.github.com> Co-authored-by: vuittont60 <81072379+vuittont60@users.noreply.github.com> Co-authored-by: Halimao <1065621723@qq.com> Co-authored-by: alex <152680487+bodhi-crypo@users.noreply.github.com> Co-authored-by: Daniel <daniel.cason@usi.ch> Co-authored-by: Josef Widder <44643235+josef-widder@users.noreply.github.com> Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com> Co-authored-by: Kukovec <jure.kukovec@gmail.com> Co-authored-by: Daniel Cason <cason@gandria> Co-authored-by: M. J. Fromberger <fromberger@interchain.io> Co-authored-by: hattizai <150505746+hattizai@users.noreply.github.com> Co-authored-by: Huulu <35667132+IssouChancla@users.noreply.github.com> Co-authored-by: katelyn martin <me+cratelyn@katelyn.world> Co-authored-by: Richard Gibson <richard.gibson@gmail.com> Co-authored-by: Jean Deruelle <jean.deruelle@gmail.com> Co-authored-by: glnro <8335464+glnro@users.noreply.github.com> Co-authored-by: Greg Szabo <greg@philosobear.com> Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com> Co-authored-by: levisyin <lilassherl@gmail.com> Co-authored-by: k0marov <95040709+k0marov@users.noreply.github.com> Co-authored-by: Adi Seredinschi <a@seredinschi.net> Co-authored-by: Pan chao <152830401+Pan-chao@users.noreply.github.com> Co-authored-by: Khanh Hoa <49144992+hoanguyenkh@users.noreply.github.com> Co-authored-by: glnro <lauren@informal.systems> Co-authored-by: toni <143221387+xyztoni@users.noreply.github.com> Co-authored-by: DragonKid <idragonkid@gmail.com> Co-authored-by: Matt Ketmo <matthieu@moquet.net> Co-authored-by: Ethan <cosinlinker@gmail.com> Co-authored-by: Evgeny Danilenko <6655321@bk.ru> Co-authored-by: Duong Minh Ngoc <153509244+minhngoc274@users.noreply.github.com> Co-authored-by: Kero <keroroxx520@gmail.com>

jmalicevic added 8 commits September 20, 2023 16:55

Added metrics to measure db access time

666b9f7

Added timing metrics

a10f348

Testnet setup

68c48e8

Merge branch 'main' into jasmina/storage-time-metrics

c2bf74e

Merge branch 'main' into jasmina/storage-time-metrics

34d1420

Revert "Testnet setup"

abeb39a

This reverts commit 68c48e8.

Adjusted metrics to not measure marshalling/unmarshalling where possible

b47e4c6

Reduced bucket interval and adjusted timing measurements

f632519

jmalicevic added metrics P:storage-optimization Priority: Give operators greater control over storage and storage optimization labels Jan 5, 2024

jmalicevic added this to the 2024-Q1 milestone Jan 5, 2024

jmalicevic self-assigned this Jan 5, 2024

jmalicevic mentioned this pull request Jan 5, 2024

metrics: add a candidate set of metrics for measuring db access times. #79

Closed

melekes reviewed Jan 8, 2024

View reviewed changes

cmd/cometbft/commands/inspect.go Outdated Show resolved Hide resolved

internal/state/store.go Show resolved Hide resolved

internal/store/store.go Outdated Show resolved Hide resolved

Sync with testing branch

bac20eb

adizere removed this from the 2024-Q1 milestone Jan 11, 2024

jmalicevic mentioned this pull request Jan 17, 2024

Storage optimizations: Q1 2024 tracking issue #2058

Closed

7 tasks

github-actions bot added the stale For use by stalebot label Jan 22, 2024

jmalicevic added 3 commits January 26, 2024 11:14

Merge remote-tracking branch 'origin/main' into jasmina/storage-time-…

34f3865

…metrics

Merge branch 'main' into jasmina/storage-time-metrics

aeeaefe

Tuned time measurements to further exlude marshalling time where poss…

f718cb8

…ible

jmalicevic changed the title ~~metrics: Metrics to measure storage~~ feat (metrics) Metrics to measure storage Jan 26, 2024

jmalicevic changed the title ~~feat (metrics) Metrics to measure storage~~ feat(storage/metrics): Metrics to measure storage Jan 26, 2024

jmalicevic added 3 commits January 26, 2024 14:44

Added metrics as special option instead of required param

099c274

Reverted change to loadscript and removed needless metrics asgn

2ffaef5

Added changelog

328756c

jmalicevic removed the stale For use by stalebot label Jan 26, 2024

jmalicevic marked this pull request as ready for review January 26, 2024 14:02

jmalicevic requested a review from a team as a code owner January 26, 2024 14:02

jmalicevic added 3 commits January 26, 2024 15:10

Renamed metric for valinfo load

dc26791

Count db access time when key is not found

45497ad

formatting

a7e1746

Merge branch 'main' into jasmina/storage-time-metrics

99bb587

greg-szabo reviewed Jan 29, 2024

View reviewed changes

andynog reviewed Jan 29, 2024

View reviewed changes

.changelog/unreleased/features/1974-statestore-metrics.md Outdated Show resolved Hide resolved

.changelog/unreleased/features/1974-store-metrics.md Outdated Show resolved Hide resolved

internal/state/store.go Show resolved Hide resolved

internal/store/store.go Show resolved Hide resolved

jmalicevic added 2 commits January 30, 2024 15:38

Altered changelog message

117fe67

Merge branch 'main' into jasmina/storage-time-metrics

1d58bb5

melekes reviewed Jan 31, 2024

View reviewed changes

.changelog/unreleased/features/1974-statestore-metrics.md Outdated Show resolved Hide resolved

internal/store/store.go Show resolved Hide resolved

internal/state/store.go Outdated Show resolved Hide resolved

jmalicevic and others added 5 commits February 6, 2024 12:48

Merge remote-tracking branch 'origin/main' into jasmina/storage-time-…

286e78e

…metrics

removed double pointer to metrics

9e10805

Added comment explaining marshalling

82478da

Applied review comment

10bd45a

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

Fixed nil_pointer in test

a635648

melekes approved these changes Feb 7, 2024

View reviewed changes

Merge branch 'main' into jasmina/storage-time-metrics

afaabb9

jmalicevic added this pull request to the merge queue Feb 13, 2024

Merged via the queue into main with commit dfd3f6c Feb 13, 2024
24 checks passed

jmalicevic deleted the jasmina/storage-time-metrics branch February 13, 2024 09:19

hvanz mentioned this pull request Feb 13, 2024

v1 QA #2100

Closed

10 tasks

mergify bot mentioned this pull request Feb 22, 2024

feat(storage/metrics): Metrics to measure storage (backport #1974) #2411

Merged

lasarojc mentioned this pull request Mar 12, 2024

chore(spec): merge main into spec/consensus-gossip #2594

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(storage/metrics): Metrics to measure storage #1974

feat(storage/metrics): Metrics to measure storage #1974

jmalicevic commented Jan 5, 2024 •

edited

melekes left a comment

github-actions bot commented Jan 22, 2024

jmalicevic commented Jan 26, 2024 •

edited

greg-szabo left a comment

greg-szabo Jan 29, 2024

jmalicevic Jan 30, 2024

greg-szabo Jan 29, 2024

jmalicevic Feb 6, 2024

greg-szabo Jan 29, 2024

andynog Feb 12, 2024

andynog left a comment

melekes commented Feb 22, 2024

mergify bot commented Feb 22, 2024 •

edited

feat(storage/metrics): Metrics to measure storage #1974

feat(storage/metrics): Metrics to measure storage #1974

Conversation

jmalicevic commented Jan 5, 2024 • edited

PR checklist

melekes left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 22, 2024

jmalicevic commented Jan 26, 2024 • edited

greg-szabo left a comment

Choose a reason for hiding this comment

greg-szabo Jan 29, 2024

Choose a reason for hiding this comment

jmalicevic Jan 30, 2024

Choose a reason for hiding this comment

greg-szabo Jan 29, 2024

Choose a reason for hiding this comment

jmalicevic Feb 6, 2024

Choose a reason for hiding this comment

greg-szabo Jan 29, 2024

Choose a reason for hiding this comment

andynog Feb 12, 2024

Choose a reason for hiding this comment

andynog left a comment

Choose a reason for hiding this comment

melekes commented Feb 22, 2024

mergify bot commented Feb 22, 2024 • edited

✅ Backports have been created

jmalicevic commented Jan 5, 2024 •

edited

jmalicevic commented Jan 26, 2024 •

edited

mergify bot commented Feb 22, 2024 •

edited