New metrics to track duplicate votes and block parts #896

ebuchman · 2023-05-31T10:30:25Z

Related to #26

Add metrics to track duplicate votes and block parts in preparation for changes to reduce bandwidth :)

PR checklist

Tests written/updated
Changelog entry added in .changelog (we use unclog to manage our changelog)
Updated relevant documentation (docs/ or spec/) and code comments

lasarojc · 2023-05-31T12:01:35Z

You missed running make metrics to update metrics.gen.go

cason

Nice to have. I've left some minor comments that shouldn't affect the feature.

My more general comment is that both metrics are more helpful when they reflect the portion of votes or block parts that are duplicated. Of course, the operator can "plot" the ratio of different metrics to get this information, but I wonder whether a different metric abstraction would be more helpful.

cason · 2023-05-31T12:50:49Z

consensus/state.go

@@ -1940,6 +1940,11 @@ func (cs *State) addProposalBlockPart(msg *BlockPartMessage, peerID p2p.ID) (add
 	}

 	cs.metrics.BlockGossipPartsReceived.With("matches_current", "true").Add(1)
+	if !added {


Suggestion: I would move this if to after line 1953 and return from the method if added == false. In fact, the if of line 1954 is not enabled in this case and we just return.

i thought it would be better to keep the metric closer to where the added is returned rather than after other logic. that 1949-1953 logic is byzantine behaviour anyways so its not like metrics are gonna be robust to that ...

cason · 2023-05-31T12:51:47Z

consensus/state.go

@@ -1940,6 +1940,11 @@ func (cs *State) addProposalBlockPart(msg *BlockPartMessage, peerID p2p.ID) (add
 	}

 	cs.metrics.BlockGossipPartsReceived.With("matches_current", "true").Add(1)
+	if !added {
+		// NOTE: we are disregarding possible duplicates above where heights dont match or we're not expecting block parts yet


In thesis, if we are not expecting block parts or if the height do the block part doesn't match ours, we shouldn't receive this message from a (correct) peer, from the way the consensus reactor is implemented.

will update comment

cason · 2023-05-31T12:53:26Z

types/part_set.go

@@ -270,9 +270,12 @@ func (ps *PartSet) Total() uint32 {
 }

 func (ps *PartSet) AddPart(part *Part) (bool, error) {
+	// TODO: remove this? would be preferable if this only returned (false, nil)


Essentially all PartSet methods handle this condition in which the block set is not initialized. This might make the consensus logic simpler, but it is not really easy to read.

I believe AddPart is only called in one place in line 1934 in consensus/state and we check right before if the parts are nil, so it should be totally redundant to have the check in here, but we thought it might break tests or something

cason · 2023-05-31T12:53:34Z

types/part_set.go

 	if ps == nil {
 		return false, nil
 	}
+


Suggested change

since i added the comment to remove the nil check above i wanted to separate it from the stuff thats not to be removed ;)

sergio-mena · 2023-06-01T17:07:32Z

types/part_set.go

@@ -270,9 +270,12 @@ func (ps *PartSet) Total() uint32 {
 }

 func (ps *PartSet) AddPart(part *Part) (bool, error) {
+	// TODO: remove this? would be preferable if this only returned (false, nil)
+	// when its a duplicate block part
 	if ps == nil {
 		return false, nil


I checked the code and this function is only called from State's addProposalBlockPart function (ignoring test code here).
If you look at addProposalBlockPart's code immediately following the call to AddPart, you see that it is acting as though the part had indeed been added. So, returning false, nil in this line is a BUG. We must return false and an error, such as fmt.Errorf("nil PartSet")

not sure I understand this, my argument was that this check is redundant since before we call AddPart we already check if the part set is nil!

sergio-mena

My only comment is orthogonal to this PR, so LGTM

ebuchman · 2023-06-01T19:48:51Z

My more general comment is that both metrics are more helpful when they reflect the portion of votes or block parts that are duplicated. Of course, the operator can "plot" the ratio of different metrics to get this information, but I wonder whether a different metric abstraction would be more helpful.

@cason what did you have in mind? my understanding is we should keep the metrics simple/raw since it's always easy to manipulate the raw data

ebuchman · 2023-06-01T20:09:59Z

Alright I think we can merge this, I spent a lot of time using these metrics today to test some changes ;)

sergio-mena · 2023-06-01T20:32:41Z

If we can get the data we want via prometheus queries (by using functions and/or combining basic metrics), I am in favour of keeping the ones in the code (which can't be changed easily) as simple as possible

* consensus: metric for duplicate votes * consensus: metric for duplicate block parts * make metrics --------- Co-authored-by: Sergio Mena <sergio@informal.systems>

cason · 2023-06-03T11:50:20Z

So, I was thinking about a metric keeping a ratio, like for instance the load metric for the CPU. But this is a nice to have, not an actual requirement, in particular because I don't know whether Prometheus actually supports this behavior.

* consensus: metric for duplicate votes * consensus: metric for duplicate block parts * make metrics --------- Co-authored-by: Ethan Buchman <ethan@coinculture.info>

…artMessage and random sleeps (#904) Addresses tendermint/tendermint#627 (finally!) This PR is based off #896 which added new metrics to measure duplicate votes and block parts received from peers. ## Summary This PR introduces two optimizations to the consensus gossip: HasProposalBlockPartMessage and random sleeps in the fast loop in the gossip routines. Control for both were added to the config file for testing purposes, but at least the HasProposalBlockPartMessage should probably not be in the config, it should just be hardcoded active once we're happy with it. On another note, we may want to consider some changes so this can be rolled out in a non-breaking way. If I'm not mistaken since the message was added to an existing p2p channel it will be interpreted by peers on older versions as malicious. My understanding is we could just move it to a new channel to solve this so the change would be backwards compatible and nodes can roll it out progressively. ### HasProposalBlockPartMessage The first is a new HasProposalBlockPartMessage. This follows identical logic to the HasVoteMessage. When a node receives a ProposalBlockPart, it broadcasts the HasProposalBlockPartMessage to all its peers. This informs them that they don't need to send the node the part, thus reducing the amount of redundant block parts gossiped over the network. ### Gossip Sleeps The second change is the introduction of a random sleep in the fast loops of the gossipVote and gossipData routines. We run these routines for each peer, and when there are things to send, the routines run in a tight loop picking and sending votes or parts to send to the peer. This reduces the effectiveness of the "HasXXXMessage" mechanism since there is minimal opportunity for the peer state to be updated with when the HasXXXMessages are received before the new votes/parts are selected to be sent. Adding a small random sleep gives the reactor a chance to breath and process inbound HasXXXMessages before picking new votes/parts to send. Randomizing the sleep ensures the peers don't all sleep the same amount, which could negate its effect. ## Results Combining the new HasBlockPartMessage with the random sleep in gossipData routine results in a ~20% bandwidth reduction on BlockPart messages in 4 node and 7 node networks under load as measured in a local testnet on my laptop, as well as 4 node networks in the cloud. Larger tests are still to be conducted, but these initial results are very promising. The random sleep in gossipVote routine also results in a ~15% bandwidth reduction on Vote messages in 7 node networks as measured in a local testnet on my laptop. Again, promising, but more tests to be done. The largest consumers of bandwidth in the comet system are mempool txs, votes, and block parts. While mempool txs are not addressed here, the latter two are, and should lead to significant reductions in bandwidth usage overall. Note the addition of a new HasProposalBlockPartMessage adds minimal additional bandwidth itself as each message contains only 3 integers (one 64 bit, two 32 bit, so 16 bytes total) compared to hundreds of bytes in a block part message. This was also confirmed experimentally. Some teaser screenshots below on small local networks using the cometbft `e2e` testing system on my laptop. ### Votes With and without the random sleep (upper bounded at 50ms) on a 7 node network, with no load: <img width="1426" alt="image" src="https://github.com/cometbft/cometbft/assets/2300911/816f86a7-49d7-4988-ae57-e52d9bd196f8"> The main lines visible are the bandwidth from the Vote (the larger one) and BlockPart messages. Note how both are reduced ~15% from adding the sleep (the left). The effect of the sleep on block times is negligible (they rise from 1.6 to 1.7 seconds per block in this experiment) ### Block Parts On a 4 node network, with and without the new HasProposalBlockPartMessage and with and without the random sleep on the gossipDataRoutine: ![image](https://github.com/cometbft/cometbft/assets/2300911/70c616e9-40ff-4f6c-b3aa-fc7b5e9adb73) Ignore the spikes on the right for now, this is the system acting up under load running all 4 nodes on my laptop ... The image shows 4 experiments. The purple line is bandwidth from mempool txs (roughly the same in all experiments), the green is bandwidth from block part messages (what we're focusing on). The experiments, and the approximate block part bandwidth when it first flattens out, are: 1. has block parts, sleep (680k) 2. no has block parts, sleep (830k) 3. has block parts, no sleep (730k) 4. no has block parts, no sleep (870k) The last experiment is the current baseline. Adding the sleep alone without the new message type (2nd experiment) has some minimal impact (870 -> 830). Adding just the new message type without the sleep (3rd experiment) has more significant impact (870 -> 730). But adding both together (1st experiiment) has the most significant impact (870->680), a reduction of ~20%. While we've done some preliminary experiments here, we still need to do larger experiments on cloud deployments, as well as writing some actual tests :) --- #### PR checklist - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments

…artMessage and random sleeps (cometbft#904) Addresses tendermint/tendermint#627 (finally!) This PR is based off cometbft#896 which added new metrics to measure duplicate votes and block parts received from peers. This PR introduces two optimizations to the consensus gossip: HasProposalBlockPartMessage and random sleeps in the fast loop in the gossip routines. Control for both were added to the config file for testing purposes, but at least the HasProposalBlockPartMessage should probably not be in the config, it should just be hardcoded active once we're happy with it. On another note, we may want to consider some changes so this can be rolled out in a non-breaking way. If I'm not mistaken since the message was added to an existing p2p channel it will be interpreted by peers on older versions as malicious. My understanding is we could just move it to a new channel to solve this so the change would be backwards compatible and nodes can roll it out progressively. The first is a new HasProposalBlockPartMessage. This follows identical logic to the HasVoteMessage. When a node receives a ProposalBlockPart, it broadcasts the HasProposalBlockPartMessage to all its peers. This informs them that they don't need to send the node the part, thus reducing the amount of redundant block parts gossiped over the network. The second change is the introduction of a random sleep in the fast loops of the gossipVote and gossipData routines. We run these routines for each peer, and when there are things to send, the routines run in a tight loop picking and sending votes or parts to send to the peer. This reduces the effectiveness of the "HasXXXMessage" mechanism since there is minimal opportunity for the peer state to be updated with when the HasXXXMessages are received before the new votes/parts are selected to be sent. Adding a small random sleep gives the reactor a chance to breath and process inbound HasXXXMessages before picking new votes/parts to send. Randomizing the sleep ensures the peers don't all sleep the same amount, which could negate its effect. Combining the new HasBlockPartMessage with the random sleep in gossipData routine results in a ~20% bandwidth reduction on BlockPart messages in 4 node and 7 node networks under load as measured in a local testnet on my laptop, as well as 4 node networks in the cloud. Larger tests are still to be conducted, but these initial results are very promising. The random sleep in gossipVote routine also results in a ~15% bandwidth reduction on Vote messages in 7 node networks as measured in a local testnet on my laptop. Again, promising, but more tests to be done. The largest consumers of bandwidth in the comet system are mempool txs, votes, and block parts. While mempool txs are not addressed here, the latter two are, and should lead to significant reductions in bandwidth usage overall. Note the addition of a new HasProposalBlockPartMessage adds minimal additional bandwidth itself as each message contains only 3 integers (one 64 bit, two 32 bit, so 16 bytes total) compared to hundreds of bytes in a block part message. This was also confirmed experimentally. Some teaser screenshots below on small local networks using the cometbft `e2e` testing system on my laptop. With and without the random sleep (upper bounded at 50ms) on a 7 node network, with no load: <img width="1426" alt="image" src="https://github.com/cometbft/cometbft/assets/2300911/816f86a7-49d7-4988-ae57-e52d9bd196f8"> The main lines visible are the bandwidth from the Vote (the larger one) and BlockPart messages. Note how both are reduced ~15% from adding the sleep (the left). The effect of the sleep on block times is negligible (they rise from 1.6 to 1.7 seconds per block in this experiment) On a 4 node network, with and without the new HasProposalBlockPartMessage and with and without the random sleep on the gossipDataRoutine: ![image](https://github.com/cometbft/cometbft/assets/2300911/70c616e9-40ff-4f6c-b3aa-fc7b5e9adb73) Ignore the spikes on the right for now, this is the system acting up under load running all 4 nodes on my laptop ... The image shows 4 experiments. The purple line is bandwidth from mempool txs (roughly the same in all experiments), the green is bandwidth from block part messages (what we're focusing on). The experiments, and the approximate block part bandwidth when it first flattens out, are: 1. has block parts, sleep (680k) 2. no has block parts, sleep (830k) 3. has block parts, no sleep (730k) 4. no has block parts, no sleep (870k) The last experiment is the current baseline. Adding the sleep alone without the new message type (2nd experiment) has some minimal impact (870 -> 830). Adding just the new message type without the sleep (3rd experiment) has more significant impact (870 -> 730). But adding both together (1st experiiment) has the most significant impact (870->680), a reduction of ~20%. While we've done some preliminary experiments here, we still need to do larger experiments on cloud deployments, as well as writing some actual tests :) --- - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments

…raloopSleep (#1706) * consensus: optimize vote and block part gossip with HasProposalBlockPartMessage and random sleeps (#904) Addresses tendermint/tendermint#627 (finally!) This PR is based off #896 which added new metrics to measure duplicate votes and block parts received from peers. This PR introduces two optimizations to the consensus gossip: HasProposalBlockPartMessage and random sleeps in the fast loop in the gossip routines. Control for both were added to the config file for testing purposes, but at least the HasProposalBlockPartMessage should probably not be in the config, it should just be hardcoded active once we're happy with it. On another note, we may want to consider some changes so this can be rolled out in a non-breaking way. If I'm not mistaken since the message was added to an existing p2p channel it will be interpreted by peers on older versions as malicious. My understanding is we could just move it to a new channel to solve this so the change would be backwards compatible and nodes can roll it out progressively. The first is a new HasProposalBlockPartMessage. This follows identical logic to the HasVoteMessage. When a node receives a ProposalBlockPart, it broadcasts the HasProposalBlockPartMessage to all its peers. This informs them that they don't need to send the node the part, thus reducing the amount of redundant block parts gossiped over the network. The second change is the introduction of a random sleep in the fast loops of the gossipVote and gossipData routines. We run these routines for each peer, and when there are things to send, the routines run in a tight loop picking and sending votes or parts to send to the peer. This reduces the effectiveness of the "HasXXXMessage" mechanism since there is minimal opportunity for the peer state to be updated with when the HasXXXMessages are received before the new votes/parts are selected to be sent. Adding a small random sleep gives the reactor a chance to breath and process inbound HasXXXMessages before picking new votes/parts to send. Randomizing the sleep ensures the peers don't all sleep the same amount, which could negate its effect. Combining the new HasBlockPartMessage with the random sleep in gossipData routine results in a ~20% bandwidth reduction on BlockPart messages in 4 node and 7 node networks under load as measured in a local testnet on my laptop, as well as 4 node networks in the cloud. Larger tests are still to be conducted, but these initial results are very promising. The random sleep in gossipVote routine also results in a ~15% bandwidth reduction on Vote messages in 7 node networks as measured in a local testnet on my laptop. Again, promising, but more tests to be done. The largest consumers of bandwidth in the comet system are mempool txs, votes, and block parts. While mempool txs are not addressed here, the latter two are, and should lead to significant reductions in bandwidth usage overall. Note the addition of a new HasProposalBlockPartMessage adds minimal additional bandwidth itself as each message contains only 3 integers (one 64 bit, two 32 bit, so 16 bytes total) compared to hundreds of bytes in a block part message. This was also confirmed experimentally. Some teaser screenshots below on small local networks using the cometbft `e2e` testing system on my laptop. With and without the random sleep (upper bounded at 50ms) on a 7 node network, with no load: <img width="1426" alt="image" src="https://github.com/cometbft/cometbft/assets/2300911/816f86a7-49d7-4988-ae57-e52d9bd196f8"> The main lines visible are the bandwidth from the Vote (the larger one) and BlockPart messages. Note how both are reduced ~15% from adding the sleep (the left). The effect of the sleep on block times is negligible (they rise from 1.6 to 1.7 seconds per block in this experiment) On a 4 node network, with and without the new HasProposalBlockPartMessage and with and without the random sleep on the gossipDataRoutine: ![image](https://github.com/cometbft/cometbft/assets/2300911/70c616e9-40ff-4f6c-b3aa-fc7b5e9adb73) Ignore the spikes on the right for now, this is the system acting up under load running all 4 nodes on my laptop ... The image shows 4 experiments. The purple line is bandwidth from mempool txs (roughly the same in all experiments), the green is bandwidth from block part messages (what we're focusing on). The experiments, and the approximate block part bandwidth when it first flattens out, are: 1. has block parts, sleep (680k) 2. no has block parts, sleep (830k) 3. has block parts, no sleep (730k) 4. no has block parts, no sleep (870k) The last experiment is the current baseline. Adding the sleep alone without the new message type (2nd experiment) has some minimal impact (870 -> 830). Adding just the new message type without the sleep (3rd experiment) has more significant impact (870 -> 730). But adding both together (1st experiiment) has the most significant impact (870->680), a reduction of ~20%. While we've done some preliminary experiments here, we still need to do larger experiments on cloud deployments, as well as writing some actual tests :) --- - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments * lint * remove VoteExtensionSize * Apply suggestions from code review Revert changes not belonging to the original PR being backported. * Remove empty change. --------- Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: lasaro <lasaro@gmail.com>

* Backport changelog for #896 * #896 was committed before , so moving it to the right release

- **Versioned protobuf files (#495)** - **Restore rpc/grpc/v1/types.pb.go** - **proto: restore deprecation notice on BroadcastAPI** - **Struct `Client` exposes sensitive data (#784)** - **Unsafe int cast in `kill` command (#783)** - **ADR-100: Data Companion Push API (#73)** - **rfc: incoming txs while catching up (#735)** - **build(deps): Bump bufbuild/buf-setup-action from 1.17.0 to 1.18.0 (#804)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.8 to 1.4.9 (#806)** - **build(deps): Bump golang.org/x/sync from 0.1.0 to 0.2.0 (#807)** - **build(deps): Bump google.golang.org/grpc from 1.54.0 to 1.55.0 (#813)** - **build(deps): Bump github.com/prometheus/client_golang from 1.15.0 to 1.15.1 (#810)** - **build(deps): Bump github.com/bufbuild/buf from 1.17.0 to 1.18.0 (#809)** - **build(deps): Bump github.com/bufbuild/buf from 1.17.0 to 1.18.0 (#809)** - **build(deps): Bump github.com/prometheus/client_model from 0.3.0 to 0.4.0 (#812)** - **build(deps): Bump github.com/prometheus/common from 0.42.0 to 0.43.0 (#811)** - **ADR-103: Protobuf definition versioning (#772)** - **spec/p2p: Specify the operation of a Reactor (#714)** - **build(deps): Bump github.com/docker/distribution (#827)** - **build(deps): Bump github.com/cloudflare/circl from 1.3.1 to 1.3.3 (#828)** - **ADR-103: Protobuf definition versioning (#772) (#817)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.9 to 1.4.10 (#830)** - **build(deps): Bump golang.org/x/net from 0.9.0 to 0.10.0 (#831)** - **build(deps): Bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#832)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.26.1 to 2.27.1 (#833)** - **rpc: Remove response data from response failure logs (#829)** - **pubsub/kvindexer:support for big numbers - v2 (#797)** - **Extend the infrastructure provider with `StartNodes` and `StopTestnet` (#796)** - **grpc: Add base gRPC server with version service (#818)** - **Digital Ocean implementation of `StartNodes` and `StopTestnet` (#846)** - **RFC 102: Improve forward compatibility of proto-generated Rust (#724)** - **mempool: Add metric to measure how many times a tx was received (#637)** - **`e2e` provider is extended with `Disconnect`, `Reconnect` and `CheckUpgraded` (#852)** - **Sort `loadtime` tool's report (#854)** - **feat: make handshake cancelable (#857)** - **proto: Restore snake_case JSON for `ExecTxResult` (#856)** - **qa: 200-nodes test on v0.38 (#877)** - **qa: rotating node test on `v0.38.x` (#883)** - **Changes to QA related files (e.g., `method.md`) (#878)** - **build(deps): Bump bufbuild/buf-setup-action from 1.18.0 to 1.19.0 (#868)** - **build(deps): Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#873)** - **build(deps): Bump github.com/bufbuild/buf from 1.18.0 to 1.19.0 (#872)** - **Adds `vote_extension_size` to manifests (#858)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.27.1 to 2.28.1 (#893)** - **build(deps): Bump github.com/prometheus/common from 0.43.0 to 0.44.0 (#892)** - **build(deps): Bump slackapi/slack-github-action from 1.23.0 to 1.24.0 (#869)** - **ADR 104: State sync from local application snapshot (#801)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 (#891)** - **mempool: slight refactor for improving readability (#894)** - **Add Vote Extension varying size testnet (#888)** - **fix: lint fail when golangci-lint bump from v1.52.2 to v1.53.x (#908)** - **build(deps): Bump github.com/bufbuild/buf from 1.19.0 to 1.20.0 (#912)** - **build(deps): Bump github.com/golangci/golangci-lint (#917)** - **build(deps): Bump github.com/spf13/viper from 1.15.0 to 1.16.0 (#913)** - **build(deps): Bump bufbuild/buf-setup-action from 1.19.0 to 1.20.0 (#910)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.28.1 to 2.28.2 (#914)** - **Minor fixes to `RELEASES.md` (#923)** - **Prevent a transaction to appear twice in the mempool (#890)** - **New metrics to track duplicate votes and block parts (#896) (#905)** - **e2e tests are being skipped; `INFRASTRUCTURE-*` bug; portGen inconsistency bug (#933)** - **build(deps): Bump github.com/bufbuild/buf from 1.20.0 to 1.21.0 (#948)** - **build(deps): Bump bufbuild/buf-setup-action from 1.20.0 to 1.21.0 (#946)** - **spec/p2p: document the p2p API used by Reactors (#851)** - **build(deps): Bump docker/login-action from 2.1.0 to 2.2.0 (#944)** - **e2e: Generate prometheus.yaml on setup (#954)** - **Update the type annotations in the light client spec (#955)** - **build(deps): Bump docker/build-push-action from 4.0.0 to 4.1.0 (#947)** - **build(deps): Bump docker/setup-buildx-action from 2.5.0 to 2.7.0 (#962)** - **Add requirement for `CheckTx` in ABCI spec (#928)** - **fix: avoid recursive call after rename to (*PeerState).MarshalJSON (#865)** - **Port upstream proto changes to cometbft package** - **changelog: Import entries for v0.34.28, v0.34.29, v0.37.1 and v0.37.2 to `main` (#974)** - **test/e2e: Fix Docker image build (#984)** - **build(deps): Bump github.com/BurntSushi/toml from 1.3.0 to 1.3.2 (#952)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.28.2 to 2.29.0 (#953)** - **Remove duplicate function `deterministicExecTxResult` (#986)** - **mempool: Fix the benchmarks (#934)** - **Add `CMT_HOME` (or remove it?) (#983)** - **Remove buf.yaml at the root of the source tree (#979)** - **build(deps): Bump docker/build-push-action from 4.1.0 to 4.1.1 (#989)** - **build(deps): Bump golang.org/x/net from 0.10.0 to 0.11.0 (#994)** - **build(deps): Bump golang.org/x/sync from 0.2.0 to 0.3.0 (#990)** - **build(deps): Bump google.golang.org/grpc from 1.55.0 to 1.56.0 (#992)** - **build(deps): Bump github.com/golangci/golangci-lint from 1.53.2 to 1.53.3 (#996)** - **build(deps): Bump github.com/prometheus/client_golang from 1.15.1 to 1.16.0 (#991)** - **spec/p2p: new structure for the p2p specification (#966)** - **consensus: optimize vote and block part gossip with HasProposalBlockPartMessage and random sleeps (#904)** - **Extend ABCI `max_block_size` parameter to give extended control to the app (#1003)** - **build(deps): Bump bufbuild/buf-setup-action from 1.21.0 to 1.22.0 (#1026)** - **Bump logs on non-fatal vote validation failures to Info level (#1022)** - **Update codeowners to include DevRel team (#1023)** - **deps: Manually bump github.com/vektra/mockery/v2 from 2.29.0 to 2.30.1 (#1009)** - **Update releasing.md to sign tags (#1034)** - **build(deps): Bump github.com/bufbuild/buf from 1.21.0 to 1.22.0 (#1027)** - **Rename ABCI enums and values to satisfy buf guidelines (#975)** - **build(deps): Bump google.golang.org/grpc from 1.56.0 to 1.56.1 (#1028)** - **docs: Added double quotes to /abci_query path param (#1015)** - **Update Docs with Finalize Block (#760)** - **Clarifies that processProposal may be called for set of transactions different from the one returned in the preceding prepareProposal (#1033)** - **Update the annotation of part_set.go (#1056)** - **build(deps): Bump docker/setup-buildx-action from 2.7.0 to 2.8.0 (#1071)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.30.1 to 2.30.16 (#1064)** - **build(deps): Bump bufbuild/buf-setup-action from 1.22.0 to 1.23.1 (#1072)** - **build(deps): Bump github.com/bufbuild/buf from 1.22.0 to 1.23.1 (#1066)** - **build(deps): Bump google.golang.org/protobuf from 1.30.0 to 1.31.0 (#1065)** - **build(deps): Bump google.golang.org/grpc from 1.56.1 to 1.56.2 (#1101)** - **build(deps): Bump golang.org/x/crypto from 0.10.0 to 0.11.0 (#1103)** - **build(deps): Bump docker/setup-buildx-action from 2.8.0 to 2.9.0 (#1105)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.30.16 to 2.31.1 (#1100)** - **build(deps): Bump golang.org/x/net from 0.11.0 to 0.12.0 (#1102)** - **ci: Trigger workflows on merge group (#1118)** - **Revert "config: add bootstrap peers (#9680)" (#1109)** - **node: Revert removal of public reactor accessors (#1120)** - **ci: Disable CodeQL check in merge queues (#1123)** - **p2p: Remove UPnP functionality (#1114)** - **ADR 107: Rename proto versions to pre-v1 betas (#1110)** - **RFC 104: Internal messaging using the actor model (#1092)** - **build(deps): Bump github.com/bufbuild/buf from 1.23.1 to 1.24.0 (#1131)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.31.1 to 2.32.0 (#1132)** - **build(deps): Bump docker/setup-buildx-action from 2.9.0 to 2.9.1 (#1133)** - **build(deps): Bump bufbuild/buf-setup-action from 1.23.1 to 1.24.0 (#1134)** - **spec: Add mempool specification in English and Quint (#997)** - **mempool: ADR for refactoring list of senders (#1032)** - **build(deps): Bump bufbuild/buf-setup-action from 1.24.0 to 1.25.0 (#1157)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.7.0 to 5.8.0 (#1159)** - **build(deps): Bump github.com/bufbuild/buf from 1.24.0 to 1.25.0 (#1160)** - **proxy: Rename "unsynchronized" to "connection-synchronized" local client creator (#1145)** - **mempool: Keep track of senders in reactor instead of implementation (#1010)** - **cmd: Remove `replay` and `replay-console` subcommands (#1170)** - **build(deps): Bump google.golang.org/grpc from 1.56.2 to 1.57.0 (#1181)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.8.0 to 5.8.1 (#1180)** - **ci: Remove Mergify automerge (#1182)** - **Add gRPC block service (#1142)** - **blocksync: export errors (#1186)** - **Porting changes related to creating a lean docker image from `pierre/fast-prototyping-1059` branch. (#1192)** - **chore: Format repo (#1193)** - **Add BlockResults gRPC service (#1168)** - **consensus: remove logic to unlock block on +2/3 prevote for nil (#1175)** - **ADR 101: Add GetLatest method to block service (#1209)** - **ADR 101: Implement pruning mechanism (#1150)** - **config: export errors (#1190)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.0 to 2.32.3 (#1218)** - **Add check for non-`nil` in `enterCommit` (#1208)** - **Log proposer's address when correctly accepting a proposal (#1079)** - **Close evidence.db OnStop (#1210)** - **build(deps): Bump bufbuild/buf-setup-action from 1.25.0 to 1.25.1 (#1213)** - **build(deps): Bump golang.org/x/crypto from 0.11.0 to 0.12.0 (#1215)** - **build(deps): Bump github.com/bufbuild/buf from 1.25.0 to 1.25.1 (#1216)** - **build(deps): Bump golang.org/x/net from 0.12.0 to 0.14.0 (#1217)** - **Forward-port: update state to prevote `nil` when proposal block does not match locked block (#1203)** - **mempool: Store peer ids as p2p.ID instead of uint16 (remove mempoolIDs) (#1191)** - **ADR-101: implement gRPC `PruningService` (#1154)** - **consensus: Avoid process proposal when we know correct nodes validated it (#1231)** - **ADR-101: Metrics to monitor the pruning (#1234)** - **build(deps): Bump bufbuild/buf-setup-action from 1.25.1 to 1.26.0 (#1241)** - **build(deps): Bump github.com/bufbuild/buf from 1.25.1 to 1.26.1 (#1239)** - **build(deps): Bump github.com/golangci/golangci-lint (#1240)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.3 to 2.32.4 (#1242)** - **abci: export errors (#1185)** - **consensus: export errors (#1211)** - **ADR-101: Pruning mechanism minor fixes (#1246)** - **chore: Bump minimum Go version on `main` to v1.21 (#1244)** - **ADR 101: Add `Close` method to gRPC client (#1251)** - **ADR 108: ADR for extending E2E infrastructure so that we can check CometBFT's behaviour with respect to ABCI++ grammar. (#902)** - **docs: Add logging guide to contributing guidelines (#1250)** - **chore: log `app_hash` as hex (#1264)** - **update language (#1263)** - **build(deps): Bump bufbuild/buf-setup-action from 1.26.0 to 1.26.1 (#1272)** - **build(deps): Bump gonum.org/v1/gonum from 0.13.0 to 0.14.0 (#1275)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.10 to 1.4.11 (#1276)** - **ADR 101: Refactor height check-related logic and tests (#1271)** - **Fixed linter** - **Revert "Fixed linter"** - **Fixed mem aliasing in for loop (#1280)** - **Remove genesis persistence in state db (#1017)** - **Revert "Remove genesis persistence in state db (#1017)" (#1294)** - **evidence: export errors (#1284)** - **Re-implement removal of genesis persistence in state db (#1295)** - **build(deps): Bump docker/setup-buildx-action from 2.9.1 to 2.10.0 (#1298)** - **node: verify genesis doc hash against the file contents rather than remarshalled JSON (#1293)** - **indexer: Implement pruning mechanism (#1176)** - **Provide relevant block data in `ExtendVote` (#1270)** - **node/state:bootstrap state api (#1057)** - **build(deps): Bump actions/checkout from 3 to 4 (#1319)** - **build(deps): Bump docker/build-push-action from 4.1.1 to 4.2.1 (#1318)** - **Fix (#1323)** - **state: remove genesis file from database (#1297)** - **Added changelog for pruning metrics (#1335)** - **Removed begin_block_events and end_block_events from rpc doc (#1338)** - **consensus: test for precommit/locking corner case (#1257)** - **TxIndexer and BlockIndexer pruning metrics (#1334)** - **Forward port changes to changelog and `UPGRADING.md` for release `v0.38.0` (#1341)** - **Update Go version specified in root README on main for v0.34.x (#1352)** - **Add test/e2e/data to gitignore (#1349)** - **docs: gRPC services and data companion pruning services (#1307)** - **Add genesis_hash flag check on node startup (#1324)** - **gRPC for Tx and Block indexer pruning (#1327)** - **build(deps): Bump docker/build-push-action from 4.2.1 to 5.0.0 (#1371)** - **build(deps): Bump docker/setup-buildx-action from 2.10.0 to 3.0.0 (#1370)** - **build(deps): Bump docker/login-action from 2.2.0 to 3.0.0 (#1368)** - **build(deps): Bump goreleaser/goreleaser-action from 4 to 5 (#1369)** - **ADR101: Moved changelog entries (#1377)** - **Max byte check (#1384)** - **Update to string (#1385)** - **mempool: Remove unused peerID constants (#1379)** - **build(deps): Bump github.com/google/uuid from 1.3.0 to 1.3.1 (#1392)** - **build(deps): Bump github.com/rs/cors from 1.9.0 to 1.10.0 (#1393)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.8.1 to 5.9.0 (#1396)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.4 to 2.34.0 (#1400)** - **build(deps): Bump github.com/go-kit/kit from 0.12.0 to 0.13.0 (#1397)** - **build(deps): Bump github.com/golangci/golangci-lint (#1398)** - **build(deps): Bump google.golang.org/grpc from 1.57.0 to 1.58.2 (#1399)** - **doc: improve documentation of BlockParams.MaxBytes (#1405)** - **rpc: Improve abci_query parameter documentation (#1184)** - **Bump p2p version following introduction of `HasProposalBlockPartMessage` (#1411)** - **RFC: Allowing Non-Determinism in `ProcessProposal` (#1391)** - **ADR-101: Data Companion Pull API (#82)** - **build(deps): Bump github.com/rs/cors from 1.10.0 to 1.10.1 (#1418)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.34.0 to 2.34.2 (#1416)** - **build(deps): Bump github.com/prometheus/client_golang (#1417)** - **fix typos (#1413)** - **Renames Tendermint to CometBFT in the RPC related test code (#1428)** - **Renames Tendermint to CometBFT in the RPC related test code. This file was missing and it was not caught in the branch testing. (#1433)** - **Reject incoming txs while node is catching up (#1119)** - **build(deps): Bump golang.org/x/crypto from 0.13.0 to 0.14.0 (#1458)** - **build(deps): Bump golang.org/x/sync from 0.3.0 to 0.4.0 (#1454)** - **build(deps): Bump golang.org/x/net from 0.15.0 to 0.16.0 (#1453)** - **build(deps): Bump bufbuild/buf-setup-action from 1.26.1 to 1.27.0 (#1449)** - **build(deps): Bump styfle/cancel-workflow-action from 0.11.0 to 0.12.0 (#1450)** - **build(deps): Bump github.com/prometheus/client_model (#1455)** - **build(deps): Bump github.com/bufbuild/buf from 1.26.1 to 1.27.0 (#1459)** - **build(deps): Bump github.com/spf13/viper from 1.16.0 to 1.17.0 (#1456)** - **test: respect P2PConfig fuzzing configuration in MultiplexTransport (#1414)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.34.2 to 2.35.3 (#1462)** - **crypto: export errors (#1463)** - **Do not race with pruning to validate headers and validators (#1467)** - **fix typos (#1487)** - **Call observer only when retain height changes (#1490)** - **build(deps): Bump pillow from 9.3.0 to 10.0.1 in /scripts/qa/reporting (#1493)** - **abci: Add relaxed local client synchronization models (#1141)** - **docs: fix typos (#1500)** - **Post-merge fixes for feature/proto-upgrade** - **Make BuildLastCommitInfo and BuildExtendedCommitInfo public (#1502)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.35.3 to 2.35.4 (#1497)** - **build(deps): Bump google.golang.org/grpc from 1.58.2 to 1.58.3 (#1498)** - **build(deps): Bump golang.org/x/net from 0.16.0 to 0.17.0 (#1499)** - **tiny change: reorder expected and actual values (#1506)** - **Fixed docker port alias (#1507)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.0 to 1.27.1 (#1527)** - **Implement ADR-107: rename proto package version suffixes to v1beta* pattern (#1510)** - **e2e: Fix flakiness in grpc tests due to pruning (#1492)** - **fix spelling of bandwidth (#1534)** - **Fix linting on `main` (#1531)** - **build(deps): Bump github.com/prometheus/common from 0.44.0 to 0.45.0 (#1524)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.0 to 1.27.1 (#1522)** - **build(deps): Bump github.com/golangci/golangci-lint (#1540)** - **fix some typos (#1541)** - **indexer-respect-height-params-on-query (#1529)** - **Correct go_package paths in service proto files (#1538)** - **build(deps): Bump google.golang.org/grpc from 1.58.3 to 1.59.0 (#1521)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.35.4 to 2.36.0 (#1523)** - **feat(mempool): export error (#1427)** - **Rename proto messages and services to satisfy default buf lints (#1533)** - **RFC 106: non-idempotent methods in data companion (#1545)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.9.0 to 5.10.0 (#1550)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.1 to 1.27.2 (#1551)** - **build(deps): Bump github.com/google/uuid from 1.3.1 to 1.4.0 (#1552)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.1 to 1.27.2 (#1553)** - **use assert.Greater instead of Equal for better error reporting (#1547)** - **rpc: Version the RPC APIs (#1412)** - **mempool: Add metric size of pool in bytes (#1512)** - **build(deps): Bump github.com/docker/docker (#1570)** - **build(deps): Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 (#1572)** - **build(deps): Bump golang.org/x/sync from 0.4.0 to 0.5.0 (#1574)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.36.0 to 2.36.1 (#1576)** - **build(deps): Bump github.com/gorilla/websocket from 1.5.0 to 1.5.1 (#1575)** - **build(deps): Bump github.com/golangci/golangci-lint (#1573)** - **chore: consolidate changelog improvements section (#1578)** - **Implementation of ADR-108 (#930)** - **Reduce the default MaxBytes to 4mb and increase MaxGas to 10 million (#1518)** - **e2e: Allow disabling the PEX reactor on all nodes in the testnet (#1580)** - **Experimental - Reduce # of connections effectively used to gossip transactions out (#1558)** - **Update proto file references to follow renaming/versioning (#1555)** - **Make `LoadBlock` also return block metadata (#1557)** - **mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584)** - **fix: docs: default db provider moved from node to config (#1588)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.2 to 1.28.0 (#1598)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.2 to 1.28.0 (#1602)** - **build(deps): Bump golang.org/x/crypto from 0.14.0 to 0.15.0 (#1599)** - **build(deps): Bump golang.org/x/net from 0.17.0 to 0.18.0 (#1597)** - **RFC 107: Internal signalling using event observers (#1164)** - **ADR 110: Remote mempool (#1565)** - **chore: aligns function descriptions with godoc standards in the types package (#1593)** - **docs: various improvements (#1603)** - **chore: Merge branch main into feature/proto-upgrade (#1612)** - **Removes wrong assertion and decreases likelyhood of block from the future has actually been already created (#1619)** - **Split /api off to separate go.mod (#1608)** - **feature: Reduce Go API surface area (ADR 109) (#1605)** - **refactor: Export func MakeHTTPDialer (#1594)** - **First draft (#1483)** - **pruning:do not attempt to prune state if no blocks are pruned (#1616)** - **Update SECURITY.md (#1626)** - **Update changelog on `main` with new releases (#1644)** - **proto: Update README (#1648)** - **ADR 111: `nop` Mempool (#1585)** - **ADR-111: Addressed Ethan's comment (#1667)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.36.1 to 2.37.1 (#1662)** - **build(deps): Bump docker/build-push-action from 5.0.0 to 5.1.0 (#1664)** - **build(deps): Bump bufbuild/buf-setup-action from 1.28.0 to 1.28.1 (#1665)** - **ci: Fix v0.34 nightly build (#1655)** - **build(deps): Bump github.com/bufbuild/buf from 1.28.0 to 1.28.1 (#1663)** - **feature/proto-upgrade: Fix generated mock code (#1673)** - **Update link in README.md (#1674)** - **Renames the semantic versioning variable to CMT (#1621)** - **mempool: add `nop` mempool (#1643)** - **e2e:fix-digital-ocean-ports (#1678)** - **Revert modularization of the test infra (#1488) (#1676)** - **docs: various small improvements (part 2) (#1683)** - **Do not block indefinitely on the semaphore (#1654)** - **Adds tests that check for FIFO ordering being broken by gossip (#1628)** - **loadtime: Add parameter for displaying results in one line (#1511)** - **Promote latest versions of Protobuf definitions to v1 packages (#1677)** - **chore: Update CHANGELOG.md (#1701)** - **build(deps): Bump github.com/google/uuid from 1.3.1 to 1.4.0 (#1696)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.37.1 to 2.38.0 (#1697)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.9.0 to 5.10.1 (#1702)** - **proto: document the `cometbft.*` versioned protos (#1694)** - **Clean up changelog entries and README files for versioned proto changes (#1707)** - **chore(proto): add missing docs (#1703)** - **feature/proto-upgrade: Last-minute fixes (#1709)** - **e2e: Support emulated latencies between docker images (#1560)** - **build(deps): Bump golang.org/x/net from 0.14.0 to 0.17.0 in /api (#1713)** - **Prepare `main` for `v1.x` backport branch (#1714)** - **build(deps): Bump golang.org/x/crypto from 0.15.0 to 0.16.0 (#1720)** - **build(deps): Bump golang.org/x/net from 0.18.0 to 0.19.0 (#1719)** - **types: validate Validator#Address field (#1715)** - **Enable Mergify backports for `v1.x` branch (#1724)** - **deps: Bump cometbft-db to v0.9.0 (#1725)** - **buf improvements prior to publishing on BSR (#1726)** - **proto: give our buf module a name (#1734)** - **deps: Bump cometbft-db to v0.9.1 (#1737)** - **chore: Update root docs (#1740)** - **fix: increase abci socket message size limit to 2GB (#1730)** - **proto: make comment in README more Markdowny (#1743)** - **fix: Txs Validate (#1687)** - **Add test missing in #1687 (#1712)** - **perf(state): batch save `State` (#1735)** - **Update CODE_OF_CONDUCT.md (#1708)** - **[e2e] Fixes prepareProposal not to return oversized set of transactions (#1756)** - **perf(store): Batch save `Block` (#1755)** - **Fix: Fix minor typos (#1780)** - **`VerifyCommitLight` and `VerifyCommitLightTrusting` _never_ check all signatures (#1750)** - **Fix typo in encoding.md (#1801)** - **chore(docs): small improvements (#1781)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.10.1 to 5.11.0 (#1797)** - **build(deps): Bump actions/setup-go from 4 to 5 (#1794)** - **build(deps): Bump actions/stale from 8 to 9 (#1795)** - **build(deps): Bump github.com/spf13/viper from 1.17.0 to 1.18.1 (#1796)** - **Chore/fix other typos (#1809)** - **chore(spec): specify which fields must be deterministic (#1804)** - **Small improvements in #1806 not present in #1750 (#1808)** - **Add changelog for #1749 (#1807)** - **e2e: Implement latency emulation for DigitalOcean (#1587)** - **add GH workflow for linting code using codespell (#1824)** - **chore(docs): explain the effect of `timeout_propose` (#1798)** - **build(deps): Bump actions/upload-artifact from 3 to 4 (#1845)** - **build(deps): Bump actions/setup-python from 2 to 5 (#1843)** - **build(deps): Bump github/codeql-action from 2 to 3 (#1847)** - **build(deps): Bump actions/checkout from 2 to 4 (#1846)** - **build(deps): Bump github.com/google/uuid from 1.4.0 to 1.5.0 (#1850)** - **build(deps): Bump google.golang.org/grpc from 1.59.0 to 1.60.0 (#1851)** - **golangci-lint: enable all the linters by default (#1838)** - **Updates go crypto package to v0.17.0 (#1859)** - **Adds checks for nil keys and signatures (#1855)** - **The store DB and the companion `BlockStore` struct are sometimes out of sync (#1856)** - **Allow blocksync to not verify all signatures (#1858)** - **Fixing the ABCI grammar and updating the code for e2e tests to account for the new grammar. (#1829)** - **docs: Fix Discord links in README (#1874)** - **build(deps): Bump github.com/prometheus/client_golang (#1909)** - **build(deps): Bump github.com/btcsuite/btcd/btcutil from 1.1.3 to 1.1.5 (#1908)** - **build(deps): Bump google.golang.org/protobuf (#1885)** - **build(deps): Bump github.com/spf13/viper from 1.18.1 to 1.18.2 (#1884)** - **build(deps): Bump google.golang.org/grpc from 1.60.0 to 1.60.1 (#1883)** - **remove unused linters that produced a warning and ensure that unparam is enabled (#1905)** - **Fix HTTP response body not being closed after reading (#1945)** - **Disable undesired linting code (#1959)** - **Extend kvstore example add with with key types (#1876)** - **Extra check in `VerifyExtension` (#1877)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.38.0 to 2.39.1 (#1882)** - **perf(crypto/merkle, crypto/tmhash): simplify+optimize SHA256 hashing of multiple byteslices (#1921)** - **Update localnet-start Makefile (#1975)** - **blocksync: wait for poolRoutine to stop in (*Reactor).OnStop (#1879)** - **fix TestListenerTimeoutReadWrite (#1947)** - **docs: fix typos (#1988)** - **this is the kindest commit I can think of. Take it that way and read the code. (#1987)** - **there's really no need for this anymore (#1986)** - **feat: DefaultHttpClient support setting proxy from env (#1900)** - **feat: lint tests (#1906)** - **chore: fix typos (#1991)** - **build(deps): Bump golang.org/x/sync from 0.5.0 to 0.6.0 (#1992)** - **feat(proxy): export errors (#1899)** - **doc:fix broken link in DOCKER/README.md (#1996)** - **feat: custom dependencies order (#1994)** - **PBTS: migrating to `main` the new version of the specification (#1973)** - **Fix paths to scripts/metricsgen in go:generate commands (#1998)** - **chore: fix some typos (#2002)** - **docs(light): fix broken link to spec (#2007)** - **chore(test/loadtime): replace tm-load-test w/ cometbft-load-test (#2009)** - **ci: Remove unused `tests` target when compiling e2e (#2022)** - **spec: fix typo in `encoding.md` (#2024)** - **mempool: Fix data races in CListMempool's height and notifiedTxsAvailable (#2021)** - **chore(test/loadtime): cleanup go.sum and golangci.yml (#2025)** - **build(deps): Bump github.com/prometheus/common from 0.45.0 to 0.46.0 (#2040)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.39.1 to 2.40.1 (#2038)** - **feat(rpc): Use default port for HTTP(S) URLs when there is no explicit port (#1903)** - **misc(go.mod): remove unused peg dependency (#2044)** - **config: Remove unused `max_batch_bytes` (#2050)** - **Merge pull request from GHSA-qr8r-m495-7hc4** - **misc(tools)!: remove `tools` package (#2046)** - **docs(guides): add missing import (#2070)** - **deps(localnode): bump alpine version (#2077)** - **misc(Makefile): add `help` target to display the help msg (#2074)** - **`e2e`: test vote extension activation via `InitChain` and `FinalizeBlock` (#2066)** - **scripts: metricsgen parses '// metrics:' pattern (#2090)** - **e2e: Add `load_max_txs` option to manifest (#2094)** - **consensus: Add `chain_size_bytes` metric (#2093)** - **feat: conventional commits (#1995)** - **Forward port of tendermint/tendermint#7605, tendermint/spec#393, and tendermint/tendermint#8142 (#2018)** - **ci: fix permissions for "Conventional PR Title" (#2099)** - **PBTS: additions and fixes on migrated spec (#2013)** - **PBTS: forward port of tendermint/tendermint#7709 (#2089)** - **feat: use go workspace (#1924)** - **fix: `ValidateUpdate`: allow no-change updates regardless of current height (#2112)** - **test(localnet): Add monitoring tools for localnet (#2108)** - **feat: add gofumpt (#2049)** - **feat(consensus): increase log level for corner case when prevoting a proposal (#2042)** - **docs: v1 - diataxis framework (#2105)** - **feat(consensus): additional sanity checks for the size of proposed blocks (#1408)** - **perf(internal/blocksync): avoid double-calling `types.BlockFromProto` (#2016)** - **build(deps): Bump google.golang.org/grpc from 1.60.1 to 1.61.0 (#2178)** - **build(deps): Bump bufbuild/buf-setup-action from 1.28.1 to 1.29.0 (#2161)** - **build(deps): Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 (#2160)** - **build(deps): Bump slackapi/slack-github-action from 1.24.0 to 1.25.0 (#2162)** - **build(deps): Bump github.com/google/uuid from 1.5.0 to 1.6.0 (#2176)** - **refactor(privval): reverse conditional + more idiomatic Go code with early returns (#2156)** - **fix(e2e): path to latency script files in DO (#2148)** - **build(deps): Bump github.com/cometbft/cometbft-db from 0.9.1 to 0.10.0 (#2175)** - **chore(Makefile): add `setup-pre-commit` target to init git hook (#2123)** - **feat: add pebbledb (https://github.com/cockroachdb/pebble) (#2132)** - **perf(internal/blocksync): do not `ValidateBlock` twice (#2026)** - **feat(e2e): Add new targets `fast` and `clean` to Makefile (#2192)** - **ci: add `spec` type in conventional-pr-title (#2191)** - **fix(privval): retry accepting a connection on errors (#2047)** - **fix(.github/workflows): correct parameters to lint_pr_title (#2199)** - **docs: update QA method steps for qa-infra changes (#2198)** - **feat!(pbts): forward port of tendermint/tendermint#7711 (#2149)** - **docs(pbts): forward port of PBTS documentation and fixes (#2124)** - **fix: if-return and early-return (#2215)** - **chore: remove amazon linux Dockerfile and update test/docker/Dockerfile (#2135)** - **feat(ci): pre-commit framework (#2214)** - **ci: add `merge` type for conventional-pr-title (#2219)** - **spec(consensus/pbts): update description for introduced parameters. (#2206)** - **perf: optimize psql indexer (#2142)** - **fix(go.mod): do not use `replace` for api (#2236)** - **fix(Makefile): gofumpt no longer needed (#2235)** - **fix(p2p/pex): gracefully shutdown `Reactor` (#2010)** - **test(mempool): fix TestMempoolUpdateDoesNotPanicWhenApplicationMissedTx (#2242)** - **fix(flowrate): fix non-determinism in flowrate tests (#2147)** - **refactor!: Implement RFC 106: remove `GetLatestBlock*` methods from data companion API (#2240)** - **build(deps): Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 (#2253)** - **chore(config): update DB section in toml.go (#2249)** - **fix(consensus): do not precommit nil if proposal block is received (#2221)** - **build(deps): Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 (#2261)** - **fix(mempool/tests): Reduce tests duration (#2263)** - **ci: run golangci-lint linters for all files (not only ones which were modified) (#2250)** - **refactor(examples): small changes to kvstore app (#2267)** - **refactor(mempool): simplify parameters of resCbFirstTime and resCbRecheck (#2272)** - **perf(mempool/cache): clear map in Reset (#2259)** - **test(abci): update e2e tests to check against ABCI 2.0 grammar (#2201)** - **docs: symbolic references to steps in the qa-infra README (#2280)** - **chore: updated doc link (#2296)** - **fix(consensus): prevote nil upon timeout when Proposal is missing (#2218)** - **build(deps): Bump pre-commit/action from 3.0.0 to 3.0.1 (#2307)** - **build(deps): Bump golang.org/x/net from 0.20.0 to 0.21.0 (#2300)** - **build(deps): Bump github.com/cometbft/cometbft-db from 0.10.0 to 0.11.0 (#2299)** - **docs: ADR-112 Proposer-Based Timestamps (#2223)** - **refactor(internal/statesync): replace ticker with `time.After(X)` (#2294)** - **fix: simultaneous rpc test failures (#2150)** - **chore(docs): alter the section about the effects of `timeout_commit` (#1892)** - **refactor(p2p)!: Refactor PeerSet to eliminate data races & improve performance (#2246)** - **feat(pbts): Update consensus params to include pbts enabled (#2231)** - **feat(storage/metrics): Metrics to measure storage (#1974)** - **feat(e2e): Log number of sent txs (success and failed) (#2328)** - **chore: enable gomoddirectives, unparam (#2290)** - **docs: images not rendering properly in docs (#2331)** - **docs: Fix references about DeliverTx (#2330)** - **docs(qa): Add table of contents to existing QA reports (#2343)** - **chore: enable perfsprint linter (#2291)** - **feat(pruning): trigger explicitly compaction upon pruning (#1972)** - **docs: explain how to skip pre-commit hook (#2358)** - **build(deps): Bump github.com/prometheus/common from 0.46.0 to 0.47.0 (#2373)** - **build(deps): Bump google.golang.org/grpc from 1.61.0 to 1.61.1 (#2372)** - **feat(e2e): Option to set max block size in bytes at genesis (#2362)** - **build(deps): Bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 (#2371)** - **revert(pbts): revert removal of BFT Time related functions (#2205)** - **fix(e2e): Reduce flakiness of TestGRPC_GetBlockResults (#2367)** - **spec(consensus): update Block Time documentation in spec (#2316)** - **spec: adaptive MSGDELAY parameter included in PBTS spec (#2318)** - **feat(pbts): Moving VoteExtensionsEnableHeight from ABCIParams to FeatureParams. (#2335)** - **fix: remove TestMempoolFIFOWithParallelCheckTx (#2364)** - **fix(docker-compose): fix subnet (#2383)** - **feat(pbts): enable `e2e` manifest config to set `PbtsEnableHeight` (#2284)** - **perf(internal/state): avoid double-saving FinalizeBlockResponse (#2017)** - **build(deps): Bump fonttools from 4.37.4 to 4.43.0 in /scripts/qa/reporting (#2407)** - **build(deps): Bump pillow from 10.0.1 to 10.2.0 in /scripts/qa/reporting (#2408)** - **fix(node): Forward compaction config params to blockstore constructor (#2418)** - **refactor!: moved `MedianTime` out of `internal` folder (#2397)** - **chore: typo fixes (#2403)** - **chore(pbts): Removes useless code (#2420)** - **build(deps): Bump github.com/prometheus/common from 0.47.0 to 0.48.0 (#2430)** - **build(deps): Bump google.golang.org/grpc from 1.61.1 to 1.62.0 (#2429)** - **test(consensus): PBTS should be enabled by default in test units (#2329)** - **test(consensus): test enabling PBTS at a given height (#2404)** - **refactor(protobuf): switch from gogofaster to gocosmos generator (#2425)** - **docs(tutorials/install): add `From Go package` (#2414)** - **docs(pbts): documentation for PBTS-related consensus parameters (#2376)** - **refactor(consensus): print err from SignAndCheckVote (#2346)** - **feat(e2e): add parameter to set a custom output directory for testnet files (#2433)** - **spec(consensus): update description of consensus parameters for PBTS (#2415)** - **fix(localnet): Grafana dashboards for storage tests - removed experimental metrics (#2448)** - **fix(jsonrpc): enable HTTP basic auth in WS client (#2434)** - **feat(pbts): Adds timestamp to the `msgInfo` written into the WAL (#2388)** - **fix: `Rollback`: wrong modification of `state.LastHeightValidatorsChanged` while rollback at a special height (#2136)** - **docs(changelog): add missing entry for #2136 (#2459)** - **fix(pbts): legacy `ABCIParams` values are properly translated to `FeatureParams` values (#2462)** - **fix(e2e): Fixing the bug in ABCI e2e tests (#2468)** - **feat(localnet): Grafana PNG rendering for easier graph exports (#2472)** - **docs(README): remove Terra (#2469)** - **docs: document PBTS adaptive delays mechanism (#2452)** - **fix(config): Moved compact variables to the right section (#2477)** - **feat(metrics): more buckets for ProposalTimestampDifference (#2479)** - **fix(metrics): more buckets for ProposalTimestampDifference (#2481)** - **feat(pbts): Make synchrony params adaptive (#2431)** - **ci: check metrics generation in CI checks (#2483)** - **feat(consensus): improve logging for timely and untimely messages (#2321)** - **feat(types): improve Proposal and SynchronyParams validation tests (#2489)** - **test(consensus): refactor TestStateLock_POLSafety tests (#2492)** - **feat(types): refactor types.AdaptiveSynchronyParams method (#2490)** - **fix: lint errors introduced with the latest merge (#2495)** - **build(deps): Bump docker/setup-buildx-action from 3.0.0 to 3.1.0 (#2500)** - **build(deps): Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 (#2508)** - **build(deps): Bump golang.org/x/crypto from 0.19.0 to 0.20.0 (#2507)** - **build(deps): Bump github.com/prometheus/common from 0.48.0 to 0.49.0 (#2506)** - **build(deps): Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2505)** - **fix(docs): include `timesync` daemon instructions (#2491)** - **feat(privval)!: DO NOT require extension signature (#2496)** - **refactor(privval): rename sign_extension to skip_sign_extension (#2519)** - **spec(abci): fixes the spec to inform about the presence of invalid extensions in `last_commit` (#2423)** - **fix(pbts): block Time must be Canonical (#2493)** - **feat(e2e): Add manifest option `clock_skew` and corresponding backend functionality (#2454)** - **fix(mempool): converting to uint64 before additions to avoid overflows (#2498)** - **refactor!: Rename `skip_sign_extension` to `skip_extension_signing` (#2522)** - **docs(ADR): ADR-108 updated to account for the new changes in the grammar and code. (#2528)** - **feat(e2e): add generator support for clock skew + env variable rename (#2488)** - **spec(proto): add `cometbft.privval.v1beta2` proto package (#2529)** - **chore: improving error handling in the PendingEvidence (#2550)** - **feat(blocksync)!: set the max number of (concurrently) downloaded blocks to {peersCount * 20} (#2467)** - **build(deps): Bump bufbuild/buf-setup-action from 1.29.0 to 1.30.0 (#2557)** - **build(deps): Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#2560)** - **build(deps): Bump golang.org/x/net from 0.21.0 to 0.22.0 (#2562)** - **fix(cmd/cometbft/commands/version): update the output for v1 (#2546)** - **build(deps): Bump docker/build-push-action from 5.1.0 to 5.2.0 (#2558)** - **build(deps): Bump github.com/prometheus/common from 0.49.0 to 0.50.0 (#2559)** - **build(deps): Bump google.golang.org/grpc from 1.62.0 to 1.62.1 (#2563)** - **build(deps): Bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#2561)** - **feat(blocksync): sort peers by download rate & multiple requests for closer blocks (#2475)** - **feat(pbts): Adjusts PBTS metrics buckets (#2578)** - **fix(blocksync): use timer instead of time.After (#2584)** - **fix: temporary fix for `api` dependency (#2589)** --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Mikhail Zabaluev <mikhail@informal.systems> Co-authored-by: Troy Kessler <43882936+troykessler@users.noreply.github.com> Co-authored-by: leven <112051166+lx-xiang@users.noreply.github.com> Co-authored-by: werty144 <anton-paramonov2000@yandex.ru> Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Sukey <35202440+sukey2008@users.noreply.github.com> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> Co-authored-by: moriluka <85185077+moriluka@users.noreply.github.com> Co-authored-by: Đỗ Việt Hoàng <hoangdv2429@gmail.com> Co-authored-by: alex <152680487+azukiboy@users.noreply.github.com> Co-authored-by: Hernán Vanzetto <15466498+hvanz@users.noreply.github.com> Co-authored-by: Daniel <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: nenadmilosevic95 <50905385+nenadmilosevic95@users.noreply.github.com> Co-authored-by: nenadmilosevic95 <nenad@informal.systems> Co-authored-by: Jacob Gadikian <jacobgadikian@gmail.com> Co-authored-by: Greg Szabo <16846635+greg-szabo@users.noreply.github.com> Co-authored-by: Emmanuel T Odeke <emm.odeke@gmail.com> Co-authored-by: jchappelow <140431406+jchappelow@users.noreply.github.com> Co-authored-by: shuoer86 <129674997+shuoer86@users.noreply.github.com> Co-authored-by: levisyin <150114626+levisyin@users.noreply.github.com> Co-authored-by: vuittont60 <81072379+vuittont60@users.noreply.github.com> Co-authored-by: Halimao <1065621723@qq.com> Co-authored-by: alex <152680487+bodhi-crypo@users.noreply.github.com> Co-authored-by: Daniel <daniel.cason@usi.ch> Co-authored-by: Josef Widder <44643235+josef-widder@users.noreply.github.com> Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com> Co-authored-by: Kukovec <jure.kukovec@gmail.com> Co-authored-by: Daniel Cason <cason@gandria> Co-authored-by: M. J. Fromberger <fromberger@interchain.io> Co-authored-by: hattizai <150505746+hattizai@users.noreply.github.com> Co-authored-by: Huulu <35667132+IssouChancla@users.noreply.github.com> Co-authored-by: katelyn martin <me+cratelyn@katelyn.world> Co-authored-by: Richard Gibson <richard.gibson@gmail.com> Co-authored-by: Jean Deruelle <jean.deruelle@gmail.com> Co-authored-by: glnro <8335464+glnro@users.noreply.github.com> Co-authored-by: Greg Szabo <greg@philosobear.com> Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com> Co-authored-by: levisyin <lilassherl@gmail.com> Co-authored-by: k0marov <95040709+k0marov@users.noreply.github.com> Co-authored-by: Adi Seredinschi <a@seredinschi.net> Co-authored-by: Pan chao <152830401+Pan-chao@users.noreply.github.com> Co-authored-by: Khanh Hoa <49144992+hoanguyenkh@users.noreply.github.com> Co-authored-by: glnro <lauren@informal.systems> Co-authored-by: toni <143221387+xyztoni@users.noreply.github.com> Co-authored-by: DragonKid <idragonkid@gmail.com> Co-authored-by: Matt Ketmo <matthieu@moquet.net> Co-authored-by: Ethan <cosinlinker@gmail.com> Co-authored-by: Evgeny Danilenko <6655321@bk.ru> Co-authored-by: Duong Minh Ngoc <153509244+minhngoc274@users.noreply.github.com> Co-authored-by: Kero <keroroxx520@gmail.com>

ebuchman added 2 commits May 31, 2023 12:07

consensus: metric for duplicate votes

e1ebb5a

consensus: metric for duplicate block parts

7a4dbf0

ebuchman requested a review from a team as a code owner May 31, 2023 10:30

ebuchman assigned sergio-mena May 31, 2023

make metrics

39d8a80

cason approved these changes May 31, 2023

View reviewed changes

sergio-mena reviewed Jun 1, 2023

View reviewed changes

sergio-mena approved these changes Jun 1, 2023

View reviewed changes

Merge branch 'v0.38.x' into bucky/consensus-duplicate-metrics

eb98008

sergio-mena merged commit bed0dba into v0.38.x Jun 1, 2023
18 checks passed

sergio-mena deleted the bucky/consensus-duplicate-metrics branch June 1, 2023 21:00

ebuchman mentioned this pull request Jun 1, 2023

consensus: optimize vote and block part gossip with HasProposalBlockPartMessage and random sleeps #904

Merged

3 tasks

sergio-mena mentioned this pull request Jun 1, 2023

New metrics to track duplicate votes and block parts (#896) -- Forward port #905

Merged

3 tasks

sergio-mena added the enhancement New feature or request label Jun 1, 2023

sergio-mena added a commit that referenced this pull request Jun 16, 2023

Add changelog for #896

0ad1e55

lasarojc mentioned this pull request Jun 20, 2023

Pinpoint inefficiencies in block, vote and transaction propagation #26

Closed

9 tasks

lasarojc mentioned this pull request Dec 8, 2023

Backport changelog entry for #896 to 0.38.x #1779

Closed

sergio-mena added a commit that referenced this pull request Dec 15, 2023

Backport changelog for #896

5d55f68

sergio-mena mentioned this pull request Dec 15, 2023

Backport changelog for PR #896 to v0.38.x #1835

Merged

3 tasks

sergio-mena added a commit that referenced this pull request Dec 15, 2023

#896 was committed before , so moving it to the right release

06c23f8

sergio-mena added a commit that referenced this pull request Dec 18, 2023

Backport changelog for PR #896 to v0.38.x (#1835)

c0cb233

* Backport changelog for #896 * #896 was committed before , so moving it to the right release

lasarojc mentioned this pull request Mar 12, 2024

chore(spec): merge main into spec/consensus-gossip #2594

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New metrics to track duplicate votes and block parts #896

New metrics to track duplicate votes and block parts #896

ebuchman commented May 31, 2023

lasarojc commented May 31, 2023

cason left a comment

cason May 31, 2023 •

edited

ebuchman Jun 1, 2023

cason May 31, 2023 •

edited

ebuchman Jun 1, 2023

cason May 31, 2023

ebuchman Jun 1, 2023

cason May 31, 2023

ebuchman Jun 1, 2023

sergio-mena Jun 1, 2023

ebuchman Jun 1, 2023

sergio-mena left a comment

ebuchman commented Jun 1, 2023

ebuchman commented Jun 1, 2023

sergio-mena commented Jun 1, 2023

cason commented Jun 3, 2023

New metrics to track duplicate votes and block parts #896

New metrics to track duplicate votes and block parts #896

Conversation

ebuchman commented May 31, 2023

PR checklist

lasarojc commented May 31, 2023

cason left a comment

Choose a reason for hiding this comment

cason May 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cason May 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sergio-mena left a comment

Choose a reason for hiding this comment

ebuchman commented Jun 1, 2023

ebuchman commented Jun 1, 2023

sergio-mena commented Jun 1, 2023

cason commented Jun 3, 2023

cason May 31, 2023 •

edited

cason May 31, 2023 •

edited