ADR-104: Out-of-band state sync support #801

cason · 2023-05-05T19:13:31Z

Closes #28.

This should be a replacement for #729.

yihuang

Can you describe more about the actual design, especially the UX part, what does user need to do to do certain task. Since we have multiple solutions here for the same problem, the UX would the decisive factor I think.

cason · 2023-05-08T06:45:07Z

Can you describe more about the actual design, especially the UX part

This is a work in progress. The above content should go into "Detailed Design" section.

In general terms, the operation should be the one described on the "Decision" section.

cason · 2023-05-08T06:46:28Z

Since we have multiple solutions here for the same problem, the UX would the decisive factor I think.

I agree we can have multiple designs for the user interfaces, but I think that this ADR (with its current content) reasonably covers the possible solutions.

yihuang · 2023-05-08T07:06:08Z

docs/architecture/adr-104-out-of-band-state-sync.md

+   application snapshots to install or restore;
+1. Operators should be able to instruct the server side of State sync on a
+   running node to export application snapshots, when they are available,
+   to a given location (e.g., at the local file system);


Is it like a standalone command that'll connect to the p2p network and download a snapshot?

No p2p, no network, no download. You instruct the node (e.g., via CLI command) to save a snapshot to disk.

Of course, you can save it to a NFS mount point, so you have network and a form of upload.

do you mean export a snapshot from the state in the local node, not discover and download a snapshot from the network?

Discovering and fetching snapshots is what the existing State sync does.

The proposed extension assumes that the node operator will discover and fetch the snapshot in some way and just point to the bootstrapping node (see item 1.) where to find the snapshot data.

this is already how state sync works for the sdk. Def adding support for some of the items here makes sense, but we would only need a way to bootstrap the node, like huang yi did. I think the app should control exports and imports of snapshots for local state sync. There isnt really a need to complicate comet with this functionality imo

Makes sense to rely on the app-side for fulfilling this requirement! Is there already app-side support for it?

we are merging the prs to make it work. they are all open and approved

Links, please?

we are merging the prs to make it work. they are all open and approved

we are merging the prs to make it work. they are all open and approved

cosmos/cosmos-sdk#16067

cosmos/cosmos-sdk#16061

docs/architecture/adr-104-out-of-band-state-sync.md

adizere

Thanks Daniel for writing this down!

docs/architecture/adr-104-out-of-band-state-sync.md

adizere · 2023-05-09T14:52:16Z

docs/architecture/adr-104-out-of-band-state-sync.md

+State sync should support the bootstrap of new nodes from application snapshots
+obtained out-of-band by node operators.


Note that the snapshot might also be obtained locally (i.e., an old, local snapshot present on the node already).

Suggested change

State sync should support the bootstrap of new nodes from application snapshots

obtained out-of-band by node operators.

State sync should support the bootstrap of new nodes from application snapshots. This ADR does not cover the mechanism for obtaining snapshots, and we assume node operators obtain this for example through an out-of-band mechanism.

State sync should support the bootstrap of new nodes from application snapshots

I would add "local" to this sentence? Otherwise, it is what State sync already does...

Is out of band the same as local though? Local to me would mean the node had a snapshot it itself stored before. Out of band can mean it was just copied or obtanined in another manner.

I would say snapshots are local, for the sake of this definition. In general, I would say, from point 2. below, that they are local because obtained out-of-band?

So local in this terminology is that the snapshots are applied from something eventually present on the node. Even though it originated from somewhere else. Sorry, this is just nit picking, nothing major :D

No, that's important. Recall that we should still pick a name for the PR :)

adizere · 2023-05-09T14:53:28Z

docs/architecture/adr-104-out-of-band-state-sync.md

+   application snapshots to install or restore;
+1. Operators should be able to instruct the server side of State sync on a
+   running node to export application snapshots, when they are available,
+   to a given location (e.g., at the local file system);


Makes sense to rely on the app-side for fulfilling this requirement! Is there already app-side support for it?

Co-authored-by: Adi Seredinschi <adizere@gmail.com>

docs/architecture/adr-104-out-of-band-state-sync.md

jmalicevic · 2023-05-18T21:11:05Z

docs/architecture/adr-104-out-of-band-state-sync.md

+1. Operators should be able to instruct the server side of state sync on a
+   running node to export application snapshots, when they are available,
+   to a given location (e.g., at the local file system);
+1. It is up to node operators to transfer application snapshots produced by the


Suggested change

1. It is up to node operators to transfer application snapshots produced by the

3. It is up to node operators to transfer application snapshots produced by the

cason

Good additions.

I think the second part of the detailed design, from line 234, is a little speculative and dives into too much detail. My opinion is that this section should essentially tell the reader how the SDK commands can be implemented using ABCI calls (step by step), leaving out details on how and where we store stuff, for instance.

docs/architecture/adr-104-out-of-band-state-sync.md

cason · 2023-05-24T12:22:22Z

docs/architecture/adr-104-out-of-band-state-sync.md

@@ -51,17 +54,29 @@ state sync, discovering and fetching application snapshots from peers in the
 network, in order to address the above mentioned limitations, while preserving
 most of state sync's operation.

+The ADR presents two solutions:
+
+> 1. The first solution is implemented by the application and was proposed and 


I would remove this > and render it a normal text block

cason · 2023-05-24T12:22:47Z

docs/architecture/adr-104-out-of-band-state-sync.md

+The ADR presents two solutions:
+
+> 1. The first solution is implemented by the application and was proposed and 
+implemented by the SDK (PR #16067 and #16061). This ADR describes the solution 


I would add links to these PRs.

docs/architecture/adr-104-out-of-band-state-sync.md

cason · 2023-05-24T12:23:44Z

docs/architecture/adr-104-out-of-band-state-sync.md

 ## Alternative Approaches

-> This section contains information around alternative options that are considered


This text is from the ADR template, we can remove in the final version.

cason · 2023-05-24T12:43:20Z

docs/architecture/adr-104-out-of-band-state-sync.md

+Thus application developers have to, within their bootstraping command:
+
+- Create a state and block store
+- Launch a light client to verify the snapshot headers


Not the headers. The light client is launched to learn the block header for the height of the snapshot.

cason · 2023-05-24T12:44:03Z

docs/architecture/adr-104-out-of-band-state-sync.md

+
+- Create a state and block store
+- Launch a light client to verify the snapshot headers
+- Use the light client's `State` and `Commit` functions to 


The light-client provided state is used to check the appHash retrieved from the application via Info ABCI call.

cason · 2023-05-24T12:44:39Z

docs/architecture/adr-104-out-of-band-state-sync.md

+- Create a state and block store
+- Launch a light client to verify the snapshot headers
+- Use the light client's `State` and `Commit` functions to 
+retrieve the proper state, verify the `AppHash` and retrieve the `LastCommit`.


It is commit, not last commit. The only point of it is to bootstrap the block store.

You are right the type is not LastCommit, but that is the last commit for the given height. Thanks for catching this.

docs/architecture/adr-104-out-of-band-state-sync.md

adizere

The new ADR content helped answer many of my questions. Thanks Jasmina and Daniel!

I think we should fix any inaccuracies in the ADR and merge this.

docs/architecture/adr-104-out-of-band-state-sync.md

docs/architecture/README.md

docs/architecture/adr-104-out-of-band-state-sync.md

Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Daniel <daniel.cason@informal.systems>

docs/architecture/adr-104-out-of-band-state-sync.md

sergio-mena · 2023-05-25T12:35:07Z

docs/architecture/adr-104-out-of-band-state-sync.md

+2. Introduce a CLI command that can explicitly tell the application to create a snapshot export, in case 
+operators decide not to generate periodical exports.
+3. Extract a snapshot from the exported format.
+4. Alter existing ABCI calls to signal to the application that we want to create a snapshot export periodically. 


If we require that the app is configured for snapshot (something that exists today), when we don't need ABCI changes, we can just call ListSnapshots and (a sequence of) LoadSnapshotChunk.

IOW, we wouldn't need to modify ABCI in an MVP

(I realize this is explained later on in the text)

sergio-mena

Thanks!

cason

Nice writing up, I think this document is covering almost every possible solution for the discussed problem, while providing an overview of the specifics that would have to be taken into consideration by any implementation.

P.S.: GitHub doesn't allow me to approve it because I've originally created the PR.

docs/architecture/adr-104-out-of-band-state-sync.md

cason · 2023-05-26T09:18:17Z

docs/architecture/adr-104-out-of-band-state-sync.md

+
+If CometBFT is not the one who created the exported file, we introduce a new ABCI call `UnpackSnapshot` 
+to send the exported snapshot as a blob of bytes to the application, which uncompresses it, installs it 
+and responds whether the snapshot is accepted and the chunk application has passed. 


Suggested change

and responds whether the snapshot is accepted and the chunk application has passed.

and responds whether the snapshot is accepted (as in `OfferSnapshot`) and the chunks application has passed (as in `LoadChunk`).

docs/architecture/adr-104-out-of-band-state-sync.md

cason · 2023-05-26T09:20:21Z

docs/architecture/adr-104-out-of-band-state-sync.md

+  }
+```
+
+Unlike networked statesync, we do not refetch individual chunks, thus if the chunk application of the exported snapshot fails, snapshot application fails. 


Suggested change

Unlike networked statesync, we do not refetch individual chunks, thus if the chunk application of the exported snapshot fails, snapshot application fails.

Unlike networked state sync, we do not re-fetch individual chunks, thus if the application of a chunk fails, then the application of the whole snapshot fails.

Co-authored-by: Daniel <daniel.cason@informal.systems>

docs/architecture/adr-104-out-of-band-state-sync.md

Co-authored-by: Daniel <daniel.cason@informal.systems>

docs/architecture/adr-104-out-of-band-state-sync.md

- **Versioned protobuf files (#495)** - **Restore rpc/grpc/v1/types.pb.go** - **proto: restore deprecation notice on BroadcastAPI** - **Struct `Client` exposes sensitive data (#784)** - **Unsafe int cast in `kill` command (#783)** - **ADR-100: Data Companion Push API (#73)** - **rfc: incoming txs while catching up (#735)** - **build(deps): Bump bufbuild/buf-setup-action from 1.17.0 to 1.18.0 (#804)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.8 to 1.4.9 (#806)** - **build(deps): Bump golang.org/x/sync from 0.1.0 to 0.2.0 (#807)** - **build(deps): Bump google.golang.org/grpc from 1.54.0 to 1.55.0 (#813)** - **build(deps): Bump github.com/prometheus/client_golang from 1.15.0 to 1.15.1 (#810)** - **build(deps): Bump github.com/bufbuild/buf from 1.17.0 to 1.18.0 (#809)** - **build(deps): Bump github.com/bufbuild/buf from 1.17.0 to 1.18.0 (#809)** - **build(deps): Bump github.com/prometheus/client_model from 0.3.0 to 0.4.0 (#812)** - **build(deps): Bump github.com/prometheus/common from 0.42.0 to 0.43.0 (#811)** - **ADR-103: Protobuf definition versioning (#772)** - **spec/p2p: Specify the operation of a Reactor (#714)** - **build(deps): Bump github.com/docker/distribution (#827)** - **build(deps): Bump github.com/cloudflare/circl from 1.3.1 to 1.3.3 (#828)** - **ADR-103: Protobuf definition versioning (#772) (#817)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.9 to 1.4.10 (#830)** - **build(deps): Bump golang.org/x/net from 0.9.0 to 0.10.0 (#831)** - **build(deps): Bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#832)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.26.1 to 2.27.1 (#833)** - **rpc: Remove response data from response failure logs (#829)** - **pubsub/kvindexer:support for big numbers - v2 (#797)** - **Extend the infrastructure provider with `StartNodes` and `StopTestnet` (#796)** - **grpc: Add base gRPC server with version service (#818)** - **Digital Ocean implementation of `StartNodes` and `StopTestnet` (#846)** - **RFC 102: Improve forward compatibility of proto-generated Rust (#724)** - **mempool: Add metric to measure how many times a tx was received (#637)** - **`e2e` provider is extended with `Disconnect`, `Reconnect` and `CheckUpgraded` (#852)** - **Sort `loadtime` tool's report (#854)** - **feat: make handshake cancelable (#857)** - **proto: Restore snake_case JSON for `ExecTxResult` (#856)** - **qa: 200-nodes test on v0.38 (#877)** - **qa: rotating node test on `v0.38.x` (#883)** - **Changes to QA related files (e.g., `method.md`) (#878)** - **build(deps): Bump bufbuild/buf-setup-action from 1.18.0 to 1.19.0 (#868)** - **build(deps): Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#873)** - **build(deps): Bump github.com/bufbuild/buf from 1.18.0 to 1.19.0 (#872)** - **Adds `vote_extension_size` to manifests (#858)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.27.1 to 2.28.1 (#893)** - **build(deps): Bump github.com/prometheus/common from 0.43.0 to 0.44.0 (#892)** - **build(deps): Bump slackapi/slack-github-action from 1.23.0 to 1.24.0 (#869)** - **ADR 104: State sync from local application snapshot (#801)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 (#891)** - **mempool: slight refactor for improving readability (#894)** - **Add Vote Extension varying size testnet (#888)** - **fix: lint fail when golangci-lint bump from v1.52.2 to v1.53.x (#908)** - **build(deps): Bump github.com/bufbuild/buf from 1.19.0 to 1.20.0 (#912)** - **build(deps): Bump github.com/golangci/golangci-lint (#917)** - **build(deps): Bump github.com/spf13/viper from 1.15.0 to 1.16.0 (#913)** - **build(deps): Bump bufbuild/buf-setup-action from 1.19.0 to 1.20.0 (#910)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.28.1 to 2.28.2 (#914)** - **Minor fixes to `RELEASES.md` (#923)** - **Prevent a transaction to appear twice in the mempool (#890)** - **New metrics to track duplicate votes and block parts (#896) (#905)** - **e2e tests are being skipped; `INFRASTRUCTURE-*` bug; portGen inconsistency bug (#933)** - **build(deps): Bump github.com/bufbuild/buf from 1.20.0 to 1.21.0 (#948)** - **build(deps): Bump bufbuild/buf-setup-action from 1.20.0 to 1.21.0 (#946)** - **spec/p2p: document the p2p API used by Reactors (#851)** - **build(deps): Bump docker/login-action from 2.1.0 to 2.2.0 (#944)** - **e2e: Generate prometheus.yaml on setup (#954)** - **Update the type annotations in the light client spec (#955)** - **build(deps): Bump docker/build-push-action from 4.0.0 to 4.1.0 (#947)** - **build(deps): Bump docker/setup-buildx-action from 2.5.0 to 2.7.0 (#962)** - **Add requirement for `CheckTx` in ABCI spec (#928)** - **fix: avoid recursive call after rename to (*PeerState).MarshalJSON (#865)** - **Port upstream proto changes to cometbft package** - **changelog: Import entries for v0.34.28, v0.34.29, v0.37.1 and v0.37.2 to `main` (#974)** - **test/e2e: Fix Docker image build (#984)** - **build(deps): Bump github.com/BurntSushi/toml from 1.3.0 to 1.3.2 (#952)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.28.2 to 2.29.0 (#953)** - **Remove duplicate function `deterministicExecTxResult` (#986)** - **mempool: Fix the benchmarks (#934)** - **Add `CMT_HOME` (or remove it?) (#983)** - **Remove buf.yaml at the root of the source tree (#979)** - **build(deps): Bump docker/build-push-action from 4.1.0 to 4.1.1 (#989)** - **build(deps): Bump golang.org/x/net from 0.10.0 to 0.11.0 (#994)** - **build(deps): Bump golang.org/x/sync from 0.2.0 to 0.3.0 (#990)** - **build(deps): Bump google.golang.org/grpc from 1.55.0 to 1.56.0 (#992)** - **build(deps): Bump github.com/golangci/golangci-lint from 1.53.2 to 1.53.3 (#996)** - **build(deps): Bump github.com/prometheus/client_golang from 1.15.1 to 1.16.0 (#991)** - **spec/p2p: new structure for the p2p specification (#966)** - **consensus: optimize vote and block part gossip with HasProposalBlockPartMessage and random sleeps (#904)** - **Extend ABCI `max_block_size` parameter to give extended control to the app (#1003)** - **build(deps): Bump bufbuild/buf-setup-action from 1.21.0 to 1.22.0 (#1026)** - **Bump logs on non-fatal vote validation failures to Info level (#1022)** - **Update codeowners to include DevRel team (#1023)** - **deps: Manually bump github.com/vektra/mockery/v2 from 2.29.0 to 2.30.1 (#1009)** - **Update releasing.md to sign tags (#1034)** - **build(deps): Bump github.com/bufbuild/buf from 1.21.0 to 1.22.0 (#1027)** - **Rename ABCI enums and values to satisfy buf guidelines (#975)** - **build(deps): Bump google.golang.org/grpc from 1.56.0 to 1.56.1 (#1028)** - **docs: Added double quotes to /abci_query path param (#1015)** - **Update Docs with Finalize Block (#760)** - **Clarifies that processProposal may be called for set of transactions different from the one returned in the preceding prepareProposal (#1033)** - **Update the annotation of part_set.go (#1056)** - **build(deps): Bump docker/setup-buildx-action from 2.7.0 to 2.8.0 (#1071)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.30.1 to 2.30.16 (#1064)** - **build(deps): Bump bufbuild/buf-setup-action from 1.22.0 to 1.23.1 (#1072)** - **build(deps): Bump github.com/bufbuild/buf from 1.22.0 to 1.23.1 (#1066)** - **build(deps): Bump google.golang.org/protobuf from 1.30.0 to 1.31.0 (#1065)** - **build(deps): Bump google.golang.org/grpc from 1.56.1 to 1.56.2 (#1101)** - **build(deps): Bump golang.org/x/crypto from 0.10.0 to 0.11.0 (#1103)** - **build(deps): Bump docker/setup-buildx-action from 2.8.0 to 2.9.0 (#1105)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.30.16 to 2.31.1 (#1100)** - **build(deps): Bump golang.org/x/net from 0.11.0 to 0.12.0 (#1102)** - **ci: Trigger workflows on merge group (#1118)** - **Revert "config: add bootstrap peers (#9680)" (#1109)** - **node: Revert removal of public reactor accessors (#1120)** - **ci: Disable CodeQL check in merge queues (#1123)** - **p2p: Remove UPnP functionality (#1114)** - **ADR 107: Rename proto versions to pre-v1 betas (#1110)** - **RFC 104: Internal messaging using the actor model (#1092)** - **build(deps): Bump github.com/bufbuild/buf from 1.23.1 to 1.24.0 (#1131)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.31.1 to 2.32.0 (#1132)** - **build(deps): Bump docker/setup-buildx-action from 2.9.0 to 2.9.1 (#1133)** - **build(deps): Bump bufbuild/buf-setup-action from 1.23.1 to 1.24.0 (#1134)** - **spec: Add mempool specification in English and Quint (#997)** - **mempool: ADR for refactoring list of senders (#1032)** - **build(deps): Bump bufbuild/buf-setup-action from 1.24.0 to 1.25.0 (#1157)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.7.0 to 5.8.0 (#1159)** - **build(deps): Bump github.com/bufbuild/buf from 1.24.0 to 1.25.0 (#1160)** - **proxy: Rename "unsynchronized" to "connection-synchronized" local client creator (#1145)** - **mempool: Keep track of senders in reactor instead of implementation (#1010)** - **cmd: Remove `replay` and `replay-console` subcommands (#1170)** - **build(deps): Bump google.golang.org/grpc from 1.56.2 to 1.57.0 (#1181)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.8.0 to 5.8.1 (#1180)** - **ci: Remove Mergify automerge (#1182)** - **Add gRPC block service (#1142)** - **blocksync: export errors (#1186)** - **Porting changes related to creating a lean docker image from `pierre/fast-prototyping-1059` branch. (#1192)** - **chore: Format repo (#1193)** - **Add BlockResults gRPC service (#1168)** - **consensus: remove logic to unlock block on +2/3 prevote for nil (#1175)** - **ADR 101: Add GetLatest method to block service (#1209)** - **ADR 101: Implement pruning mechanism (#1150)** - **config: export errors (#1190)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.0 to 2.32.3 (#1218)** - **Add check for non-`nil` in `enterCommit` (#1208)** - **Log proposer's address when correctly accepting a proposal (#1079)** - **Close evidence.db OnStop (#1210)** - **build(deps): Bump bufbuild/buf-setup-action from 1.25.0 to 1.25.1 (#1213)** - **build(deps): Bump golang.org/x/crypto from 0.11.0 to 0.12.0 (#1215)** - **build(deps): Bump github.com/bufbuild/buf from 1.25.0 to 1.25.1 (#1216)** - **build(deps): Bump golang.org/x/net from 0.12.0 to 0.14.0 (#1217)** - **Forward-port: update state to prevote `nil` when proposal block does not match locked block (#1203)** - **mempool: Store peer ids as p2p.ID instead of uint16 (remove mempoolIDs) (#1191)** - **ADR-101: implement gRPC `PruningService` (#1154)** - **consensus: Avoid process proposal when we know correct nodes validated it (#1231)** - **ADR-101: Metrics to monitor the pruning (#1234)** - **build(deps): Bump bufbuild/buf-setup-action from 1.25.1 to 1.26.0 (#1241)** - **build(deps): Bump github.com/bufbuild/buf from 1.25.1 to 1.26.1 (#1239)** - **build(deps): Bump github.com/golangci/golangci-lint (#1240)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.3 to 2.32.4 (#1242)** - **abci: export errors (#1185)** - **consensus: export errors (#1211)** - **ADR-101: Pruning mechanism minor fixes (#1246)** - **chore: Bump minimum Go version on `main` to v1.21 (#1244)** - **ADR 101: Add `Close` method to gRPC client (#1251)** - **ADR 108: ADR for extending E2E infrastructure so that we can check CometBFT's behaviour with respect to ABCI++ grammar. (#902)** - **docs: Add logging guide to contributing guidelines (#1250)** - **chore: log `app_hash` as hex (#1264)** - **update language (#1263)** - **build(deps): Bump bufbuild/buf-setup-action from 1.26.0 to 1.26.1 (#1272)** - **build(deps): Bump gonum.org/v1/gonum from 0.13.0 to 0.14.0 (#1275)** - **build(deps): Bump github.com/cosmos/gogoproto from 1.4.10 to 1.4.11 (#1276)** - **ADR 101: Refactor height check-related logic and tests (#1271)** - **Fixed linter** - **Revert "Fixed linter"** - **Fixed mem aliasing in for loop (#1280)** - **Remove genesis persistence in state db (#1017)** - **Revert "Remove genesis persistence in state db (#1017)" (#1294)** - **evidence: export errors (#1284)** - **Re-implement removal of genesis persistence in state db (#1295)** - **build(deps): Bump docker/setup-buildx-action from 2.9.1 to 2.10.0 (#1298)** - **node: verify genesis doc hash against the file contents rather than remarshalled JSON (#1293)** - **indexer: Implement pruning mechanism (#1176)** - **Provide relevant block data in `ExtendVote` (#1270)** - **node/state:bootstrap state api (#1057)** - **build(deps): Bump actions/checkout from 3 to 4 (#1319)** - **build(deps): Bump docker/build-push-action from 4.1.1 to 4.2.1 (#1318)** - **Fix (#1323)** - **state: remove genesis file from database (#1297)** - **Added changelog for pruning metrics (#1335)** - **Removed begin_block_events and end_block_events from rpc doc (#1338)** - **consensus: test for precommit/locking corner case (#1257)** - **TxIndexer and BlockIndexer pruning metrics (#1334)** - **Forward port changes to changelog and `UPGRADING.md` for release `v0.38.0` (#1341)** - **Update Go version specified in root README on main for v0.34.x (#1352)** - **Add test/e2e/data to gitignore (#1349)** - **docs: gRPC services and data companion pruning services (#1307)** - **Add genesis_hash flag check on node startup (#1324)** - **gRPC for Tx and Block indexer pruning (#1327)** - **build(deps): Bump docker/build-push-action from 4.2.1 to 5.0.0 (#1371)** - **build(deps): Bump docker/setup-buildx-action from 2.10.0 to 3.0.0 (#1370)** - **build(deps): Bump docker/login-action from 2.2.0 to 3.0.0 (#1368)** - **build(deps): Bump goreleaser/goreleaser-action from 4 to 5 (#1369)** - **ADR101: Moved changelog entries (#1377)** - **Max byte check (#1384)** - **Update to string (#1385)** - **mempool: Remove unused peerID constants (#1379)** - **build(deps): Bump github.com/google/uuid from 1.3.0 to 1.3.1 (#1392)** - **build(deps): Bump github.com/rs/cors from 1.9.0 to 1.10.0 (#1393)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.8.1 to 5.9.0 (#1396)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.32.4 to 2.34.0 (#1400)** - **build(deps): Bump github.com/go-kit/kit from 0.12.0 to 0.13.0 (#1397)** - **build(deps): Bump github.com/golangci/golangci-lint (#1398)** - **build(deps): Bump google.golang.org/grpc from 1.57.0 to 1.58.2 (#1399)** - **doc: improve documentation of BlockParams.MaxBytes (#1405)** - **rpc: Improve abci_query parameter documentation (#1184)** - **Bump p2p version following introduction of `HasProposalBlockPartMessage` (#1411)** - **RFC: Allowing Non-Determinism in `ProcessProposal` (#1391)** - **ADR-101: Data Companion Pull API (#82)** - **build(deps): Bump github.com/rs/cors from 1.10.0 to 1.10.1 (#1418)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.34.0 to 2.34.2 (#1416)** - **build(deps): Bump github.com/prometheus/client_golang (#1417)** - **fix typos (#1413)** - **Renames Tendermint to CometBFT in the RPC related test code (#1428)** - **Renames Tendermint to CometBFT in the RPC related test code. This file was missing and it was not caught in the branch testing. (#1433)** - **Reject incoming txs while node is catching up (#1119)** - **build(deps): Bump golang.org/x/crypto from 0.13.0 to 0.14.0 (#1458)** - **build(deps): Bump golang.org/x/sync from 0.3.0 to 0.4.0 (#1454)** - **build(deps): Bump golang.org/x/net from 0.15.0 to 0.16.0 (#1453)** - **build(deps): Bump bufbuild/buf-setup-action from 1.26.1 to 1.27.0 (#1449)** - **build(deps): Bump styfle/cancel-workflow-action from 0.11.0 to 0.12.0 (#1450)** - **build(deps): Bump github.com/prometheus/client_model (#1455)** - **build(deps): Bump github.com/bufbuild/buf from 1.26.1 to 1.27.0 (#1459)** - **build(deps): Bump github.com/spf13/viper from 1.16.0 to 1.17.0 (#1456)** - **test: respect P2PConfig fuzzing configuration in MultiplexTransport (#1414)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.34.2 to 2.35.3 (#1462)** - **crypto: export errors (#1463)** - **Do not race with pruning to validate headers and validators (#1467)** - **fix typos (#1487)** - **Call observer only when retain height changes (#1490)** - **build(deps): Bump pillow from 9.3.0 to 10.0.1 in /scripts/qa/reporting (#1493)** - **abci: Add relaxed local client synchronization models (#1141)** - **docs: fix typos (#1500)** - **Post-merge fixes for feature/proto-upgrade** - **Make BuildLastCommitInfo and BuildExtendedCommitInfo public (#1502)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.35.3 to 2.35.4 (#1497)** - **build(deps): Bump google.golang.org/grpc from 1.58.2 to 1.58.3 (#1498)** - **build(deps): Bump golang.org/x/net from 0.16.0 to 0.17.0 (#1499)** - **tiny change: reorder expected and actual values (#1506)** - **Fixed docker port alias (#1507)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.0 to 1.27.1 (#1527)** - **Implement ADR-107: rename proto package version suffixes to v1beta* pattern (#1510)** - **e2e: Fix flakiness in grpc tests due to pruning (#1492)** - **fix spelling of bandwidth (#1534)** - **Fix linting on `main` (#1531)** - **build(deps): Bump github.com/prometheus/common from 0.44.0 to 0.45.0 (#1524)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.0 to 1.27.1 (#1522)** - **build(deps): Bump github.com/golangci/golangci-lint (#1540)** - **fix some typos (#1541)** - **indexer-respect-height-params-on-query (#1529)** - **Correct go_package paths in service proto files (#1538)** - **build(deps): Bump google.golang.org/grpc from 1.58.3 to 1.59.0 (#1521)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.35.4 to 2.36.0 (#1523)** - **feat(mempool): export error (#1427)** - **Rename proto messages and services to satisfy default buf lints (#1533)** - **RFC 106: non-idempotent methods in data companion (#1545)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.9.0 to 5.10.0 (#1550)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.1 to 1.27.2 (#1551)** - **build(deps): Bump github.com/google/uuid from 1.3.1 to 1.4.0 (#1552)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.1 to 1.27.2 (#1553)** - **use assert.Greater instead of Equal for better error reporting (#1547)** - **rpc: Version the RPC APIs (#1412)** - **mempool: Add metric size of pool in bytes (#1512)** - **build(deps): Bump github.com/docker/docker (#1570)** - **build(deps): Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 (#1572)** - **build(deps): Bump golang.org/x/sync from 0.4.0 to 0.5.0 (#1574)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.36.0 to 2.36.1 (#1576)** - **build(deps): Bump github.com/gorilla/websocket from 1.5.0 to 1.5.1 (#1575)** - **build(deps): Bump github.com/golangci/golangci-lint (#1573)** - **chore: consolidate changelog improvements section (#1578)** - **Implementation of ADR-108 (#930)** - **Reduce the default MaxBytes to 4mb and increase MaxGas to 10 million (#1518)** - **e2e: Allow disabling the PEX reactor on all nodes in the testnet (#1580)** - **Experimental - Reduce # of connections effectively used to gossip transactions out (#1558)** - **Update proto file references to follow renaming/versioning (#1555)** - **Make `LoadBlock` also return block metadata (#1557)** - **mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584)** - **fix: docs: default db provider moved from node to config (#1588)** - **build(deps): Bump github.com/bufbuild/buf from 1.27.2 to 1.28.0 (#1598)** - **build(deps): Bump bufbuild/buf-setup-action from 1.27.2 to 1.28.0 (#1602)** - **build(deps): Bump golang.org/x/crypto from 0.14.0 to 0.15.0 (#1599)** - **build(deps): Bump golang.org/x/net from 0.17.0 to 0.18.0 (#1597)** - **RFC 107: Internal signalling using event observers (#1164)** - **ADR 110: Remote mempool (#1565)** - **chore: aligns function descriptions with godoc standards in the types package (#1593)** - **docs: various improvements (#1603)** - **chore: Merge branch main into feature/proto-upgrade (#1612)** - **Removes wrong assertion and decreases likelyhood of block from the future has actually been already created (#1619)** - **Split /api off to separate go.mod (#1608)** - **feature: Reduce Go API surface area (ADR 109) (#1605)** - **refactor: Export func MakeHTTPDialer (#1594)** - **First draft (#1483)** - **pruning:do not attempt to prune state if no blocks are pruned (#1616)** - **Update SECURITY.md (#1626)** - **Update changelog on `main` with new releases (#1644)** - **proto: Update README (#1648)** - **ADR 111: `nop` Mempool (#1585)** - **ADR-111: Addressed Ethan's comment (#1667)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.36.1 to 2.37.1 (#1662)** - **build(deps): Bump docker/build-push-action from 5.0.0 to 5.1.0 (#1664)** - **build(deps): Bump bufbuild/buf-setup-action from 1.28.0 to 1.28.1 (#1665)** - **ci: Fix v0.34 nightly build (#1655)** - **build(deps): Bump github.com/bufbuild/buf from 1.28.0 to 1.28.1 (#1663)** - **feature/proto-upgrade: Fix generated mock code (#1673)** - **Update link in README.md (#1674)** - **Renames the semantic versioning variable to CMT (#1621)** - **mempool: add `nop` mempool (#1643)** - **e2e:fix-digital-ocean-ports (#1678)** - **Revert modularization of the test infra (#1488) (#1676)** - **docs: various small improvements (part 2) (#1683)** - **Do not block indefinitely on the semaphore (#1654)** - **Adds tests that check for FIFO ordering being broken by gossip (#1628)** - **loadtime: Add parameter for displaying results in one line (#1511)** - **Promote latest versions of Protobuf definitions to v1 packages (#1677)** - **chore: Update CHANGELOG.md (#1701)** - **build(deps): Bump github.com/google/uuid from 1.3.1 to 1.4.0 (#1696)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.37.1 to 2.38.0 (#1697)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.9.0 to 5.10.1 (#1702)** - **proto: document the `cometbft.*` versioned protos (#1694)** - **Clean up changelog entries and README files for versioned proto changes (#1707)** - **chore(proto): add missing docs (#1703)** - **feature/proto-upgrade: Last-minute fixes (#1709)** - **e2e: Support emulated latencies between docker images (#1560)** - **build(deps): Bump golang.org/x/net from 0.14.0 to 0.17.0 in /api (#1713)** - **Prepare `main` for `v1.x` backport branch (#1714)** - **build(deps): Bump golang.org/x/crypto from 0.15.0 to 0.16.0 (#1720)** - **build(deps): Bump golang.org/x/net from 0.18.0 to 0.19.0 (#1719)** - **types: validate Validator#Address field (#1715)** - **Enable Mergify backports for `v1.x` branch (#1724)** - **deps: Bump cometbft-db to v0.9.0 (#1725)** - **buf improvements prior to publishing on BSR (#1726)** - **proto: give our buf module a name (#1734)** - **deps: Bump cometbft-db to v0.9.1 (#1737)** - **chore: Update root docs (#1740)** - **fix: increase abci socket message size limit to 2GB (#1730)** - **proto: make comment in README more Markdowny (#1743)** - **fix: Txs Validate (#1687)** - **Add test missing in #1687 (#1712)** - **perf(state): batch save `State` (#1735)** - **Update CODE_OF_CONDUCT.md (#1708)** - **[e2e] Fixes prepareProposal not to return oversized set of transactions (#1756)** - **perf(store): Batch save `Block` (#1755)** - **Fix: Fix minor typos (#1780)** - **`VerifyCommitLight` and `VerifyCommitLightTrusting` _never_ check all signatures (#1750)** - **Fix typo in encoding.md (#1801)** - **chore(docs): small improvements (#1781)** - **build(deps): Bump github.com/go-git/go-git/v5 from 5.10.1 to 5.11.0 (#1797)** - **build(deps): Bump actions/setup-go from 4 to 5 (#1794)** - **build(deps): Bump actions/stale from 8 to 9 (#1795)** - **build(deps): Bump github.com/spf13/viper from 1.17.0 to 1.18.1 (#1796)** - **Chore/fix other typos (#1809)** - **chore(spec): specify which fields must be deterministic (#1804)** - **Small improvements in #1806 not present in #1750 (#1808)** - **Add changelog for #1749 (#1807)** - **e2e: Implement latency emulation for DigitalOcean (#1587)** - **add GH workflow for linting code using codespell (#1824)** - **chore(docs): explain the effect of `timeout_propose` (#1798)** - **build(deps): Bump actions/upload-artifact from 3 to 4 (#1845)** - **build(deps): Bump actions/setup-python from 2 to 5 (#1843)** - **build(deps): Bump github/codeql-action from 2 to 3 (#1847)** - **build(deps): Bump actions/checkout from 2 to 4 (#1846)** - **build(deps): Bump github.com/google/uuid from 1.4.0 to 1.5.0 (#1850)** - **build(deps): Bump google.golang.org/grpc from 1.59.0 to 1.60.0 (#1851)** - **golangci-lint: enable all the linters by default (#1838)** - **Updates go crypto package to v0.17.0 (#1859)** - **Adds checks for nil keys and signatures (#1855)** - **The store DB and the companion `BlockStore` struct are sometimes out of sync (#1856)** - **Allow blocksync to not verify all signatures (#1858)** - **Fixing the ABCI grammar and updating the code for e2e tests to account for the new grammar. (#1829)** - **docs: Fix Discord links in README (#1874)** - **build(deps): Bump github.com/prometheus/client_golang (#1909)** - **build(deps): Bump github.com/btcsuite/btcd/btcutil from 1.1.3 to 1.1.5 (#1908)** - **build(deps): Bump google.golang.org/protobuf (#1885)** - **build(deps): Bump github.com/spf13/viper from 1.18.1 to 1.18.2 (#1884)** - **build(deps): Bump google.golang.org/grpc from 1.60.0 to 1.60.1 (#1883)** - **remove unused linters that produced a warning and ensure that unparam is enabled (#1905)** - **Fix HTTP response body not being closed after reading (#1945)** - **Disable undesired linting code (#1959)** - **Extend kvstore example add with with key types (#1876)** - **Extra check in `VerifyExtension` (#1877)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.38.0 to 2.39.1 (#1882)** - **perf(crypto/merkle, crypto/tmhash): simplify+optimize SHA256 hashing of multiple byteslices (#1921)** - **Update localnet-start Makefile (#1975)** - **blocksync: wait for poolRoutine to stop in (*Reactor).OnStop (#1879)** - **fix TestListenerTimeoutReadWrite (#1947)** - **docs: fix typos (#1988)** - **this is the kindest commit I can think of. Take it that way and read the code. (#1987)** - **there's really no need for this anymore (#1986)** - **feat: DefaultHttpClient support setting proxy from env (#1900)** - **feat: lint tests (#1906)** - **chore: fix typos (#1991)** - **build(deps): Bump golang.org/x/sync from 0.5.0 to 0.6.0 (#1992)** - **feat(proxy): export errors (#1899)** - **doc:fix broken link in DOCKER/README.md (#1996)** - **feat: custom dependencies order (#1994)** - **PBTS: migrating to `main` the new version of the specification (#1973)** - **Fix paths to scripts/metricsgen in go:generate commands (#1998)** - **chore: fix some typos (#2002)** - **docs(light): fix broken link to spec (#2007)** - **chore(test/loadtime): replace tm-load-test w/ cometbft-load-test (#2009)** - **ci: Remove unused `tests` target when compiling e2e (#2022)** - **spec: fix typo in `encoding.md` (#2024)** - **mempool: Fix data races in CListMempool's height and notifiedTxsAvailable (#2021)** - **chore(test/loadtime): cleanup go.sum and golangci.yml (#2025)** - **build(deps): Bump github.com/prometheus/common from 0.45.0 to 0.46.0 (#2040)** - **build(deps): Bump github.com/vektra/mockery/v2 from 2.39.1 to 2.40.1 (#2038)** - **feat(rpc): Use default port for HTTP(S) URLs when there is no explicit port (#1903)** - **misc(go.mod): remove unused peg dependency (#2044)** - **config: Remove unused `max_batch_bytes` (#2050)** - **Merge pull request from GHSA-qr8r-m495-7hc4** - **misc(tools)!: remove `tools` package (#2046)** - **docs(guides): add missing import (#2070)** - **deps(localnode): bump alpine version (#2077)** - **misc(Makefile): add `help` target to display the help msg (#2074)** - **`e2e`: test vote extension activation via `InitChain` and `FinalizeBlock` (#2066)** - **scripts: metricsgen parses '// metrics:' pattern (#2090)** - **e2e: Add `load_max_txs` option to manifest (#2094)** - **consensus: Add `chain_size_bytes` metric (#2093)** - **feat: conventional commits (#1995)** - **Forward port of tendermint/tendermint#7605, tendermint/spec#393, and tendermint/tendermint#8142 (#2018)** - **ci: fix permissions for "Conventional PR Title" (#2099)** - **PBTS: additions and fixes on migrated spec (#2013)** - **PBTS: forward port of tendermint/tendermint#7709 (#2089)** - **feat: use go workspace (#1924)** - **fix: `ValidateUpdate`: allow no-change updates regardless of current height (#2112)** - **test(localnet): Add monitoring tools for localnet (#2108)** - **feat: add gofumpt (#2049)** - **feat(consensus): increase log level for corner case when prevoting a proposal (#2042)** - **docs: v1 - diataxis framework (#2105)** - **feat(consensus): additional sanity checks for the size of proposed blocks (#1408)** - **perf(internal/blocksync): avoid double-calling `types.BlockFromProto` (#2016)** - **build(deps): Bump google.golang.org/grpc from 1.60.1 to 1.61.0 (#2178)** - **build(deps): Bump bufbuild/buf-setup-action from 1.28.1 to 1.29.0 (#2161)** - **build(deps): Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 (#2160)** - **build(deps): Bump slackapi/slack-github-action from 1.24.0 to 1.25.0 (#2162)** - **build(deps): Bump github.com/google/uuid from 1.5.0 to 1.6.0 (#2176)** - **refactor(privval): reverse conditional + more idiomatic Go code with early returns (#2156)** - **fix(e2e): path to latency script files in DO (#2148)** - **build(deps): Bump github.com/cometbft/cometbft-db from 0.9.1 to 0.10.0 (#2175)** - **chore(Makefile): add `setup-pre-commit` target to init git hook (#2123)** - **feat: add pebbledb (https://github.com/cockroachdb/pebble) (#2132)** - **perf(internal/blocksync): do not `ValidateBlock` twice (#2026)** - **feat(e2e): Add new targets `fast` and `clean` to Makefile (#2192)** - **ci: add `spec` type in conventional-pr-title (#2191)** - **fix(privval): retry accepting a connection on errors (#2047)** - **fix(.github/workflows): correct parameters to lint_pr_title (#2199)** - **docs: update QA method steps for qa-infra changes (#2198)** - **feat!(pbts): forward port of tendermint/tendermint#7711 (#2149)** - **docs(pbts): forward port of PBTS documentation and fixes (#2124)** - **fix: if-return and early-return (#2215)** - **chore: remove amazon linux Dockerfile and update test/docker/Dockerfile (#2135)** - **feat(ci): pre-commit framework (#2214)** - **ci: add `merge` type for conventional-pr-title (#2219)** - **spec(consensus/pbts): update description for introduced parameters. (#2206)** - **perf: optimize psql indexer (#2142)** - **fix(go.mod): do not use `replace` for api (#2236)** - **fix(Makefile): gofumpt no longer needed (#2235)** - **fix(p2p/pex): gracefully shutdown `Reactor` (#2010)** - **test(mempool): fix TestMempoolUpdateDoesNotPanicWhenApplicationMissedTx (#2242)** - **fix(flowrate): fix non-determinism in flowrate tests (#2147)** - **refactor!: Implement RFC 106: remove `GetLatestBlock*` methods from data companion API (#2240)** - **build(deps): Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 (#2253)** - **chore(config): update DB section in toml.go (#2249)** - **fix(consensus): do not precommit nil if proposal block is received (#2221)** - **build(deps): Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 (#2261)** - **fix(mempool/tests): Reduce tests duration (#2263)** - **ci: run golangci-lint linters for all files (not only ones which were modified) (#2250)** - **refactor(examples): small changes to kvstore app (#2267)** - **refactor(mempool): simplify parameters of resCbFirstTime and resCbRecheck (#2272)** - **perf(mempool/cache): clear map in Reset (#2259)** - **test(abci): update e2e tests to check against ABCI 2.0 grammar (#2201)** - **docs: symbolic references to steps in the qa-infra README (#2280)** - **chore: updated doc link (#2296)** - **fix(consensus): prevote nil upon timeout when Proposal is missing (#2218)** - **build(deps): Bump pre-commit/action from 3.0.0 to 3.0.1 (#2307)** - **build(deps): Bump golang.org/x/net from 0.20.0 to 0.21.0 (#2300)** - **build(deps): Bump github.com/cometbft/cometbft-db from 0.10.0 to 0.11.0 (#2299)** - **docs: ADR-112 Proposer-Based Timestamps (#2223)** - **refactor(internal/statesync): replace ticker with `time.After(X)` (#2294)** - **fix: simultaneous rpc test failures (#2150)** - **chore(docs): alter the section about the effects of `timeout_commit` (#1892)** - **refactor(p2p)!: Refactor PeerSet to eliminate data races & improve performance (#2246)** - **feat(pbts): Update consensus params to include pbts enabled (#2231)** - **feat(storage/metrics): Metrics to measure storage (#1974)** - **feat(e2e): Log number of sent txs (success and failed) (#2328)** - **chore: enable gomoddirectives, unparam (#2290)** - **docs: images not rendering properly in docs (#2331)** - **docs: Fix references about DeliverTx (#2330)** - **docs(qa): Add table of contents to existing QA reports (#2343)** - **chore: enable perfsprint linter (#2291)** - **feat(pruning): trigger explicitly compaction upon pruning (#1972)** - **docs: explain how to skip pre-commit hook (#2358)** - **build(deps): Bump github.com/prometheus/common from 0.46.0 to 0.47.0 (#2373)** - **build(deps): Bump google.golang.org/grpc from 1.61.0 to 1.61.1 (#2372)** - **feat(e2e): Option to set max block size in bytes at genesis (#2362)** - **build(deps): Bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 (#2371)** - **revert(pbts): revert removal of BFT Time related functions (#2205)** - **fix(e2e): Reduce flakiness of TestGRPC_GetBlockResults (#2367)** - **spec(consensus): update Block Time documentation in spec (#2316)** - **spec: adaptive MSGDELAY parameter included in PBTS spec (#2318)** - **feat(pbts): Moving VoteExtensionsEnableHeight from ABCIParams to FeatureParams. (#2335)** - **fix: remove TestMempoolFIFOWithParallelCheckTx (#2364)** - **fix(docker-compose): fix subnet (#2383)** - **feat(pbts): enable `e2e` manifest config to set `PbtsEnableHeight` (#2284)** - **perf(internal/state): avoid double-saving FinalizeBlockResponse (#2017)** - **build(deps): Bump fonttools from 4.37.4 to 4.43.0 in /scripts/qa/reporting (#2407)** - **build(deps): Bump pillow from 10.0.1 to 10.2.0 in /scripts/qa/reporting (#2408)** - **fix(node): Forward compaction config params to blockstore constructor (#2418)** - **refactor!: moved `MedianTime` out of `internal` folder (#2397)** - **chore: typo fixes (#2403)** - **chore(pbts): Removes useless code (#2420)** - **build(deps): Bump github.com/prometheus/common from 0.47.0 to 0.48.0 (#2430)** - **build(deps): Bump google.golang.org/grpc from 1.61.1 to 1.62.0 (#2429)** - **test(consensus): PBTS should be enabled by default in test units (#2329)** - **test(consensus): test enabling PBTS at a given height (#2404)** - **refactor(protobuf): switch from gogofaster to gocosmos generator (#2425)** - **docs(tutorials/install): add `From Go package` (#2414)** - **docs(pbts): documentation for PBTS-related consensus parameters (#2376)** - **refactor(consensus): print err from SignAndCheckVote (#2346)** - **feat(e2e): add parameter to set a custom output directory for testnet files (#2433)** - **spec(consensus): update description of consensus parameters for PBTS (#2415)** - **fix(localnet): Grafana dashboards for storage tests - removed experimental metrics (#2448)** - **fix(jsonrpc): enable HTTP basic auth in WS client (#2434)** - **feat(pbts): Adds timestamp to the `msgInfo` written into the WAL (#2388)** - **fix: `Rollback`: wrong modification of `state.LastHeightValidatorsChanged` while rollback at a special height (#2136)** - **docs(changelog): add missing entry for #2136 (#2459)** - **fix(pbts): legacy `ABCIParams` values are properly translated to `FeatureParams` values (#2462)** - **fix(e2e): Fixing the bug in ABCI e2e tests (#2468)** - **feat(localnet): Grafana PNG rendering for easier graph exports (#2472)** - **docs(README): remove Terra (#2469)** - **docs: document PBTS adaptive delays mechanism (#2452)** - **fix(config): Moved compact variables to the right section (#2477)** - **feat(metrics): more buckets for ProposalTimestampDifference (#2479)** - **fix(metrics): more buckets for ProposalTimestampDifference (#2481)** - **feat(pbts): Make synchrony params adaptive (#2431)** - **ci: check metrics generation in CI checks (#2483)** - **feat(consensus): improve logging for timely and untimely messages (#2321)** - **feat(types): improve Proposal and SynchronyParams validation tests (#2489)** - **test(consensus): refactor TestStateLock_POLSafety tests (#2492)** - **feat(types): refactor types.AdaptiveSynchronyParams method (#2490)** - **fix: lint errors introduced with the latest merge (#2495)** - **build(deps): Bump docker/setup-buildx-action from 3.0.0 to 3.1.0 (#2500)** - **build(deps): Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 (#2508)** - **build(deps): Bump golang.org/x/crypto from 0.19.0 to 0.20.0 (#2507)** - **build(deps): Bump github.com/prometheus/common from 0.48.0 to 0.49.0 (#2506)** - **build(deps): Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2505)** - **fix(docs): include `timesync` daemon instructions (#2491)** - **feat(privval)!: DO NOT require extension signature (#2496)** - **refactor(privval): rename sign_extension to skip_sign_extension (#2519)** - **spec(abci): fixes the spec to inform about the presence of invalid extensions in `last_commit` (#2423)** - **fix(pbts): block Time must be Canonical (#2493)** - **feat(e2e): Add manifest option `clock_skew` and corresponding backend functionality (#2454)** - **fix(mempool): converting to uint64 before additions to avoid overflows (#2498)** - **refactor!: Rename `skip_sign_extension` to `skip_extension_signing` (#2522)** - **docs(ADR): ADR-108 updated to account for the new changes in the grammar and code. (#2528)** - **feat(e2e): add generator support for clock skew + env variable rename (#2488)** - **spec(proto): add `cometbft.privval.v1beta2` proto package (#2529)** - **chore: improving error handling in the PendingEvidence (#2550)** - **feat(blocksync)!: set the max number of (concurrently) downloaded blocks to {peersCount * 20} (#2467)** - **build(deps): Bump bufbuild/buf-setup-action from 1.29.0 to 1.30.0 (#2557)** - **build(deps): Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#2560)** - **build(deps): Bump golang.org/x/net from 0.21.0 to 0.22.0 (#2562)** - **fix(cmd/cometbft/commands/version): update the output for v1 (#2546)** - **build(deps): Bump docker/build-push-action from 5.1.0 to 5.2.0 (#2558)** - **build(deps): Bump github.com/prometheus/common from 0.49.0 to 0.50.0 (#2559)** - **build(deps): Bump google.golang.org/grpc from 1.62.0 to 1.62.1 (#2563)** - **build(deps): Bump golang.org/x/crypto from 0.20.0 to 0.21.0 (#2561)** - **feat(blocksync): sort peers by download rate & multiple requests for closer blocks (#2475)** - **feat(pbts): Adjusts PBTS metrics buckets (#2578)** - **fix(blocksync): use timer instead of time.After (#2584)** - **fix: temporary fix for `api` dependency (#2589)** --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Mikhail Zabaluev <mikhail@informal.systems> Co-authored-by: Troy Kessler <43882936+troykessler@users.noreply.github.com> Co-authored-by: leven <112051166+lx-xiang@users.noreply.github.com> Co-authored-by: werty144 <anton-paramonov2000@yandex.ru> Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Sukey <35202440+sukey2008@users.noreply.github.com> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> Co-authored-by: moriluka <85185077+moriluka@users.noreply.github.com> Co-authored-by: Đỗ Việt Hoàng <hoangdv2429@gmail.com> Co-authored-by: alex <152680487+azukiboy@users.noreply.github.com> Co-authored-by: Hernán Vanzetto <15466498+hvanz@users.noreply.github.com> Co-authored-by: Daniel <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: nenadmilosevic95 <50905385+nenadmilosevic95@users.noreply.github.com> Co-authored-by: nenadmilosevic95 <nenad@informal.systems> Co-authored-by: Jacob Gadikian <jacobgadikian@gmail.com> Co-authored-by: Greg Szabo <16846635+greg-szabo@users.noreply.github.com> Co-authored-by: Emmanuel T Odeke <emm.odeke@gmail.com> Co-authored-by: jchappelow <140431406+jchappelow@users.noreply.github.com> Co-authored-by: shuoer86 <129674997+shuoer86@users.noreply.github.com> Co-authored-by: levisyin <150114626+levisyin@users.noreply.github.com> Co-authored-by: vuittont60 <81072379+vuittont60@users.noreply.github.com> Co-authored-by: Halimao <1065621723@qq.com> Co-authored-by: alex <152680487+bodhi-crypo@users.noreply.github.com> Co-authored-by: Daniel <daniel.cason@usi.ch> Co-authored-by: Josef Widder <44643235+josef-widder@users.noreply.github.com> Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com> Co-authored-by: Kukovec <jure.kukovec@gmail.com> Co-authored-by: Daniel Cason <cason@gandria> Co-authored-by: M. J. Fromberger <fromberger@interchain.io> Co-authored-by: hattizai <150505746+hattizai@users.noreply.github.com> Co-authored-by: Huulu <35667132+IssouChancla@users.noreply.github.com> Co-authored-by: katelyn martin <me+cratelyn@katelyn.world> Co-authored-by: Richard Gibson <richard.gibson@gmail.com> Co-authored-by: Jean Deruelle <jean.deruelle@gmail.com> Co-authored-by: glnro <8335464+glnro@users.noreply.github.com> Co-authored-by: Greg Szabo <greg@philosobear.com> Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com> Co-authored-by: levisyin <lilassherl@gmail.com> Co-authored-by: k0marov <95040709+k0marov@users.noreply.github.com> Co-authored-by: Adi Seredinschi <a@seredinschi.net> Co-authored-by: Pan chao <152830401+Pan-chao@users.noreply.github.com> Co-authored-by: Khanh Hoa <49144992+hoanguyenkh@users.noreply.github.com> Co-authored-by: glnro <lauren@informal.systems> Co-authored-by: toni <143221387+xyztoni@users.noreply.github.com> Co-authored-by: DragonKid <idragonkid@gmail.com> Co-authored-by: Matt Ketmo <matthieu@moquet.net> Co-authored-by: Ethan <cosinlinker@gmail.com> Co-authored-by: Evgeny Danilenko <6655321@bk.ru> Co-authored-by: Duong Minh Ngoc <153509244+minhngoc274@users.noreply.github.com> Co-authored-by: Kero <keroroxx520@gmail.com>

ADR-104: Out-of-band state sync support (start)

72e892a

cason self-assigned this May 5, 2023

cason added 2 commits May 5, 2023 21:25

ADR-104: changelog, status, some references

1c36145

ADR-104: Manual application state transfer alternative

add02d1

adizere mentioned this pull request May 7, 2023

ADR 103 Local State Sync Support (ported from Tendermint Core repo) #729

Closed

4 tasks

cason mentioned this pull request May 8, 2023

State sync from local snapshot: Implementation #29

Closed

3 tasks

cason added 2 commits May 8, 2023 05:59

ADR-104: Consequences and some general additions

ea729cc

ADR-104: Alternative Approaches section improved

e3a3e8d

yihuang suggested changes May 8, 2023

View reviewed changes

ADR-104: added link to original issue on old repo

23b9a2c

yihuang reviewed May 8, 2023

View reviewed changes

tac0turtle reviewed May 8, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Outdated Show resolved Hide resolved

adizere reviewed May 9, 2023

View reviewed changes

cometbft deleted a comment from tac0turtle May 9, 2023

cason added the wip Work in progress label May 9, 2023

cason and others added 2 commits May 9, 2023 20:56

ADR 104: applying reviewer suggestion

3d81e1e

Co-authored-by: Adi Seredinschi <adizere@gmail.com>

ADR 104: using lowercase for state sync

bf86fd2

adizere added this to the 2023-Q2 milestone May 11, 2023

jmalicevic reviewed May 18, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Show resolved Hide resolved

jmalicevic reviewed May 18, 2023

View reviewed changes

added descriptions of SDK and ABCI based solutions

c7d1d38

cason commented May 24, 2023

View reviewed changes

adizere approved these changes May 24, 2023

View reviewed changes

adizere reviewed May 24, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Outdated Show resolved Hide resolved

adizere marked this pull request as ready for review May 24, 2023 15:25

adizere requested a review from a team as a code owner May 24, 2023 15:25

adizere mentioned this pull request May 24, 2023

Add Go API support to bootstrap node after it was state synced #884

Closed

Apply suggestions from code review

12f521f

Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Daniel <daniel.cason@informal.systems>

sergio-mena reviewed May 25, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Show resolved Hide resolved

sergio-mena reviewed May 25, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Outdated Show resolved Hide resolved

sergio-mena reviewed May 25, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Outdated Show resolved Hide resolved

sergio-mena reviewed May 25, 2023

View reviewed changes

sergio-mena approved these changes May 25, 2023

View reviewed changes

jmalicevic added 4 commits May 25, 2023 21:30

Minor fixes

fcf06be

Added additional arguments against 103

ca0a68b

Clarified bootstrap

3badcad

minor fizes

3718dd2

cason assigned jmalicevic and unassigned cason May 26, 2023

cason commented May 26, 2023

View reviewed changes

jmalicevic and others added 4 commits May 26, 2023 13:48

Apply suggestions from code review

8443a84

Co-authored-by: Daniel <daniel.cason@informal.systems>

Addressed @cason's comments

6b05f49

Merge branch 'main' into cason/adr-104

a31ff23

Updated status

c53a5a4

cason commented May 30, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Show resolved Hide resolved

cason commented May 30, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Outdated Show resolved Hide resolved

jmalicevic and others added 3 commits May 30, 2023 11:53

Apply suggestions from code review

75809e8

Co-authored-by: Daniel <daniel.cason@informal.systems>

Merge branch 'cason/adr-104' of github.com:cometbft/cometbft into adr104

5d91794

fixed readme

98f181a

cason commented May 30, 2023

View reviewed changes

docs/architecture/adr-104-out-of-band-state-sync.md Show resolved Hide resolved

Update docs/architecture/adr-104-out-of-band-state-sync.md

286e9af

cason merged commit 4e130bd into main May 30, 2023
23 checks passed

cason deleted the cason/adr-104 branch May 30, 2023 10:31

cason mentioned this pull request May 30, 2023

State sync from local snapshot #27

Closed

2 tasks

adizere removed the wip Work in progress label May 31, 2023

jmalicevic mentioned this pull request Oct 19, 2023

Backport state bootstrap API to v0.37.x #1513

Closed

lasarojc mentioned this pull request Mar 12, 2024

chore(spec): merge main into spec/consensus-gossip #2594

Merged

		State sync should support the bootstrap of new nodes from application snapshots
		obtained out-of-band by node operators.

	1. It is up to node operators to transfer application snapshots produced by the
	3. It is up to node operators to transfer application snapshots produced by the

		## Alternative Approaches

		> This section contains information around alternative options that are considered

	and responds whether the snapshot is accepted and the chunk application has passed.
	and responds whether the snapshot is accepted (as in `OfferSnapshot`) and the chunks application has passed (as in `LoadChunk`).

	Unlike networked statesync, we do not refetch individual chunks, thus if the chunk application of the exported snapshot fails, snapshot application fails.
	Unlike networked state sync, we do not re-fetch individual chunks, thus if the application of a chunk fails, then the application of the whole snapshot fails.

ADR-104: Out-of-band state sync support #801

ADR-104: Out-of-band state sync support #801

Conversation

cason commented May 5, 2023 • edited Loading

yihuang left a comment • edited Loading

Choose a reason for hiding this comment

cason commented May 8, 2023

cason commented May 8, 2023

yihuang May 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cason May 8, 2023 • edited Loading

Choose a reason for hiding this comment

yihuang May 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tac0turtle May 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adizere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cason left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adizere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sergio-mena left a comment

Choose a reason for hiding this comment

cason left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cason commented May 5, 2023 •

edited

Loading

yihuang left a comment •

edited

Loading

yihuang May 8, 2023 •

edited

Loading

cason May 8, 2023 •

edited

Loading

yihuang May 8, 2023 •

edited

Loading

tac0turtle May 10, 2023 •

edited

Loading

cason left a comment •

edited

Loading