Skip to content
This repository has been archived by the owner on Aug 24, 2022. It is now read-only.

Flakey Test: Invalid Header "time from the future" #241

Closed
ejfitzgerald opened this issue Mar 2, 2021 · 1 comment
Closed

Flakey Test: Invalid Header "time from the future" #241

ejfitzgerald opened this issue Mar 2, 2021 · 1 comment

Comments

@ejfitzgerald
Copy link
Member

The latest tests on master fail because due to an invalid timestamp.

2021/03/02 09:11:40 verify non adjacent from #2 to #1826 failed: invalid header: new header has a time from the future 2021-03-02 09:41:53.670730121 +0000 UTC (now: 2021-03-02 09:41:40.078103274 +0000 UTC m=+1818.218398677; max clock drift: 10s)
daeMOn63 added a commit that referenced this issue Apr 30, 2021
* fix #241 : flaky test reporting invalid header time
Backported from v0.34.10 fix at tendermint/tendermint@4b99502
daeMOn63 added a commit that referenced this issue Apr 30, 2021
* mvp blacklist alg

* move reinstatement after if statement in pex reactor

* separate reinstatement of peers from discovery of new peers

* lint fix

* build(deps): bump google.golang.org/grpc from 1.27.1 to 1.28.0 (#4551)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.27.1 to 1.28.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.27.1...v1.28.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* build(deps): bump github.com/tendermint/tm-db from 0.4.1 to 0.5.0 (#4554)

Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.4.1 to 0.5.0.
- [Release notes](https://github.com/tendermint/tm-db/releases)
- [Changelog](https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md)
- [Commits](tendermint/tm-db@v0.4.1...v0.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* reconfigure mutexes

* ban function requires a ban duration as an argument

* make banTime an argument and set default in PEX reactor instead of AddrBook

* basic test for banning peers

* added banned address error

* banned addresses can't be added again

* added isBanned check in addrbook

* added logs for more information

* lint fix

* deps: bump deps that bot cant (#4555)

* deps: bump deps that bot cant

- bumping deps that dependat bot does not do.

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* run go mod tidy

* fix go.sum

* tools/build: delete stale tools (#4558)

* lite2: add benchmarking tests (#4514)

Closes #4392
Refs #4504

* privval: return error on getpubkey (#4534)

closes #3602

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* made suggested changes

* cannot decrease ban time

* update Changelog with PR #4548

* moved to changelog_pending

* Use docs-staging branch (#4561)

Co-authored-by: Marko <marbar3778@yahoo.com>

* build(deps): bump github.com/golang/protobuf from 1.3.4 to 1.3.5 (#4563)

Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.3.4 to 1.3.5.
- [Release notes](https://github.com/golang/protobuf/releases)
- [Commits](golang/protobuf@v1.3.4...v1.3.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* removal: remove build folder (#4565)

- remove tools/build folder

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* Update docs website (#4564)

* update theme

* Update version

* Updated Questions section in the footer

* Remove links to Riot chat

* Typo

* Add Discord link

* Update docs theme to the latest version

* Use docs-staging branch for staging website

* Resolve merge conflicts

* Update version

* Add google analytics

Co-authored-by: Marko <marbar3778@yahoo.com>

* fix: fix spelling of comment (#4566)

- :P

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* build(deps): bump github.com/prometheus/client_golang (#4574)

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md)
- [Commits](prometheus/client_golang@v1.5.0...v1.5.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* rpc: fix panic when `Subscribe` is called (#4570)

but HTTP client is not running.

`Subscribe`, `Unsubscribe(All)` methods return an error now.

Closes #4568

* Dockerfile updated with defaults (#4577)

* build(deps): bump github.com/gorilla/websocket from 1.4.1 to 1.4.2 (#4584)

Bumps [github.com/gorilla/websocket](https://github.com/gorilla/websocket) from 1.4.1 to 1.4.2.
- [Release notes](https://github.com/gorilla/websocket/releases)
- [Commits](gorilla/websocket@v1.4.1...v1.4.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* deps: run go mod tidy (#4587)

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* Docker image port fix (#4589)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* format: add format cmd & goimport repo (#4586)

* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes

* minor link fixing (#4598)

* blockchain: enable v2 to be set (#4597)

* blockchain: enable v2 to be set

- enable v2 to be set via config params

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* replace tab with space

* correctly spell usability

* change link checker to run daily (#4601)

* lite2: cache headers in bisection (#4562)

Closes: #4546

The algorithm uses an array to store the headers and validators and populates it at every bisection (which is an unsuccessful verification). When a successful verification finally occurs it updates the new trusted header, trims that header from the cache (the array) and sets the depth pointer back to 0. Instead of retrieving new headers it will use the cached headers, incrementing in depth until it reaches the end of the cache which by then it will start to retrieve new headers from the provider. 

Mathematically, this method doesn't properly bisect after the first round but it will always choose a pivot header that is within 1/8th of the upper header's height. I.e. if we are trying to jump 128 headers, the maximum offset from bisection height (64) is 64 + 16(128/8) = 80, therefore a better heuristic would be to obtain the new pivot header height as the middle of these two numbers which would therefore mean to multiply it by 9/16ths instead of 1/2  (sorry this might be a bit more complicated in writing but I can try better explain if someone is interested). Therefore I would also, upon consensus, propose that we change the pivot height to 9/16th's of the previous height

* tools: remove need to install buf (#4605)

* tools: remove need to install buf

- using buf docker image instead of needing devs to install it

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix ci for lint and break checking

* create style guide and update contributors section (#4576)

* docs: validator setup & Key info (#4604)

- defined what variables needed to be changed in the `config.toml` in order to run a validator.
- Briefly explained how a sentry node archtecture should look
- add section explaing importance of key secruity

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* all: name reactors when they are initialized (#4608)

Previously, many reactors were initialized with the name "Reactor," which made it difficult to log which reactor was doing what. This changes those reactors' names to something more descriptive.

* build(deps): bump github.com/spf13/cobra from 0.0.6 to 0.0.7 (#4612)

Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 0.0.6 to 0.0.7.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](spf13/cobra@v0.0.6...0.0.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* build(deps): bump github.com/tendermint/tm-db from 0.5.0 to 0.5.1 (#4613)

Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.5.0 to 0.5.1.
- [Release notes](https://github.com/tendermint/tm-db/releases)
- [Changelog](https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md)
- [Commits](tendermint/tm-db@v0.5.0...v0.5.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* proto: use docker to generate stubs (#4615)

* proto: use docker to generate stubs

- provide an option to developers to use docker to generate proto stubs

closes #4579

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* rpc: add codespace to ResultBroadcastTx (#4611)

Closes: #4606

* lite2: use bisection for some of backward verification (#4575)

Closes: #4537

Uses SignedHeaderBefore to find header before unverified header and then bisection to verify the header. Only when header is between first and last trusted header height else if before the first trusted header height then regular backwards verification is used.

* lite2: make maxClockDrift an option (#4616)

Closes #4607

* rpc/client: split out client packages (#4628)

* rpc/client: initial split into directories

* lite2: split out test package

* rpc/client: simplify client constructurs

* updated docs

* updated changelog

* ADR-053: strengthen and simplify the state sync ABCI interface (#4610)

* adr-053: strengthened and simplified ABCI interface

* remove internal_error reason and open questions

* add follow-up task for block backfill

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* add support for block pruning via ABCI Commit response (#4588)

* Added BlockStore.DeleteBlock()

* Added initial block pruner prototype

* wip

* Added BlockStore.PruneBlocks()

* Added consensus setting for block pruning

* Added BlockStore base

* Error on replay if base does not have blocks

* Handle missing blocks when sending VoteSetMaj23Message

* Error message tweak

* Properly update blockstore state

* Error message fix again

* blockchain: ignore peer missing blocks

* Added FIXME

* Added test for block replay with truncated history

* Handle peer base in blockchain reactor

* Improved replay error handling

* Added tests for Store.PruneBlocks()

* Fix non-RPC handling of truncated block history

* Panic on missing block meta in needProofBlock()

* Updated changelog

* Handle truncated block history in RPC layer

* Added info about earliest block in /status RPC

* Reorder height and base in blockchain reactor messages

* Updated changelog

* Fix tests

* Appease linter

* Minor review fixes

* Non-empty BlockStores should always have base > 0

* Update code to assume base > 0 invariant

* Added blockstore tests for pruning to 0

* Make sure we don't prune below the current base

* Added BlockStore.Size()

* config: added retain_blocks recommendations

* Update v1 blockchain reactor to handle blockstore base

* Added state database pruning

* Propagate errors on missing validator sets

* Comment tweaks

* Improved error message

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* use ABCI field ResponseCommit.retain_height instead of retain-blocks config option

* remove State.RetainHeight, return value instead

* fix minor issues

* rename pruneHeights() to pruneBlocks()

* noop to fix GitHub borkage

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* upgrading: add note on rpc/client subpackages (#4636)

* upgrading: add note on rpc/client subpackages

* clarify constructors

* config: allow fastsync.version = v2 (#4639)

* lite2: Prevent falsely returned double voting error (#4620)

* prevent faulty double voting error

* create test

* clean tests

* clean tests

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* lite2: Default to http scheme in provider.New (#4649)

Closes: #4643

* p2p: PEX message abuse should ban as well as disconnect (#4621)

* mark unsolicited and too frequent messaged as bad

* add tests

* update changelog and fix error

* revised error types

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* lint: add review dog (#4652)

* lint: add review dog

- golangci is being deprecated on the 15th

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* check evidence hasn't already been stored (#4632)

Add Has function, create better handling of errors when adding evidence, usage of error types.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* build(deps): bump google.golang.org/grpc from 1.28.0 to 1.28.1 (#4653)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.28.0 to 1.28.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.28.0...v1.28.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* run go mod tidy

* Docs website update (#4658)

* mempool: reserve IDs in InitPeer instead of AddPeer

* p2p: limit the number of incoming connections

to p2p.max_num_inbound_peers + len(p2p.unconditional_peer_ids)

* update changelog and version

* README: specify supported versions (#4660)

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* build(deps): bump github.com/spf13/viper from 1.6.2 to 1.6.3 (#4664)

Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.6.2 to 1.6.3.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](spf13/viper@v1.6.2...v1.6.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* cli: add command to generate shell completion scripts (#4665)

How to use it:

```
$ . <(tendermint completion)
```

Note that the completion command does not show up in the help screen,
though it comes with its own --help option.

This is a port of the feature provided by cosmos-sdk.

* mergify: use PR title and body for squash merge commit (#4669)

* blockchain/v2: allow setting nil switch, for CustomReactors()

<!-- < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < ☺
v                               ✰  Thanks for creating a PR! ✰    
v    Before smashing the submit button please review the checkboxes.
v    If a checkbox is n/a - please still include it but + a little note why
☺ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >  -->

Fixes an issue reported in tendermint/tendermint#4595 (comment).

Not sure if this is sufficient to fully remove the reactor, but it fixes the immediate problem.
______

For contributor use:

- [x] Wrote tests
- [x] ~Updated CHANGELOG_PENDING.md~
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [x] ~Updated relevant documentation (`docs/`) and code comments~
- [x] Re-reviewed `Files changed` in the Github PR explorer

* evidence: both MaxAgeDuration and MaxAgeNumBlocks need to be surpassed (#4667)

for evidence to be considered expired. otherwise, a cabal group can
  manipulate block time to make a particular evidence too old.

Refs tendermint/tendermint#2565 (comment)
Refs tendermint/tendermint#2653
spec PR: tendermint/spec#87

* dep: bump protobuf, cobra, btcutil & std lib deps (#4676)

* docs/architecture: add adr-55 for proto repo design (#4623)

* changelog: add entries from secruity releases



______

For contributor use:

- [ ] Wrote tests
- [ ] Updated CHANGELOG_PENDING.md
- [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] Updated relevant documentation (`docs/`) and code comments
- [ ] Re-reviewed `Files changed` in the Github PR explorer

* docs: amend adr-54 with changes in the sdk (#4684)

* blockchain/v2: don't broadcast base if height is 0

## Description

Fixes a bug where the reactor would broadcast a base with height=0.

______

For contributor use:

- [ ] Wrote tests
- [ ] Updated CHANGELOG_PENDING.md
- [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] Updated relevant documentation (`docs/`) and code comments
- [x] Re-reviewed `Files changed` in the Github PR explorer

* build(deps): bump @vuepress/plugin-google-analytics in /docs (#4692)

Bumps [@vuepress/plugin-google-analytics](https://github.com/vuejs/vuepress/tree/HEAD/packages/@vuepress/plugin-google-analytics) from 1.3.1 to 1.4.1.
- [Release notes](https://github.com/vuejs/vuepress/releases)
- [Changelog](https://github.com/vuejs/vuepress/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/vuepress/commits/v1.4.1/packages/@vuepress/plugin-google-analytics)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* minor release process change

## Description

The minor release process is changing in order to not have major release changes sitting in the pull request tab.

This changes from taking master and releasing from master to creating a branch that you cherry-pick commits into. 

There are two options on labeling which pull requests to include in a minor release:
1. Use the label `R:minor` to know which pull requests to include then remove the label when those pull requests have been included in a release.
2. Create an Issue where pull request numbers are added. then the issue is closed when the release is done.

this process should be followed after 0.33.3

______

For contributor use:

- [ ] Wrote tests
- [ ] Updated CHANGELOG_PENDING.md
- [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] Updated relevant documentation (`docs/`) and code comments
- [ ] Re-reviewed `Files changed` in the Github PR explorer

* lite2: verify ConsensusHash in rpc client


______

For contributor use:

- [ ] Wrote tests
- [ ] Updated CHANGELOG_PENDING.md
- [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] Updated relevant documentation (`docs/`) and code comments
- [x] Re-reviewed `Files changed` in the Github PR explorer

* docs/architecture: create adr 56: prove amnesia attack

## Description

ADR to address the process for proving an amnesia attack (as a form of global evidence) from `PotentialAmnesiaEvidence` detected by light clients


______

For contributor use:

- [ ] Wrote tests
- [ ] Updated CHANGELOG_PENDING.md
- [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] Updated relevant documentation (`docs/`) and code comments
- [ ] Re-reviewed `Files changed` in the Github PR explorer

* bump ABCI version to 0.16.2 due to ResponseCommit.retain_height

See #4588 for original change.

I believe this is appropriate. Anything else that needs to be updated?

______

For contributor use:

- [ ] ~Wrote tests~
- [x] Updated CHANGELOG_PENDING.md
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] ~Updated relevant documentation (`docs/`) and code comments~
- [x] Re-reviewed `Files changed` in the Github PR explorer

* fix linter warnings

* clarify service logging

The service logging can be a bit unclear. For example, with state sync it would log:

```
I[2020-04-20|08:40:47.366] Starting StateSync     module=statesync impl=Reactor
I[2020-04-20|08:40:47.834] Starting state sync    module=statesync
```

Where the first message is the reactor service startup, and the second message is the start of the actual state sync process. This clarifies the first message by changing it to `Starting StateSync service`.

______

For contributor use:

- [ ] ~Wrote tests~
- [ ] ~Updated CHANGELOG_PENDING.md~
- [ ] ~Linked to Github issue with discussion and accepted design OR link to spec that describes this work.~
- [ ] ~Updated relevant documentation (`docs/`) and code comments~
- [x] Re-reviewed `Files changed` in the Github PR explorer

* blockstore: allow initial SaveBlock() at any height

Followup from #4588. Allow the first `SaveBlock()` call in an empty block store to be at any height, to start from a truncated block history. Subsequent `SaveBlock()` calls must be for contiguous blocks.

______

For contributor use:

- [x] Wrote tests
- [ ] ~Updated CHANGELOG_PENDING.md~
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [x] Updated relevant documentation (`docs/`) and code comments
- [x] Re-reviewed `Files changed` in the Github PR explorer

* ci: transition some ci to github actions

## Description

move tests for abci_cli, abci_app and app_tests to github actions


______

For contributor use:

- [ ] Wrote tests
- [ ] Updated CHANGELOG_PENDING.md
- [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] Updated relevant documentation (`docs/`) and code comments
- [ ] Re-reviewed `Files changed` in the Github PR explorer

* p2p/pex: fix DATA RACE

in TestPEXReactorDialsPeerUpToMaxAttemptsInSeedMode

Closes #4668

______

For contributor use:

- [x] Wrote tests
- [ ] ~~Updated CHANGELOG_PENDING.md~~
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] ~~Updated relevant documentation (`docs/`) and code comments~~
- [x] Re-reviewed `Files changed` in the Github PR explorer

* lite2/rpc: verify block results and validators (#4703)

Closes: #4695

Verify /block_results and /validators responses from an HTTP client using the light client.

Added count and total to /validators response.

Refs #3113

* rc1/v0.33.4

Prepare release 0.33.4

* kick CI

* abci/server: print panic & stack trace to STDERR if logger is not set

Closes #4382

* test: fix p2p test build breakage caused by Debian testing (#4880)

* types: implement Header#ValidateBasic (#4638)

- Move core stateless validation of the Header type to a ValidateBasic method.
- Call header.ValidateBasic during a SignedHeader validation.
- Call header.ValidateBasic during a PhantomValidatorEvidence validation.
- Call header.ValidateBasic during a LunaticValidatorEvidence validation.

lite tests are skipped since the package is deprecated, no need to waste time on it

closes: #4572

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* lite2: fix TestVerifyAdjacentHeaders (#4884)

* Makefile: parse TENDERMINT_BUILD_OPTIONS (#4738) (#4883)

Reduce the number of targets and make the buildsystem more
flexible by parsing the TENDERMINT_BUILD_OPTIONS command
line variable (a-la Debian, inspired by dpkg-buildpackage's
DEB_BUILD_OPTIONS), e.g:

 $ make install TENDERMINT_BUILD_OPTIONS='cleveldb'

replaces the old:

 $ make install_c

Options can be mix&match'd, e.g.:

 $ make install TENDERMINT_BUILD_OPTIONS='cleveldb race nostrip'

Three options are available:
- nostrip: don't strip debugging symbols nor DWARF tables.
- cleveldb: use cleveldb as db backend instead of goleveldb;
  it switches on the CGO_ENABLED Go environment variale.
- race: pass -race to go build and enable data race detection.

This changeset is a port of gaia pull request: cosmos/gaia#363.

Co-authored-by: Alessio Treglia <alessio@tendermint.com>

* lite: fix HTTP provider error handling (#4882)

* lite: fix HTTP provider error handling

Fixes #4739, kind of. See #4740 for the proper fix.

---

For contributor use:

- [x] Wrote tests
- [x] Updated CHANGELOG_PENDING.md
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [x] Updated relevant documentation (`docs/`) and code comments
- [x] Re-reviewed `Files changed` in the Github PR explorer
- [x] Applied Appropriate Labels

* adapt tests to missing pull request

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* privval: remove deprecated `OldFilePV` (#4889)

* privval: remove deprecated `OldFilePV`

The old format was deprecated in v0.28. It's time we remove it.

* fix changelog

* blockchain/v2: backport fixes (#4887)

* blockchain/v2: fix excessive CPU usage due to spinning on closed channels (#4761)

The event loop uses a `select` on multiple channels. However, reading from a closed channel in Go always yields the channel's zero value. The processor and scheduler close their channels when done, and since these channels are always ready to receive, the event loop keeps spinning on them.

This changes `routine.terminate()` to not close the channel, and also removes `stopDemux` and instead uses `events` channel closure to signal event loop termination.

Fixes #4687.

* blockchain/v2: respect fast_sync option (#4772)

Not thoroughly tested, but seems to work. Will do further testing as this is integrated with state sync.

Fixes #4688.

* types: return an error if voting power overflows

in VerifyCommitTrusting

Closes #4755

* rpc: handle panics during panic handling (#4888)

Fixes #4802. The Go HTTP server has a global panic handler for requests, so it was not as severe as first thought.

This fix can still panic, since we try to send a `500` response - if that happens, the Go HTTP server will terminate the connection. Otherwise, the client will get a 200 response, which we should avoid. I'm sort of torn on whether it's even necessary to include this fix, instead of just letting the HTTP server deal with it.

* mempool: allow ReapX and CheckTx functions to run in parallel

allow ReapX and CheckTx functions to run in parallel, making it not possible to block certain proposers from creating a new block.

Closes: #2972

* pex: use highwayhash for pex bucket

Closes #2998

Spec PR: tendermint/spec#94

* privval: retry GetPubKey/SignVote/SignProposal N times before

returning an error

Closes #4707

* rpc: use a struct to wrap all the global objects

Closes #3433

* rpc: refactor lib folder (#4836)

Closes tendermint/tendermint#3857

Moves `lib/` folder to `jsonrpc/`.

Renames:

**packages**

`rpc` package -> `jsonrpc` package
`rpcclient` package -> `client` package
`rpcserver` package -> `server` package

**structs and interfaces**

```
JSONRPCClient to Client
JSONRPCRequestBatch to RequestBatch
JSONRPCCaller to Caller
```

**functions**

```
StartHTTPServer to Serve
StartHTTPAndTLSServer to ServeTLS

rpc/jsonrpc/client: rename NewURIClient to NewURI

NewJSONRPCClient to New
NewJSONRPCClientWithHTTPClient to NewWithHTTPClient
NewWSClient to NewWS
```

**misc**

- unexpose `ResponseWriterWrapper`
- remove unused http_params.go

* rpc/core: do not lock ConsensusState mutex

in /validators, /consensus_params and /status

Closes #3161

* p2p: return masked IP (not the actual IP) in addrbook#groupKey

Closes #4846 
Spec tendermint/spec#96

* lite2: fix pivot height during bisection

fix bug with assigning the new pivot height during bisection. PR: #4850

* proto: add proto files for ibc unblock (#4853) (#4906)

these proto files are meant to help unblock ibc in their quest of migrating the ibc module to proto.

* prepare v0.33.5 RC2

* types: verify commit fully

Since the light client work introduced in v0.33 it appears full nodes
are no longer fully verifying commit signatures during block execution -
they stop after +2/3. See in VerifyCommit:
https://github.com/tendermint/tendermint/blob/0c7fd316eb006c0afc13996c00ac8bde1078b32c/types/validator_set.go#L700-L703

This means proposers can propose blocks that contain valid +2/3
signatures and then the rest of the signatures can be whatever they
want. They can claim that all the other validators signed just by
including a CommitSig with arbitrary signature data. While this doesn't
seem to impact safety of Tendermint per se, it means that Commits may
contain a lot of invalid data. This is already true of blocks, since
they can include invalid txs filled with garbage, but in that case the
application knows they they are invalid and can punish the proposer. But
since applications dont verify commit signatures directly (they trust
tendermint to do that), they won't be able to detect it.

This can impact incentivization logic in the application that depends on
the LastCommitInfo sent in BeginBlock, which includes which validators
signed. For instance, Gaia incentivizes proposers with a bonus for
including more than +2/3 of the signatures. But a proposer can now claim
that bonus just by including arbitrary data for the final -1/3 of
validators without actually waiting for their signatures. There may be
other tricks that can be played because of this.

In general, the full node should be a fully verifying machine. While
it's true that the light client can avoid verifying all signatures by
stopping after +2/3, the full node can not. Thus the light client and
full node should use distinct VerifyCommit functions if one is going to
stop after +2/3 or otherwise perform less validation (for instance light
clients can also skip verifying votes for nil while full nodes can not).

See a commit with a bad signature that verifies here: 56367fd. From what
I can tell, Tendermint will go on to think this commit is valid and
forward this data to the app, so the app will think the second validator
actually signed when it clearly did not.

* consensus: Do not allow signatures for a wrong block in commits

Closes #4926

The dump consensus state had this:

      "last_commit": {
        "votes": [
          "Vote{0:04CBBF43CA3E 385085/00/2(Precommit) 1B73DA9FC4C8 42C97B86D89D @ 2020-05-27T06:46:51.042392895Z}",
          "Vote{1:055799E028FA 385085/00/2(Precommit) 652B08AD61EA 0D507D7FA3AB @ 2020-06-28T04:57:29.20793209Z}",
          "Vote{2:056024CFA910 385085/00/2(Precommit) 652B08AD61EA C8E95532A4C3 @ 2020-06-28T04:57:29.452696998Z}",
          "Vote{3:0741C95814DA 385085/00/2(Precommit) 652B08AD61EA 36D567615F7C @ 2020-06-28T04:57:29.279788593Z}",

Note there's a precommit in there from the first val from May (2020-05-27) while the rest are from today (2020-06-28). It suggests there's a validator from an old instance of the network at this height (they're using the same chain-id!). Obviously a single bad validator shouldn't be an issue. But the Commit refactor work introduced a bug.

When we propose a block, we get the block.LastCommit by calling MakeCommit on the set of precommits we saw for the last height. This set may include precommits for a different block, and hence the block.LastCommit we propose may include precommits that aren't actually for the last block (but of course +2/3 will be). Before v0.33, we just skipped over these precommits during verification. But in v0.33, we expect all signatures for a blockID to be for the same block ID! Thus we end up proposing a block that we can't verify.

* update changelog and bump version

* changelog: tweak 0.33.6 entry

* privval: if remote signer errors, don't retry (#5140)

Closes #5112

* prepare 0.33.7 release (#5202)

* changelog: rc0/33.8 (#5222)

* only retrieve pubkey once for all validators (partially fixes #4865) (#4895)

in consensus/state.go, when calulating metrics, retrieve address (ergo, pubkey) once prior to iterating over validatorset to ensure we do not make excessive calls to signer.

Partially closes: #4865

* consensus: only call privValidator.GetPubKey once per block (#5143)

Closes #4865

* privval: increase read/write timeout to 5s and calculate ping interval based on it (#5638)

Partially closes #5550

* bump version and update changelog

* use fmt.Sprint for string to int conversion

* chores: remove proto CI file

* chores: upgrade CI tasks

* chores: bump go version in CI tests

* chores: restore lint version

* fix: flaky test reporting invalid header time (#250)

* fix #241 : flaky test reporting invalid header time
Backported from v0.34.10 fix at tendermint/tendermint@4b99502

Co-authored-by: Callum Michael Waters <cmwaters19@gmail.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Tess Rinearson <tess.rinearson@gmail.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Co-authored-by: Denis Fadeev <denis@fadeev.org>
Co-authored-by: Greg Szabo <16846635+greg-szabo@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongwoo Lee <whylee259@gmail.com>
Co-authored-by: Erik Grinaker <erik@interchain.berlin>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Joe Bowman <joe@chorus.one>
@daeMOn63
Copy link
Contributor

Fixed in v0.16.2

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants