Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply Tendermint 0.33.1 and 0.33.2 #49

Merged
merged 129 commits into from
Apr 2, 2020
Merged

Apply Tendermint 0.33.1 and 0.33.2 #49

merged 129 commits into from
Apr 2, 2020

Conversation

zemyblue
Copy link
Member

@zemyblue zemyblue commented Mar 31, 2020

Apply Tendermint 0.33.1 and 0.33.2

Main Change is follows

0.33.1

  • [rpc] #3333 Add order_by to /tx_search endpoint, allowing to change default ordering from asc to desc
  • [proto] #4369 Add make proto-gen and protobuf linting

0.33.2

  • Default used golang version is changed to 1.13.
  • [lite2] #4535 Remove AutoClient on Start/Stop
  • [types] #4417 VerifyCommitX() functions should return as soon as +2/3 threashold is reached.

Tendermint's detail changelogs


  • Referenced an issue explaining the need for the change
  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Updated CHANGELOG_PENDING.md

tac0turtle and others added 30 commits January 17, 2020 11:31
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.6.1 to 1.6.2.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](spf13/viper@v1.6.1...v1.6.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
* rpc: check nil blockmeta

- fixes #4319

- check if block meta is nil

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry
* rpc: update urls to rpc website

* rpc: check blockMeta is not nil in Commit

Refs #4319
* Separate ADR Tendermint Mode from ADR-051

* Update docs/architecture/adr-052-tendermint-mode.md

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Marko <marbar3778@yahoo.com>

* Apply suggestions from code review

Co-Authored-By: Marko <marbar3778@yahoo.com>

* remove line of mode info of rpc

* Add link to ADR table of contents

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Co-authored-by: Marko <marbar3778@yahoo.com>


    Fullnode mode : fullnode mode does not have any capability to participate on consensus
    Validator mode : this mode is exactly same as existing state machine behavior. sync without voting on consensus, and participate consensus when fully synced
    Seed mode : lightweight seed mode only for maintain an address book, p2p like TenderSeed

Separate ADR Tendermint Mode from ADR-051 #4262
* lite2: move AutoClient into Client

Most of the users will want auto update feature, so it makes sense to
move it into the Client itself, rather than having a separate
abstraction (it makes the code cleaner, but introduces an extra thing
the user will need to learn).

Also, add `FirstTrustedHeight` func to Client to get first trusted height.

* fix db store tests

* separate examples for auto and manual clients

* AutoUpdate tries to update to latest state

NOT 1 header at a time

* fix errors

* lite2: make Logger an option

remove SetLogger func

* fix lite cmd

* lite2: make concurrency assumptions explicit

* fixes after my own review

* no need for nextHeightFn

sequence func will download intermediate headers

* correct comment
* lite2: advance to latest header

without any exponential steps

rename autoUpdate to autoUpdateRoutine

* lite2: wait in Cleanup until goroutines finished running
* docs: minor doc fixes

- minor doc fixes that i ran into while reading things
- test if we have github actions

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* no github actions yet

* add with

* revert and change wording
* Add adr-051 to docs

* add details

* Update docs/architecture/adr-051-double-signing-protection-with-tendermint-mode.md

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* rename adr-051 for only double singing protection

* remove contents about tendermint mode

* change title to Double Signing Rist Reduction

* rename adr md file

* add a adr link to ToC

Co-authored-by: b-harvest <38277329+dlguddus@users.noreply.github.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
* lite2: add Start method

There are few reasons to do that:

1) separation of state and dynamics (some users will want to delay
   starting the light client; does not matter we should not allow them
   to create a light client object)
2) less important, but some users might not need autoUpdateRoutine and
   removeNoLongerTrustedHeadersRoutine routines

* lite2: wait till routines are finished in Stop

because they are started in Start, it feels more natural to wait for
them to finish in Stop.

* lite2: add TrustedValidatorSet func
I have added order_by which can be "asc" or "desc" (should be in string format) in the tx_search RPC method.

Fixes: #3333

Author: @princesinha19
* docs: update links to rpc

- links to rpc have not been updated. thank you @okwme

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* Update docs/app-dev/indexing-transactions.md
* lite2: add Start method

There are few reasons to do that:

1) separation of state and dynamics (some users will want to delay
   starting the light client; does not matter we should not allow them
   to create a light client object)
2) less important, but some users might not need autoUpdateRoutine and
   removeNoLongerTrustedHeadersRoutine routines

* lite2: wait till routines are finished in Stop

because they are started in Start, it feels more natural to wait for
them to finish in Stop.

* lite2: add TrustedValidatorSet func

* refactor cleanup code

* changed restore header and val function to handle negative height

* reverted restoreTrustedHeaderAndNextVals() functionality

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
not MaxOpenConnections

Fixes #4311

Also, set MaxBodyBytes, MaxHeaderBytes and WriteTimeout similar to HTTP
server.
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.26.0 to 1.27.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.26.0...v1.27.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Closes #4350

Checks that the chain ID of the witness and that of the lite client are the same before updating the witness list.
Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.3.2 to 1.3.3.
- [Release notes](https://github.com/golang/protobuf/releases)
- [Commits](golang/protobuf@v1.3.2...v1.3.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
* lite2: make witnesses mandatory

at least one witness is required

* lite2: return an error if there are no witnesses

tendermint/tendermint#4358 (review)

* cmd/lite: add witnesses flag

* fix linter errors
* update guides with correct path to libs/kv proto files

* Apply suggestions from code review

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* format something to rerun ci

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
…354)

Closes issue #4338

Uses a wrapper function around both the signedHeader and validatorSet calls to the primary provider which attempts to retrieve the information 5 times before deeming the provider unavailable and replacing the primary provider with the first alternative before trying recursively again (until all alternatives are depleted)

Employs a mutex lock for any operations involving the providers of the light client to ensure no operations occurs whilst the new primary is chosen.

Commits:

* created swapProvider function

* eliminates old primary provider after replacement. Uses a mutex when changing providers

* renamed to replaceProvider

* created wrapped functions for signed header and val set

* created test for primary provider replacement

* implemented suggested revisions

* created Witnesses() and Primary()

* modified backoffAndJitterTime

* modified backoffAndJitterTime

* changed backoff base and jitter to functional arguments

* implemented suggested changes

* removed backoff function

* changed exp function to match go version

* halved the backoff time

* removed seeding and added comments

* fixed incorrect test

* extract backoff timeout calc into a function

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Closes #4328

When TrustedHeader(height) is called, if the height is less than the trusted height but the header is not in the trusted store then a function finds the previous lowest height with a trusted header and performs a forwards sequential verification to the header of the height that was given. If no error is found it updates the trusted store with the header and validator set for that height and can then return them to the user.

Commits:

* drafted trusted header

* created function to find previous trusted height

* updates missing headers less than the trusted height

* minor cosmetic tweaks

* incorporated suggestions

* lite2: implement Backwards verification

and add SignedHeaderAfter func to Store interface

Refs tendermint/tendermint#4328 (comment)

* remove unused method

* write tests

* start with next height in SignedHeaderAfter func

* fix linter errors

* address Callum's comments

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
As opposed to checking a random witness, all witnesses provided should be used as a reference against the header provided by the primary node. This increases security (at the tradeoff of speed) but also gives control to the user. The more witnesses provided, the more secure the lite client can be.
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.27.0 to 1.27.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.27.0...v1.27.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
mergify bot and others added 12 commits March 5, 2020 10:48
* docs: fix links

- fix broken links

closes #4522

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix more links

* add enable and disable

* remvoe deadlink
* readme: add discord to readme

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add our id
We first introduced auto-update as a separate struct AutoClient, which
was wrapping Client and calling Update periodically.

// AutoClient can auto update itself by fetching headers every N seconds.
type AutoClient struct {
    base         *Client
    updatePeriod time.Duration
    quit         chan struct{}

    trustedHeaders chan *types.SignedHeader
    errs           chan error
}

// NewAutoClient creates a new client and starts a polling goroutine.
func NewAutoClient(base *Client, updatePeriod time.Duration) *AutoClient {
    c := &AutoClient{
        base:           base,
        updatePeriod:   updatePeriod,
        quit:           make(chan struct{}),
        trustedHeaders: make(chan *types.SignedHeader),
        errs:           make(chan error),
    }
    go c.autoUpdate()
    return c
}

// TrustedHeaders returns a channel onto which new trusted headers are posted.
func (c *AutoClient) TrustedHeaders() <-chan *types.SignedHeader {
    return c.trustedHeaders
}

// Err returns a channel onto which errors are posted.
func (c *AutoClient) Errs() <-chan error {
    return c.errs
}

// Stop stops the client.
func (c *AutoClient) Stop() {
    close(c.quit)
}

func (c *AutoClient) autoUpdate() {
    ticker := time.NewTicker(c.updatePeriod)
    defer ticker.Stop()

    for {
        select {
        case <-ticker.C:
            lastTrustedHeight, err := c.base.LastTrustedHeight()
            if err != nil {
                c.errs <- err
                continue
            }

            if lastTrustedHeight == -1 {
                // no headers yet => wait
                continue
            }

            newTrustedHeader, err := c.base.Update(time.Now())
            if err != nil {
                c.errs <- err
                continue
            }

            if newTrustedHeader != nil {
                 c.trustedHeaders <- newTrustedHeader
            }
        case <-c.quit:
            return
        }
    }
}

Later we merged it into the Client itself with the assumption that most clients will want it.

But now I am not sure. Neither IBC nor cosmos/relayer are using it. It increases complexity (Start/Stop methods).

That said, I think it makes sense to remove it until we see a need for it (until we better understand usage behavior). We can always introduce it later 😅. Maybe in the form of AutoClient.
error itself is not enough since it only signals if there were any
errors. Either (types.SignedHeader) or (success bool) is needed to
indicate the status of the operation. Returning a header is optimal
since most of the clients will want to get a newly verified header
anyway.
Closes: #4420 

Created a new error ErrInvalidHeaderwhich can be formed during the verification process verifier.go and will result in the replacement of the primary provider with a witness by executing: replacePrimaryProvider()

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
* adr: crypto encoding for proto work

- this adr is meant to help with deciding on how to move forward with keys in tendermint.

* minor change

* fix gomod

* add a third option

* fix spelling

* add first part of descision

* breakdown keys and where they are used

* add some wording

* minor wording fix

* question

* change proto messages

* minor update

* undo go.mod changes

* add a few things based on comemnts

* push, push it real good

* minor explanation on interface type

* touch up
Bumps [github.com/Workiva/go-datastructures](https://github.com/Workiva/go-datastructures) from 1.0.51 to 1.0.52.
- [Release notes](https://github.com/Workiva/go-datastructures/releases)
- [Commits](Workiva/go-datastructures@v1.0.51...v1.0.52)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
* rc1/v0.33.2

- release candiadte for 0.33.2

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
@zemyblue zemyblue self-assigned this Mar 31, 2020
@zemyblue zemyblue added the C: enhancement Classification: New feature or its request, or improvement in maintainability of code label Mar 31, 2020
* Fix conflicted part of `GenerateVRFProof` on `MockPV`
@zemyblue zemyblue added this to In progress in Change the method of electing proposer using VRF via automation Apr 1, 2020
@zemyblue zemyblue changed the title [WIP] Apply Tendermint 0.33.1 and 0.33.2 Apply Tendermint 0.33.1 and 0.33.2 Apr 1, 2020
@zemyblue zemyblue requested review from egonspace and torao April 1, 2020 05:57
@@ -47,35 +47,35 @@ func (pvs PrivValidatorsByAddress) Swap(i, j int) {
// MockPV implements PrivValidator without any safety or persistence.
// Only use it for testing.
type MockPV struct {
privKey crypto.PrivKey
PrivKey crypto.PrivKey
Copy link
Contributor

@egonspace egonspace Apr 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot find the usage this field out of this package. I don't know why is this field accessor changed to public. How about to leave it as private like prior?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked the reason to the original commiter (tendermint/tendermint#4461)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this code is changed by Tendermint developer.
I'll apply the changes if there's any change in the original Tendermint.

lite2/provider/mock/mock.go Show resolved Hide resolved
lite2/provider/mock/mock.go Show resolved Hide resolved
p2p/conn/secret_connection.go Show resolved Hide resolved
Copy link
Contributor

@egonspace egonspace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zemyblue
Copy link
Member Author

zemyblue commented Apr 2, 2020

This will merge to master.
And I'll also merge to develop branch manually.
Thank you.

@zemyblue zemyblue merged commit b8a83c0 into Finschia:master Apr 2, 2020
Change the method of electing proposer using VRF automation moved this from In progress to Done Apr 2, 2020
@zemyblue zemyblue deleted the feature/apply_0.33.1_0.33.2 branch April 2, 2020 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: enhancement Classification: New feature or its request, or improvement in maintainability of code
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet