R4R: improve ResetTestRootWithChainID() concurrency safety #3291
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## develop #3291 +/- ##
===========================================
+ Coverage 63.66% 63.84% +0.17%
===========================================
Files 215 215
Lines 17733 17722 -11
===========================================
+ Hits 11290 11314 +24
+ Misses 5504 5478 -26
+ Partials 939 930 -9
|
alessio
changed the title
liamsi
reviewed
Feb 10, 2019
liamsi
reviewed
Feb 10, 2019
melekes
reviewed
Feb 10, 2019
melekes
reviewed
Feb 11, 2019
melekes
reviewed
Feb 13, 2019
There was a problem hiding this comment.
Great work 👍 I have one comment about the inconsistency between the cleanup function and returning configDirs. I think it's better to stick with just one. Either everything returns cleanup function or config directories. Cleanup function seems like a more cleaner approach to me.
Rely on ioutil.TempDir() to create test root directories and ensure multiple same-chain id test cases can run in parallel.
Co-Authored-By: alessio <quadrispro@ubuntu.com>
In modern UNIX and UNIX-like systems /tmp is very often mounted as tmpfs. This might speed test execution a bit.
|
Please do not rebase the branch next time 🙇 It makes impossible for Github to track changes (i.e. there's no "Changes since you last viewed" button anymore) ☝️ , so we're forced to review the entire PR again. Thank you! |
melekes
reviewed
Feb 16, 2019
| @@ -100,8 +100,8 @@ func GetGRPCClient() core_grpc.BroadcastAPIClient { | |||
| } | |||
|
|
|||
| // StartTendermint starts a test tendermint server in a go routine and returns when it is initialized | |||
| func StartTendermint(app abci.Application) *nm.Node { | |||
| node := NewTendermint(app) | |||
| func StartTendermint(app abci.Application) (*nm.Node, func()) { | |||
There was a problem hiding this comment.
Can we create StopTendermint() function which removes a config directory?
// node/node.go
func (n *Node) Config() *cfg.Config {
return n.config
}
// rpc/test/helpers.go
func StopTendermint() {
node.Stop()
node.Wait()
os.RemoveAll(node.Config().RootDir)
}There was a problem hiding this comment.
If you don't mind, I will do it in an ensuing PR
Co-Authored-By: alessio <quadrispro@ubuntu.com>
Co-Authored-By: alessio <quadrispro@ubuntu.com>
Co-Authored-By: alessio <quadrispro@ubuntu.com>
melekes
approved these changes
Feb 18, 2019
melekes
reviewed
Feb 18, 2019
| @@ -142,6 +153,7 @@ LOOP: | |||
|
|
|||
| // clean up WAL file from the previous iteration | |||
| walFile := cs.config.WalFile() | |||
| ensureDir(filepath.Dir(walFile), 0700) | |||
melekes
added a commit
that referenced
this pull request
Feb 18, 2019
- rpc/test/helpers.go add StopTendermint(node) func - remove ensureDir(filepath.Dir(walFile), 0700) - mempool/mempool_test.go add type cleanupFunc func()
melekes
added a commit
that referenced
this pull request
Feb 18, 2019
* changelog: use issue number instead of PR number * follow up to #3291 - rpc/test/helpers.go add StopTendermint(node) func - remove ensureDir(filepath.Dir(walFile), 0700) - mempool/mempool_test.go add type cleanupFunc func() * cmd/show_validator: wrap err to make it more clear
thanethomson
added a commit
to thanethomson/tendermint
that referenced
this pull request
Feb 19, 2019
melekes
pushed a commit
that referenced
this pull request
Feb 20, 2019
As per #3043, this adds a ticker to sync the WAL every 2s while the WAL is running. * Flush WAL every 2s This adds a ticker that flushes the WAL every 2s while the WAL is running. This is related to #3043. * Fix spelling * Increase timeout to 2mins for slower build environments * Make WAL sync interval configurable * Add TODO to replace testChan with more comprehensive testBus * Remove extraneous debug statement * Remove testChan in favour of using system time As per #3300 (comment), this removes the `testChan` WAL member and replaces the approach with a system time-oriented one. In this new approach, we keep track of the system time at which each flush and periodic flush successfully occurred. The naming of the various functions is also updated here to be more consistent with "flushing" as opposed to "sync'ing". * Update naming convention and ensure lock for timestamp update * Add Flush method as part of WAL interface Adds a `Flush` method as part of the WAL interface to enforce the idea that we can manually trigger a WAL flush from outside of the WAL. This is employed in the consensus state management to flush the WAL prior to signing votes/proposals, as per #3043 (comment) * Update CHANGELOG_PENDING * Remove mutex approach and replace with DI The dependency injection approach to dealing with testing concerns could allow similar effects to some kind of "testing bus"-based approach. This commit introduces an example of this, where instead of relying on (potentially fragile) timing of things between the code and the test, we inject code into the function under test that can signal the test through a channel. This allows us to avoid the `time.Sleep()`-based approach previously employed. * Update comment on WAL flushing during vote signing Co-Authored-By: thanethomson <connect@thanethomson.com> * Simplify flush interval definition Co-Authored-By: thanethomson <connect@thanethomson.com> * Expand commentary on WAL disk flushing Co-Authored-By: thanethomson <connect@thanethomson.com> * Add broken test to illustrate WAL sync test problem Removes test-related state (dependency injection code) from the WAL data structure and adds test code to illustrate the problem with using `WALGenerateNBlocks` and `wal.SearchForEndHeight` to test periodic sync'ing. * Fix test error messages * Use WAL group buffer size to check for flush A function is added to `libs/autofile/group.go#Group` in order to return the size of the buffered data (i.e. data that has not yet been flushed to disk). The test now checks that, prior to a `time.Sleep`, the group buffer has data in it. After the `time.Sleep` (during which time the periodic flush should have been called), the buffer should be empty. * Remove config root dir removal from #3291 * Add godoc for NewWAL mentioning periodic sync
pinkiebell
added a commit
to leapdao/tendermint
that referenced
this pull request
Mar 28, 2019
* docs: update ecosystem.json: add Rust ABCI (#2945)
* types: ValidatorSet.Update preserves Accum (#2941)
* types: ValidatorSet.Update preserves ProposerPriority
This solves the other issue discovered as part of #2718,
where Accum (now called ProposerPriority) is reset to
0 every time a validator is updated.
* update changelog
* add test
* update comment
* Update types/validator_set_test.go
Co-Authored-By: ebuchman <ethan@coinculture.info>
* Add some ProposerPriority tests (#2946)
* WIP: tests for #2785
* rebase onto develop
* add Bucky's test without changing ValidatorSet.Update
* make TestValidatorSetBasic fail
* add ProposerPriority preserving fix to ValidatorSet.Update to fix
TestValidatorSetBasic
* fix randValidator_ to stay in bounds of MaxTotalVotingPower
* check for expected proposer and remove some duplicate code
* actually limit the voting power of random validator ...
* fix test
* Bucky/v0.27.0 (#2950)
* update changelog
* changelog, upgrading, version
* explicitly type MaxTotalVotingPower to int64 (#2953)
* check if deliverTxResCh is still open, return an err otherwise (#2947)
deliverTxResCh, like any other eventBus (pubsub) channel, is closed when
eventBus is stopped. We must check if the channel is still open. The
alternative approach is to not close any channels, which seems a bit
odd.
Fixes #2408
* docs: add client#Start/Stop to examples in RPC docs (#2939)
follow-up on https://github.com/tendermint/tendermint/pull/2936
* Minor log changes (#2959)
* node: allow state and code to have diff block versions
* node: pex is a log module
* p2p: panic on transport error (#2968)
* p2p: panic on transport error
Addresses #2823. Currently, the acceptRoutine exits if the transport returns
an error trying to accept a new connection. Once this happens, the node
can't accept any new connections. So here, we panic instead. While we
could potentially be more intelligent by rerunning the acceptRoutine, the
error may indicate something more fundamental (eg. file desriptor limit)
that requires a restart anyways. We can leave it to process managers to
handle that restart, and notify operators about the panic.
* changelog
* p2p: fix peer count mismatch #2332 (#2969)
* p2p: test case for peer count mismatch #2332
* p2p: fix peer count mismatch #2332
* changelog
* use httptest.Server to scrape Prometheus metrics
* kv indexer: add separator to start key when matching ranges (#2925)
* kv indexer: add separator to start key when matching ranges
to avoid including false positives
Refs #2908
* refactor code
* add a test case
* update changelog and upgrading (#2974)
* don't ignore key when executing CONTAINS (#2924)
Fixes #2912
* return an error if validator set is empty in genesis file and after InitChain (#2971)
Fixes #2951
* docs: relative links in docs/spec/readme.md, js-amino lib (#2977)
Co-Authored-By: zramsay <zach.ramsay@gmail.com>
* turn off strict routability every time (#2983)
previously, we're turning it off only when --populate-persistent-peers
flag was used, which is obviously incorrect.
Fixes https://github.com/cosmos/cosmos-sdk/issues/2983
* Make mempool fail txs with negative gas wanted (#2994)
This is only one part of #2989. We also need to fix the application,
and add rules to consensus to ensure this.
* p2p: set MConnection#created during init (#2990)
Fixes #2715
In crawlPeersRoutine, which is performed when seedMode is run, there is
logic that disconnects the peer's state information at 3-hour intervals
through the duration value. The duration value is calculated by
referring to the created value of MConnection. When MConnection is
created for the first time, the created value is not initiated, so it is
not disconnected every 3 hours but every time it is disconnected. So,
normal nodes are connected to seedNode and disconnected immediately, so
address exchange does not work properly.
https://github.com/tendermint/tendermint/blob/master/p2p/pex/pex_reactor.go#L629
This point is not work correctly.
I think,
https://github.com/tendermint/tendermint/blob/master/p2p/conn/connection.go#L148
created variable is missing the current time setting.
* Make testing logger that doesn't write to stdout (#2997)
* add UnconfirmedTxs/NumUnconfirmedTxs methods to HTTP/Local clients (#2964)
* docs: fixes from 'first time' review (#2999)
* docs: enable full-text search (#3004)
* mempool: add a comment and missing changelog entry (#2996)
Refs #2994
* circleci: add a job to automatically update docs (#3005)
* docs: add edit on Github links (#3014)
* docs: update DOCS_README (#3019)
Co-Authored-By: zramsay <zach.ramsay@gmail.com>
* mempool: notifyTxsAvailable if there're txs left, but recheck=false (#2991)
(left after committing a block)
Fixes #2961
--------------
ORIGINAL ISSUE
Tendermint version : 0.26.4-b771798d
ABCI app : kv-store
Environment:
OS (e.g. from /etc/os-release): macOS 10.14.1 What happened: Set
mempool.recheck = false and create empty block = false in config.toml.
When transactions get added right between a new empty block is being
proposed and committed, the proposer won't propose new block for that
transactions immediately after. That transactions are stuck in the
mempool until a new transaction is added and trigger the proposer.
What you expected to happen: If there is a transaction left in the
mempool, new block should be proposed immediately.
Have you tried the latest version: yes
How to reproduce it (as minimally and precisely as possible): Fire two
transaction using broadcast_tx_sync with specific delay between them.
(You may need to do it multiple time before the right delay is found, on
my machine the delay is 0.98s)
Logs (paste a small part showing an error (< 10 lines) or link a
pastebin, gist, etc. containing more of the log file):
https://pastebin.com/0Wt6uhPF
Config (you can paste only the changes you've made): [mempool] recheck =
false create_empty_block = false
Anything else we need to know: In mempool.go, we found that proposer
will immediately propose new block if
Last committed block has some transaction (causing AppHash to changed)
or mem.notifyTxsAvailable() is called. Our scenario is as followed.
A transaction is fired, it will create 1 block with 1 tx (line 1-4 in
the log) and 1 empty block. After the empty block is proposed but before
it is committed, second transaction is fired and added to mempool. (line
8-16) Now, since the last committed block is empty and
mem.notifyTxsAvailable() will be called only if mempool.recheck = true.
The proposer won't immediately propose new block, causing the second
transaction to stuck in mempool until another transaction is added to
mempool and trigger mem.notifyTxsAvailable().
* During replay, when appHeight==0, only saveState if stateHeight is also 0 (#3006)
* optimize addProposalBlockPart
* optimize addProposalBlockPart
* if ProposalBlockParts and LockedBlockParts both exist,let LockedBlockParts overwrite ProposalBlockParts.
* fix tryAddBlock
* broadcast lockedBlockParts in higher priority
* when appHeight==0, it's better fetch genDoc than state.validators.
* not save state if replay from height 1
* only save state if replay from height 1 when stateHeight is also 1
* only save state if replay from height 1 when stateHeight is also 1
* only save state if replay from height 0 when stateHeight is also 0
* handshake info's response version only update when stateHeight==0
* save the handshake responseInfo appVersion
* 2980 fix cors (#3021)
* config: cors options are arrays of strings, not strings
Fixes #2980
* docs: update tendermint-core/configuration.html page
* set allow_duplicate_ip to false
* in `tendermint testnet`, set allow_duplicate_ip to true
Refs #2712
* fixes after Ismail's review
* Revert "set allow_duplicate_ip to false"
This reverts commit 24c1094ebcf2bd35f2642a44d7a1e5fb5c178fb1.
* #2980 fix cors doc (#3013)
* docs: networks/docker-compose: small fixes (#3017)
* Bucky/v0.27.1 (#3022)
* update changelog
* linkify
* changelog and version
* Revert to using defers in addrbook. (#3025)
* Revert to using defers in addrbook. ValidateBasic->Validate since it requires DNS
* Update CHANGELOG_PENDING
* makeNodeInfo returns error (#3029)
* makeNodeInfo returns error
* version and changelog
* crypto: revert to mainline Go crypto lib (#3027)
* crypto: revert to mainline Go crypto lib
We used to use a fork for a modified bcrypt so we could pass our own
randomness but this was largely unecessary, unused, and a burden.
So now we just use the mainline Go crypto lib.
* changelog
* fix tests
* version and changelog
* fix docs / proxy app (#2988)
* fix docs / proxy app, closes #2986
* counter_serial
* review comments
* list all possible options
* add changelog entries
* mempool: move tx to back, not front (#3036)
because we pop txs from the front if the cache is full
Refs #3035
* update go version & other cleanup (#3018)
* update go version & other cleanup
* fix lints
* go one.eleven.four
* keep circle on 1.11.3 for now
* set allow_duplicate_ip to false (#2992)
* config: cors options are arrays of strings, not strings
Fixes #2980
* docs: update tendermint-core/configuration.html page
* set allow_duplicate_ip to false
* in `tendermint testnet`, set allow_duplicate_ip to true
Refs #2712
* fixes after Ismail's review
* docs: add rpc link to docs navbar and re-org sidebar (#3041)
* add rpc to docs navbar and close #3000
* Update config.js
* circleci: update go version (#3051)
* mempool: move tx to back, not front (#3036)
because we pop txs from the front if the cache is full
Refs #3035
* changelog and version
* R4R: Split immutable and mutable parts of priv_validator.json (#2870)
* split immutable and mutable parts of priv_validator.json
* fix bugs
* minor changes
* retrig test
* delete scripts/wire2amino.go
* fix test
* fixes from review
* privval: remove mtx
* rearrange priv_validator.go
* upgrade path
* write tests for the upgrade
* fix for unsafe_reset_all
* add test
* add reset test
* Update node_info.go (#3059)
* Remove privval.GetAddress(), memoize pubkey (#2948)
privval: remove GetAddress(), memoize pubkey
* 3070 [docs] unindent text as it is supposed to behave the same as the parts before (#3075)
* add signing spec (#3061)
* add signing spec
* fixes from review
* more fixes
* fixes from review
* fix build scripts (#3085)
* fix build scripts
Search for the right variable when introspecting Go code. `Version` was
renamed to `TMCoreSemVer`.
This is regression introduced in
b95ac688af14d130e6ad0b580ed9a8181f6c487c
* fix all `Version` introspections.
Use `TMCoreSemVer` instead of `Version`
* cs: prettify logging of ignored votes (#3086)
Refs #3038
* Don't use pointer receivers for PubKeyMultisigThreshold (#3100)
* Don't use pointer receivers for PubKeyMultisigThreshold
* test that showcases panic when PubKeyMultisigThreshold are used in sdk:
- deserialization will fail in `readInfo` which tries to read a
`crypto.PubKey` into a `localInfo` (called by
cosmos-sdk/client/keys.GetKeyInfo)
* Update changelog
* Rename routeTable to nameTable, multisig key is no longer a pointer
* sed -i 's/PubKeyAminoRoute/PubKeyAminoName/g' `grep -lrw PubKeyAminoRoute .`
upon Jae's request
* AminoRoutes -> AminoNames
* sed -e 's/PrivKeyAminoRoute/PrivKeyAminoName/g'
* Update crypto/encoding/amino/amino.go
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* Ensure multisig keys have 20-byte address (#3103)
* Ensure multisig keys have 20-byte address
Use crypto.AddressHash() to avoid returning 32-byte long address.
Closes: #3102
* fix pointer
* fix test
* update README (#3097)
* update README
* fix from review
* rpc: include peer's remote IP in `/net_info` (#3052)
Refs #3047
* docs: fix p2p readme links (#3109)
* Make SecretConnection thread safe (#3111)
* p2p/conn: add failing tests
* p2p/conn: make SecretConnection thread safe
* changelog
* fix from review
* Close and retry a RemoteSigner on err (#2923)
* Close and recreate a RemoteSigner on err
* Update changelog
* Address Anton's comments / suggestions:
- update changelog
- restart TCPVal
- shut down on `ErrUnexpectedResponse`
* re-init remote signer client with fresh connection if Ping fails
- add/update TODOs in secret connection
- rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose
* account for `conn returned by waitConnection can be `nil`
- also add TODO about RemoteSigner conn field
* Tests for retrying: IPC / TCP
- shorter info log on success
- set conn and use it in tests to close conn
* Tests for retrying: IPC / TCP
- shorter info log on success
- set conn and use it in tests to close conn
- add rwmutex for conn field in IPC
* comments and doc.go
* fix ipc tests. fixes #2677
* use constants for tests
* cleanup some error statements
* fixes #2784, race in tests
* remove print statement
* minor fixes from review
* update comment on sts spec
* cosmetics
* p2p/conn: add failing tests
* p2p/conn: make SecretConnection thread safe
* changelog
* IPCVal signer refactor
- use a .reset() method
- don't use embedded RemoteSignerClient
- guard RemoteSignerClient with mutex
- drop the .conn
- expose Close() on RemoteSignerClient
* apply IPCVal refactor to TCPVal
* remove mtx from RemoteSignerClient
* consolidate IPCVal and TCPVal, fixes #3104
- done in tcp_client.go
- now called SocketVal
- takes a listener in the constructor
- make tcpListener and unixListener contain all the differences
* delete ipc files
* introduce unix and tcp dialer for RemoteSigner
* rename files
- drop tcp_ prefix
- rename priv_validator.go to file.go
* bring back listener options
* fix node
* fix priv_val_server
* fix node test
* minor cleanup and comments
* Hotfix/validating query result length (#3053)
* Validating that there are txs in the query results before loop throught the array
* Created tests to validate the error has been fixed
* Added comments
* Fixing misspeling
* check if the variable "skipCount" is bigger than zero. If it is not, we set it to 0. If it, we do not do anything.
* using function that validates the skipCount variable
* undo Gopkg.lock changes
* [WIP] Fill in consensus core details in ADR 030 (#2696)
* Initial work towards making ConsensusCore spec complete
* Initial version of executor and complete consensus
* Bucky/v0.28.0 (#3119)
* changelog pending and upgrading
* linkify and version bump
* changelog shuffle
* More ProposerPriority tests (#2966)
* more proposer priority tests
- test that we don't reset to zero when updating / adding
- test that same power validators alternate
* add another test to track / simulate similar behaviour as in #2960
* address some of Chris' review comments
* address some more of Chris' review comments
* fix order of BlockID and Timestamp in Vote and Proposal (#3078)
* Consistent order fields of Timestamp/BlockID fields in CanonicalVote and
CanonicalProposal
* update spec too
* Introduce and use IsZero & IsComplete:
- update IsZero method according to spec and introduce IsComplete
- use methods in validate basic to validate: proposals come with a
"complete" blockId and votes are either complete or empty
- update spec: BlockID.IsNil() -> BlockID.IsZero() and fix typo
* BlockID comes first
* fix tests
* Simple merkle rfc compatibility (#2713)
* Begin simple merkle compatibility PR
* Fix query_test
* Use trillian test vectors
* Change the split point per RFC 6962
* update spec
* refactor innerhash to match spec
* Update changelog
* Address @liamsi's comments
* Write the comment requested by @liamsi
* Adds tests for Unix sockets
As per #3115, adds simple Unix socket connect/accept deadline tests in
pretty much the same way as the TCP connect/accept deadline tests work.
* update ADR-020 (#3116)
* [log] fix year format (#3125)
Refs #3060
* makefile: fix build-docker-localnode target (#3122)
cd does not work because it's executed in a subprocess
so it has to be either chained by && or ;
See https://stackoverflow.com/q/1789594/820520
for more details.
Fixes #3058
* return maxPerPage (not defaultPerPage) if per_page is greater than max (#3124)
it's more user-friendly.
Refs #3065
* Make privval listener testing generic
This cuts out two tests by constructing test cases and iterating through
them, rather than having separate sets of tests for TCP and Unix listeners.
This is as per the feedback from #3121.
* privval: fixes from review (#3126)
https://github.com/tendermint/tendermint/pull/2923#pullrequestreview-192065694
* docs: update link for rpc docs (#3129)
* Dropping "construct" prefix as per #3121
* add "chain_id" label for all metrics (#3123)
* add "chain_id" label for all metrics
Refs #3082
* fix labels extraction
* Expanding tests to cover Unix sockets version of client (#3132)
* Adds a random suffix to temporary Unix sockets during testing
* Adds Unix domain socket tests for client (FAILING)
This adds Unix domain socket tests for the privval client. Right now,
one of the tests (TestRemoteSignerRetry) fails, probably because the
Unix domain socket state is known instantaneously on both sides by the
OS. Committing this to collaborate on the error.
* Removes extraneous logging
* Completes testing of Unix sockets client version
This completes the testing of the client connecting via Unix sockets.
There are two specific tests (TestSocketPVDeadline and
TestRemoteSignerRetryTCPOnly) that are only relevant to TCP connections.
* Renames test to show TCP-specificity
* Adds testing into closures for consistency (forgot previously)
* Moves test specific to RemoteSigner into own file
As per discussion on #3132, `TestRemoteSignerRetryTCPOnly` doesn't
really belong with the client tests. This moves it into its own file
related to the `RemoteSigner` class.
* update changelog and upgrading (#3133)
* fix changelog fmt (#3134)
* fixes from review (#3137)
* Add comment to simple_merkle get_split_point (#3136)
* Add comment to simple_merkle get_split_point
* fix grammar error
* docs: fix broken link (#3142)
* Consolidates deadline tests for privval Unix/TCP (#3143)
* Consolidates deadline tests for privval Unix/TCP
Following on from #3115 and #3132, this converts fundamental timeout
errors from the client's `acceptConnection()` method so that these can
be detected by the test for the TCP connection.
Timeout deadlines are now tested for both TCP and Unix domain socket
connections.
There is also no need for the additional secret connection code: the
connection will time out at the `acceptConnection()` phase for TCP
connections, and will time out when attempting to obtain the
`RemoteSigner`'s public key for Unix domain socket connections.
* Removes extraneous logging
* Adds IsConnTimeout helper function
This commit adds a helper function to detect whether an error is either
a fundamental networking timeout error, or an `ErrConnTimeout` error
specific to the `RemoteSigner` class.
* Adds a test for the IsConnTimeout() helper function
* Separates tests logically for IsConnTimeout
* docs: fix RPC links (#3141)
* Bucky/fix evidence halt (#34)
* consensus: createProposalBlock function
* blockExecutor.CreateProposalBlock
- factored out of consensus pkg into a method on blockExec
- new private interfaces for mempool ("txNotifier") and evpool with one function each
- consensus tests still require more mempool methods
* failing test for CreateProposalBlock
* Fix bug in include evidece into block
* evidence: change maxBytes to maxSize
* MaxEvidencePerBlock
- changed to return both the max number and the max bytes
- preparation for #2590
* changelog
* fix linter
* Fix from review
Co-Authored-By: ebuchman <ethan@coinculture.info>
* Add ParadigmCore to ecosystem.json (#3145)
Adding the OrderStream network client (alpha), ParadigmCore, to the `ecosystem.json` file.
* json2wal: increase reader's buffer size (#3147)
```
panic: failed to unmarshal json: unexpected end of JSON input
goroutine 1 [running]:
main.main()
/root/gelgo/src/github.com/tendermint/tendermint/scripts/json2wal/main.go:66 +0x73f
```
Closes #3146
* changelog and version
* update btcd fork for v0.1.1 (#3164)
* update btcd fork for v0.1.1
* changelog
* Normalize priorities to not exceed total voting power (#3049)
* more proposer priority tests
- test that we don't reset to zero when updating / adding
- test that same power validators alternate
* add another test to track / simulate similar behaviour as in #2960
* address some of Chris' review comments
* address some more of Chris' review comments
* temporarily pushing branch with the following changes:
The total power might change if:
- a validator is added
- a validator is removed
- a validator is updated
Decrement the accums (of all validators) directly after any of these events
(by the inverse of the change)
* Fix 2960 by re-normalizing / scaling priorities to be in bounds of total
power, additionally:
- remove heap where it doesn't make sense
- avg. only at the end of IncrementProposerPriority instead of each
iteration
- update (and slightly improve)
TestAveragingInIncrementProposerPriorityWithVotingPower to reflect
above changes
* Fix 2960 by re-normalizing / scaling priorities to be in bounds of total
power, additionally:
- remove heap where it doesn't make sense
- avg. only at the end of IncrementProposerPriority instead of each
iteration
- update (and slightly improve)
TestAveragingInIncrementProposerPriorityWithVotingPower to reflect
above changes
* fix tests
* add comment
* update changelog pending & some minor changes
* comment about division will floor the result & fix typo
* Update TestLargeGenesisValidator:
- remove TODO and increase large genesis validator's voting power
accordingly
* move changelog entry to P2P Protocol
* Ceil instead of flooring when dividing & update test
* quickly fix failing TestProposerPriorityDoesNotGetResetToZero:
- divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower)
* fix typo: rename getValWitMostPriority -> getValWithMostPriority
* test proposer frequencies
* return absolute value for diff. keep testing
* use for loop for div
* cleanup, more tests
* spellcheck
* get rid of using floats: manually ceil where necessary
* Remove float, simplify, fix tests to match chris's proof (#3157)
* [types] hash of ConsensusParams includes only a subset of fields (#3165)
* types: dont hash entire ConsensusParams
* update encoding spec
* update blockchain spec
* spec: consensus params hash
* changelog
* mempool: enforce maxMsgSize limit in CheckTx (#3168)
- fixes #3008
- reactor requires encoded messages are less than maxMsgSize
- requires size of tx + amino-overhead to not exceed maxMsgSize
* fix DynamicVerifier for large validator set changes (#3171)
* base verifier: bc->bv and check chainid
* improve some comments
* comments in dynamic verifier
* fix comment in doc about BaseVerifier
It requires the validator set to perfectly match.
* failing test for #2862
* move errTooMuchChange to types. fixes #2862
* changelog, comments
* ic -> dv
* update comment, link to issue
* update spec for Merkle RFC 6962 (#3175)
* spec: specify when MerkleRoot is on hashes
* remove unnecessary hash methods
* update changelog
* fix test
* Prepare v0.29.0 (#3184)
* update changelog and upgrading
* add note about max voting power in abci spec
* update version
* changelog
* fix changelog indent (#3190)
* p2p: file descriptor leaks (#3150)
* close peer's connection to avoid fd leak
Fixes #2967
* rename peer#Addr to RemoteAddr
* fix test
* fixes after Ethan's review
* bring back the check
* changelog entry
* write a test for switch#acceptRoutine
* increase timeouts? :(
* remove extra assertNPeersWithTimeout
* simplify test
* assert number of peers (just to be safe)
* Cleanup in OnStop
* run tests with verbose flag on CircleCI
* spawn a reading routine to prevent connection from closing
* get port from the listener
random port is faster, but often results in
```
panic: listen tcp 127.0.0.1:44068: bind: address already in use [recovered]
panic: listen tcp 127.0.0.1:44068: bind: address already in use
goroutine 79 [running]:
testing.tRunner.func1(0xc0001bd600)
/usr/local/go/src/testing/testing.go:792 +0x387
panic(0x974d20, 0xc0001b0500)
/usr/local/go/src/runtime/panic.go:513 +0x1b9
github.com/tendermint/tendermint/p2p.MakeSwitch(0xc0000f42a0, 0x0, 0x9fb9cc, 0x9, 0x9fc346, 0xb, 0xb42128, 0x0, 0x0, 0x0, ...)
/home/vagrant/go/src/github.com/tendermint/tendermint/p2p/test_util.go:182 +0xa28
github.com/tendermint/tendermint/p2p.MakeConnectedSwitches(0xc0000f42a0, 0x2, 0xb42128, 0xb41eb8, 0x4f1205, 0xc0001bed80, 0x4f16ed)
/home/vagrant/go/src/github.com/tendermint/tendermint/p2p/test_util.go:75 +0xf9
github.com/tendermint/tendermint/p2p.MakeSwitchPair(0xbb8d20, 0xc0001bd600, 0xb42128, 0x2f7, 0x4f16c0)
/home/vagrant/go/src/github.com/tendermint/tendermint/p2p/switch_test.go:94 +0x4c
github.com/tendermint/tendermint/p2p.TestSwitches(0xc0001bd600)
/home/vagrant/go/src/github.com/tendermint/tendermint/p2p/switch_test.go:117 +0x58
testing.tRunner(0xc0001bd600, 0xb42038)
/usr/local/go/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:878 +0x353
exit status 2
FAIL github.com/tendermint/tendermint/p2p 0.350s
```
* docs: explain how someone can run his/her own ABCI app on localnet (#3195)
Closes #3192.
* docs: update pubsub ADR (#3131)
* docs: update pubsub ADR
* third version
* docs: fix lite client formatting (#3198)
Closes #3180
* only log "Reached max attempts to dial" once (#3144)
Closes #3037
* [WIP] fix halting issue (#3197)
fix halting issue
* update changelog
* bump version
* fix changelog
* R4R: Config TestRoot modification for LCD test (#3177)
* add ResetTestRootWithChainID
* modify chainid
* adr: style fixes (#3206)
- links to issues
- fix a few markdown glitches
- inline code
- etc
* add design philosophy doc (#3034)
* note about TmCoreSemVer
* types: comments on user vs internal events
Distinguish between user events and internal consensus events
* pubsub: comments
* adr-033 update
* start eventBus & indexerService before replay and use them while replaying blocks (#3194)
so if we did not index the last block (because of panic or smth else), we index it during replay
Closes #3186
* refactor TestListenerConnectDeadlines to avoid data races (#3201)
Fixes #3179
* add go-deadlock tool to help detect deadlocks (#3218)
* add go-deadlock tool to help detect deadlocks
Run it with `make test_with_deadlock`. After it's done, use Git to
cleanup `git checkout .`
Link: https://github.com/sasha-s/go-deadlock/
Replaces https://github.com/tendermint/tendermint/pull/3148
* add a target to cleanup changes
* alias amino imports (#3219)
As per conversation here: https://github.com/tendermint/tendermint/pull/3218#discussion_r251364041
This is the result of running the following code on the repo:
```bash
find . -name '*.go' | grep -v 'vendor/' | xargs -n 1 goimports -w
```
* docs: fix links (#3220)
Because there's nothing worse than having to copy/paste a link from a
web page to navigate to it :grin:
* hardcode rpc link (#3223)
* mempool: correct args order in the log msg (#3221)
Before: Unexpected tx response from proxy during recheck\n Expected: {r.CheckTx.Data}, got: {memTx.tx}
After: Unexpected tx response from proxy during recheck\n Expected: {memTx.tx}, got: {tx}
Closes #3214
* secret connection: check for low order points (#3040)
> Implement a check for the blacklisted low order points, ala the X25519 has_small_order() function in libsodium
(#3010 (comment))
resolves first half of #3010
* pubsub: fixes after Ethan's review (#3212)
in https://github.com/tendermint/tendermint/pull/3209
* update gometalinter to 3.0.0 (#3233)
in the attempt to fix https://circleci.com/gh/tendermint/tendermint/43165
also
code is simplified by running gofmt -s .
remove unused vars
enable linters we're currently passing
remove deprecated linters
* Use ethereum's secp256k1 lib (#3234)
* switch from fork (tendermint/btcd) to orig package (btcsuite/btcd); also
- remove obsolete check in test `size != -1` is always true
- WIP as the serialization still needs to be wrapped
* WIP: wrap signature & privkey, pubkey needs to be wrapped as well
* wrap pubkey too
* use "github.com/ethereum/go-ethereum/crypto/secp256k1" if cgo is
available, else use "github.com/btcsuite/btcd/btcec" and take care of
lower-S when verifying
Annoyingly, had to disable pruning when importing
github.com/ethereum/go-ethereum/ :-/
* update comment
* update comment
* emulate signature_nocgo.go for additional benchmarks:
https://github.com/ethereum/go-ethereum/blob/592bf6a59cac9697f0491b24e5093cb759d7e44c/crypto/signature_nocgo.go#L60-L76
* use our format (r || s) in lower-s form when in the non-cgo case
* remove comment about using the C library directly
* vendor github.com/btcsuite/btcd too
* Add test for the !cgo case
* update changelog pending
Closes #3162 #3163
Refs #1958, #2091, tendermint/btcd#1
* fix FlushStop (#3247)
* p2p/pex: failing test
* p2p/conn: add stopMtx for FlushStop and OnStop
* changelog
* WAL: better errors and new fail point (#3246)
* privval: more info in errors
* wal: change Debug logs to Info
* wal: log and return error on corrupted wal instead of panicing
* fail: Exit right away instead of sending interupt
* consensus: FAIL before handling our own vote
allows to replicate #3089:
- run using `FAIL_TEST_INDEX=0`
- delete some bytes from the end of the WAL
- start normally
Results in logs like:
```
I[2019-02-03|18:12:58.225] Searching for height module=consensus wal=/Users/ethanbuchman/.tendermint/data/cs.wal/wal height=1 min=0 max=0
E[2019-02-03|18:12:58.225] Error on catchup replay. Proceeding to start ConsensusState anyway module=consensus err="failed to read data: EOF"
I[2019-02-03|18:12:58.225] Started node module=main nodeInfo="{ProtocolVersion:{P2P:6 Block:9 App:1} ID_:35e87e93f2e31f305b65a5517fd2102331b56002 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-J8JvJH Version:0.29.1 Channels:4020212223303800 Moniker:Ethans-MacBook-Pro.local Other:{TxIndex:on RPCAddress:tcp://0.0.0.0:26657}}"
E[2019-02-03|18:12:58.226] Couldn't connect to any seeds module=p2p
I[2019-02-03|18:12:59.229] Timed out module=consensus dur=998.568ms height=1 round=0 step=RoundStepNewHeight
I[2019-02-03|18:12:59.230] enterNewRound(1/0). Current: 1/0/RoundStepNewHeight module=consensus height=1 round=0
I[2019-02-03|18:12:59.230] enterPropose(1/0). Current: 1/0/RoundStepNewRound module=consensus height=1 round=0
I[2019-02-03|18:12:59.230] enterPropose: Our turn to propose module=consensus height=1 round=0 proposer=AD278B7767B05D7FBEB76207024C650988FA77D5 privValidator="PrivValidator{AD278B7767B05D7FBEB76207024C650988FA77D5 LH:1, LR:0, LS:2}"
E[2019-02-03|18:12:59.230] enterPropose: Error signing proposal module=consensus height=1 round=0 err="Error signing proposal: Step regression at height 1 round 0. Got 1, last step 2"
I[2019-02-03|18:13:02.233] Timed out module=consensus dur=3s height=1 round=0 step=RoundStepPropose
I[2019-02-03|18:13:02.233] enterPrevote(1/0). Current: 1/0/RoundStepPropose module=consensus
I[2019-02-03|18:13:02.233] enterPrevote: ProposalBlock is nil module=consensus height=1 round=0
E[2019-02-03|18:13:02.234] Error signing vote module=consensus height=1 round=0 vote="Vote{0:AD278B7767B0 1/00/1(Prevote) 000000000000 000000000000 @ 2019-02-04T02:13:02.233897Z}" err="Error signing vote: Conflicting data"
```
Notice the EOF, the step regression, and the conflicting data.
* wal: change errors to be DataCorruptionError
* exit on corrupt WAL
* fix log
* fix new line
* Introduce CommitSig alias for Vote in Commit (#3245)
* types: memoize height/round in commit instead of first vote
* types: commit.ValidateBasic in VerifyCommit
* types: new CommitSig alias for Vote
In preparation for reducing the redundancy in Commits, we introduce the
CommitSig as an alias for Vote. This is non-breaking on the protocol,
and minor breaking on the Go API, as Commit now contains a list of
CommitSig instead of Vote.
* remove dependence on ToVote
* update some comments
* fix tests
* fix tests
* fixes from review
* Revert "quick fix for CircleCI (#2279)"
This reverts commit 1cf6712a36e8ecc843a68aa373748e89e0afecba.
* switch to golangci-lint from gometalinter :speedboat:
* remove or comment out unused code
* preallocating memory when we can
* comment out clist tests w/ depend on runtime.SetFinalizer
* use nolint label instead of commenting
* remove gotype linter
WARN [runner/nolint] Found unknown linters in //nolint directives: gotype
* rename TestGCRandom to _TestGCRandom
* cmn: GetFreePort (#3255)
* fix non deterministic test failures and race in privval socket (#3258)
* node: decrease retry conn timeout in test
Should fix #3256
The retry timeout was set to the default, which is the same as the
accept timeout, so it's no wonder this would fail. Here we decrease the
retry timeout so we can try many times before the accept timeout.
* p2p: increase handshake timeout in test
This fails sometimes, presumably because the handshake timeout is so low
(only 50ms). So increase it to 1s. Should fix #3187
* privval: fix race with ping. closes #3237
Pings happen in a go-routine and can happen concurrently with other
messages. Since we use a request/response protocol, we expect to send a
request and get back the corresponding response. But with pings
happening concurrently, this assumption could be violated. We were using
a mutex, but only a RWMutex, where the RLock was being held for sending
messages - this was to allow the underlying connection to be replaced if
it fails. Turns out we actually need to use a full lock (not just a read
lock) to prevent multiple requests from happening concurrently.
* node: fix test name. DelayedStop -> DelayedStart
* autofile: Wait() method
In the TestWALTruncate in consensus/wal_test.go we remove the WAL
directory at the end of the test. However the wal.Stop() does not
properly wait for the autofile group to finish shutting down. Hence it
was possible that the group's go-routine is still running when the
cleanup happens, which causes a panic since the directory disappeared.
Here we add a Wait() method to properly wait until the go-routine exits
so we can safely clean up. This fixes #2852.
* p2p/conn: don't hold stopMtx while waiting (#3254)
* p2p/conn: fix deadlock in FlushStop/OnStop
* makefile: set_with_deadlock
* close doneSendRoutine at end of sendRoutine
* conn: initialize channs in OnStart
* Add remote signer test harness (KMS) (#3149)
* WIP: Starts adding remote signer test harness
This commit adds a new command to Tendermint to allow for us to build a
standalone binary to test remote signers such as KMS
(https://github.com/tendermint/kms).
Right now, all it does is test that the local public key matches the
public key reported by the client, and fails at the point where it
attempts to get the client to sign a proposal.
* Fixes typo
* Fixes proposal validation test
This commit fixes the proposal validation test as per #3149. It also
moves the test harness into its own internal package to isolate its
exports from the `privval` package.
* Adds vote signing validation
* Applying recommendations from #3149
* Adds function descriptions for test harness
* Adds ability to ask remote signer to shut down
Prior to this commit, the remote signer needs to manually be shut down,
which is not ideal for automated testing. This commit allows us to send
a poison pill message to the KMS to let it shut down gracefully once
testing is done (whether the tests pass or fail).
* Adds tests for remote signer test harness
This commit makes some minor modifications to a few files to allow for
testing of the remote signer test harness. Two tests are added here:
checking for a fully successful (the ideal) case, and for the case where
the maximum number of retries has been reached when attempting to accept
incoming connections from the remote signer.
* Condenses serialization of proposals and votes using existing Tendermint functions
* Removes now-unnecessary amino import and codec
* Adds error message for vote signing failure
* Adds key extraction command for integration test
Took the code from here:
https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to
create a simple utility command to extract a key from a local Tendermint
validator for use in KMS integration testing.
* Makes path expansion success non-compulsory
* Fixes segfault on SIGTERM
We need an additional variable to keep track of whether we're
successfully connected, otherwise hitting Ctrl+Break during execution
causes a segmentation fault. This now allows for a clean shutdown.
* Consolidates shutdown checks
* Adds comments indicating codes for easy lookup
* Adds Docker build for remote signer harness
Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to
override the image name and Dockerfile using environment variables.
Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow
for building the `remote_val_harness` Docker image.
* Adds build_remote_val_harness_docker_image to .PHONY
* Removes remote signer poison pill messaging functionality
* Reduces fluff code in command line parsing
As per
https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788,
this reduces the amount of fluff code in the PR down to the bare
minimum.
* Fixes ordering of error check and info log
* Moves remove_val_harness cmd into tools folder
It seems to make sense to rather keep the remote signer test harness in
its own tool folder (now rather named `tm-signer-harness` to keep with
the tool naming convention). It is actually a separate tool, not meant
to be one of the core binaries, but supplementary and supportive.
* Updates documentation for tm-signer-harness
* Refactors flag parsing to be more compact and less redundant
* Adds version sub-command help
* Removes extraneous flags parsing
* Adds CHANGELOG_PENDING entry for tm-signer-harness
* Improves test coverage
Adds a few extra parameters to the `MockPV` type to fake broken vote and
proposal signing. Also adds some more tests for the test harness so as
to increase coverage for failed cases.
* Fixes formatting for CHANGELOG_PENDING.md
* Fix formatting for documentation config
* Point users towards official Tendermint docs for tools documentation
* Point users towards official Tendermint docs for tm-signer-harness
* Remove extraneous constant
* Rename TestHarness.sc to TestHarness.spv for naming consistency
* Refactor to remove redundant goroutine
* Refactor conditional to cleaner switch statement and better error handling for listener protocol
* Remove extraneous goroutine
* Add note about installing tmkms via Cargo
* Fix typo in naming of output signing key
* Add note about where to find chain ID
* Replace /home/user with ~/ for brevity
* Fixes "signer.key" typo
* Minor edits for clarification for tm-signer-harness bulid/setup process
* p2p: fix infinite loop in addrbook (#3232)
* failing test
* fix infinite loop in addrbook
There are cases where we only have a small number of addresses marked
good ("old"), but the selection mechanism keeps trying to select more of these
addresses, and hence ends up in an infinite loop. Here we fix this to
only try and select such "old" addresses if we have enough of them. Note this
means, if we don't have enough of them, we may return more "new"
addresses than otherwise expected by the newSelectionBias.
This whole GetSelectionWithBias method probably needs to be rewritten,
but this is a quick fix for the issue.
* changelog
* fix infinite loop if not enough new addrs
* fix another potential infinite loop
if a.nNew == 0 -> pickFromOldBucket=true, but we don't have enough items
(a.nOld > len(oldBucketToAddrsMap) false)
* Revert "fix another potential infinite loop"
This reverts commit 146540c1125597162bd89820d611f6531f5e5e4b.
* check num addresses instead of buckets, new test
* fixed the int division
* add slack to bias % in test, lint fixes
* Added checks for selection content in test
* test cleanup
* Apply suggestions from code review
Co-Authored-By: ebuchman <ethan@coinculture.info>
* address review comments
* change after Anton's review comments
* use the same docker image we use for testing
when building a binary for localnet
* switch back to circleci classic
* more review comments
* more review comments
* refactor addrbook_test
* build linux binary inside docker
in attempt to fix
```
--> Running dep
+ make build-linux
GOOS=linux GOARCH=amd64 make build
make[1]: Entering directory `/home/circleci/.go_workspace/src/github.com/tendermint/tendermint'
CGO_ENABLED=0 go build -ldflags "-X github.com/tendermint/tendermint/version.GitCommit=`git rev-parse --short=8 HEAD`" -tags 'tendermint' -o build/tendermint ./cmd/tendermint/
p2p/pex/addrbook.go:373:13: undefined: math.Round
```
* change dir from /usr to /go
* use concrete Go version for localnet binary
* check for nil addresses just to be sure
* addrbook_test: preallocate memory for bookSizes (#3268)
Fixes https://circleci.com/gh/tendermint/tendermint/44901
* prepare v0.29.2 (#3272)
* update changelog
* linkify
* bump version
* update main changelog
* final fixes
* entry for wal fix
* changelog preamble
* remove a line
* secp256k1: change build tags (#3277)
* remove MixEntropy (#3278)
* remove MixEntropy
* changelog
* review comment: cleaner constant for N/2, delete secp256k1N and use (#3279)
`secp256k1.S256().N` directly instead
* treat validator updates as set (#3222)
* Initial commit for 3181..still early
* unit test updates
* unit test updates
* fix check of dups accross updates and deletes
* simplify the processChange() func
* added overflow check utest
* Added checks for empty valset, new utest
* deepcopy changes in processUpdate()
* moved to new API, fixed tests
* test cleanup
* address review comments
* make sure votePower > 0
* gofmt fixes
* handle duplicates and invalid values
* more work on tests, review comments
* Renamed and explained K
* make TestVal private
* split verifyUpdatesAndComputeNewPriorities.., added check for deletes
* return error if validator set is empty after processing changes
* address review comments
* lint err
* Fixed the total voting power and added comments
* fix lint
* fix lint
* Sec/bucky/35 commit duplicate evidence (#36)
Don't add committed evidence to evpool
* Reject blocks with committed evidence (#37)
* evidence: NewEvidencePool takes evidenceDB
* evidence: failing TestStoreCommitDuplicate
tendermint/security#35
* GetEvidence -> GetEvidenceInfo
* fix TestStoreCommitDuplicate
* comment in VerifyEvidence
* add check if evidence was already seen
- modify EventPool interface (EventStore is not known in ApplyBlock):
- add IsCommitted method to iface
- add test
* update changelog
* fix TestStoreMark:
- priority in evidence info gets reset to zero after evidence gets committed
* review comments: simplify EvidencePool.IsCommitted
- delete obsolete EvidenceStore.IsCommitted
* add simple test for IsCommitted
* update changelog: this is actually breaking (PR number still missing)
* fix TestStoreMark:
- priority in evidence info gets reset to zero after evidence gets
committed
* review suggestion: simplify return
* types.NewCommit (#3275)
* types.NewCommit
* use types.NewCommit everywhere
* fix log in unsafe_reset
* memoize height and round in constructor
* notes about deprecating toVote
* bring back memoizeHeightRound
* Prepare v0.30.0 (#3287)
* changelog, upgrading, version
* update for evidence fixes
* linkify
* fix an entry
* fix failure in TestProposerFrequency (#3293)
```
--- FAIL: TestProposerFrequency (2.50s)
panic: empty validator set [recovered]
panic: empty validator set
goroutine 91 [running]:
testing.tRunner.func1(0xc000a98c00)
/usr/local/go/src/testing/testing.go:792 +0x6a7
panic(0xeae7e0, 0x11fbb30)
/usr/local/go/src/runtime/panic.go:513 +0x1b9
github.com/tendermint/tendermint/types.(*ValidatorSet).RescalePriorities(0xc0000e7380, 0x0)
/go/src/github.com/tendermint/tendermint/types/validator_set.go:106 +0x1ac
github.com/tendermint/tendermint/state.TestProposerFrequency(0xc000a98c00)
/go/src/github.com/tendermint/tendermint/state/state_test.go:335 +0xb44
testing.tRunner(0xc000a98c00, 0x111a4d8)
/usr/local/go/src/testing/testing.go:827 +0x163
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:878 +0x65a
FAIL github.com/tendermint/tendermint/state 3.139s
```
* make govet linter pass (#3292)
* make govet linter pass
Refs #3262
* close PipeReader and check for err
* make gosec linter pass (#3294)
* not related to linter: remove obsolete constants:
- `Insecure` and `Secure` and type `Security` are not used anywhere
* not related to linter: update example
- NewInsecure was deleted; change example to NewRemoteDB
* address: Binds to all network interfaces (gosec):
- bind to localhost instead of 0.0.0.0
- regenerate test key and cert for this purpose (was valid for ::) and
otherwise we would see:
transport: authentication handshake failed: x509: certificate is
valid for ::, not 127.0.0.1\"
(used https://github.com/google/keytransparency/blob/master/scripts/gen_server_keys.sh
to regenerate certs)
* use sha256 in tests instead of md5; time difference is negligible
* nolint usage of math/rand in test and add comment on its import
- crypto/rand is slower and we do not need sth more secure in tests
* enable linter in circle-ci
* another nolint math/rand in test
* replace another occurrence of md5
* consistent comment about importing math/rand
* test blockExec does not panic if all vals removed (#3241)
Fix for #3224
Also address #2084
* types: validator set update tests (#3284)
* types: validator set update tests
* docs: abci val updates must not include duplicates
* consensus: flush wal on stop (#3297)
Should fix #3295
Also partial fix of #3043
* update codeowners (#3305)
limit the scope of @zramsay because he's annoyed by notifications
* return an error on `show_validator` (#3316)
* return a command error prior to init
* add a pending log entry
* Update CHANGELOG_PENDING.md
Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com>
closes: #3314
* fix test-vectors to actually match the sign bytes: (#3312)
- sign bytes are length prefixed
- change test to use SignBytes methods instead of calling amino methods
directly
Resolves #3311
ref: #2455 #2508
* rpc/net_info: change RemoteIP type from net.IP to String (#3309)
* rpc/net_info: change RemoteIP type from net.IP to String
Before:
"AAAAAAAAAAAAAP//rB8ktw=="
which is amino-encoded net.IP byte slice
After:
"192.0.2.1"
Fixes #3251
* rpc/net_info: non-empty response in docs
* cs/wal: refuse to encode msg that is bigger than maxMsgSizeBytes (#3303)
Earlier this week somebody posted this in GoS Riot chat:
```
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 878916964 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 825701731 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 1631073634 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 912418148 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.600] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[failed to read data: EOF]"
E[2019-02-12|10:38:37.600] Error on catchup replay. Proceeding to start ConsensusState anyway module=consensus err="Cannot replay height 7242. WAL does not contain #ENDHEIGHT for 7241"
E[2019-02-12|10:38:37.861] Error dialing peer module=p2p err="dial tcp 35.183.126.181:26656: i/o timeout
```
Note the length error messages. What has happened is the length field got corrupted probably. I've looked at the code and noticed that we don't check the msg size during encoding. This PR fixes that. It also improves a few error messages in WALDecoder.
* cs: reset triggered timeout precommit (#3310)
* Reset TriggeredTimeoutPrecommit as part of updateToState
* Add failing test and fix
* fix DATA RACE in TestResetTimeoutPrecommitUponNewHeight
```
WARNING: DATA RACE
Read at 0x00c001691d28 by goroutine 691:
github.com/tendermint/tendermint/consensus.decideProposal()
/go/src/github.com/tendermint/tendermint/consensus/common_test.go:133 +0x121
github.com/tendermint/tendermint/consensus.TestResetTimeoutPrecommitUponNewHeight()
/go/src/github.com/tendermint/tendermint/consensus/state_test.go:1389 +0x958
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
Previous write at 0x00c001691d28 by goroutine 931:
github.com/tendermint/tendermint/consensus.(*ConsensusState).updateToState()
/go/src/github.com/tendermint/tendermint/consensus/state.go:562 +0x5b2
github.com/tendermint/tendermint/consensus.(*ConsensusState).finalizeCommit()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1340 +0x141e
github.com/tendermint/tendermint/consensus.(*ConsensusState).tryFinalizeCommit()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1255 +0x66e
github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit.func1()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1201 +0x135
github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1232 +0x94b
github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1657 +0x132e
github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1503 +0x8f
github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
/go/src/github.com/tendermint/tendermint/consensus/state.go:694 +0xa1e
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd
Goroutine 691 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:878 +0x659
testing.runTests.func1()
/usr/local/go/src/testing/testing.go:1119 +0xa8
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
testing.runTests()
testing.(*M).Run()
/usr/local/go/src/testing/testing.go:1034 +0x2ee
main.main()
_testmain.go:216 +0x332
```
* fix another DATA RACE by locking consensus
```
WARNING: DATA RACE
Read at 0x00c009b835a8 by goroutine 871:
github.com/tendermint/tendermint/consensus.(*ConsensusState).createProposalBlock()
/go/src/github.com/tendermint/tendermint/consensus/state.go:955 +0x7c
github.com/tendermint/tendermint/consensus.decideProposal()
/go/src/github.com/tendermint/tendermint/consensus/common_test.go:127 +0x53
github.com/tendermint/tendermint/consensus.TestResetTimeoutPrecommitUponNewHeight()
/go/src/github.com/tendermint/tendermint/consensus/state_test.go:1389 +0x958
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
Previous write at 0x00c009b835a8 by goroutine 931:
github.com/tendermint/tendermint/consensus.(*ConsensusState).updateHeight()
/go/src/github.com/tendermint/tendermint/consensus/state.go:446 +0xb7
github.com/tendermint/tendermint/consensus.(*ConsensusState).updateToState()
/go/src/github.com/tendermint/tendermint/consensus/state.go:542 +0x22f
github.com/tendermint/tendermint/consensus.(*ConsensusState).finalizeCommit()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1340 +0x141e
github.com/tendermint/tendermint/consensus.(*ConsensusState).tryFinalizeCommit()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1255 +0x66e
github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit.func1()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1201 +0x135
github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1232 +0x94b
github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1657 +0x132e
github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1503 +0x8f
github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
/go/src/github.com/tendermint/tendermint/consensus/state.go:694 +0xa1e
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd
```
* Fix failing test
* Delete profile.out
* fix data races
* improve ResetTestRootWithChainID() concurrency safety (#3291)
* improve ResetTestRootWithChainID() concurrency safety
Rely on ioutil.TempDir() to create test root directories and ensure
multiple same-chain id test cases can run in parallel.
* Update config/toml.go
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* clean up test directories after completion
Closes: #1034
* Remove redundant EnsureDir call
* s/PanicSafety()/panic()/s
* Put create dir functionality back in ResetTestRootWithChainID
* Place test directories in OS's tempdir
In modern UNIX and UNIX-like systems /tmp is very often
mounted as tmpfs. This might speed test execution a bit.
* Set 0700 to a const
* rootsDirs -> configRootDirs
* Don't double remove directories
* Avoid global variables
* Fix consensus tests
* Reduce defer stack
* Address review comments
* Try to fix tests
* Update CHANGELOG_PENDING.md
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* Update consensus/common_test.go
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* Update consensus/common_test.go
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* 3291 follow-up (#3323)
* changelog: use issue number instead of PR number
* follow up to #3291
- rpc/test/helpers.go add StopTendermint(node) func
- remove ensureDir(filepath.Dir(walFile), 0700)
- mempool/mempool_test.go add type cleanupFunc func()
* cmd/show_validator: wrap err to make it more clear
* p2p: check secret conn id matches dialed id (#3321)
ref: [#3010 (comment)](https://github.com/tendermint/tendermint/issues/3010#issuecomment-464287627)
> I tried searching for code where we authenticate a peer against its NetAddress.ID and couldn't find it. I don't see a reason to switch to Noise, but a need to ensure that the node's ID is authenticated e.g. after dialing from the address book.
* p2p: check secret conn id matches dialed id
* Fix all p2p tests & make code compile
* add simple test for dialing with wrong ID
* update changelog
* address review comments
* yet another place where to use IDAddressString and fix
testSetupMultiplexTransport
* update changelog and bump version
* cs: sync WAL more frequently (#3300)
As per #3043, this adds a ticker to sync the WAL every 2s while the WAL is running.
* Flush WAL every 2s
This adds a ticker that flushes the WAL every 2s while the WAL is
running. This is related to #3043.
* Fix spelling
* Increase timeout to 2mins for slower build environments
* Make WAL sync interval configurable
* Add TODO to replace testChan with more comprehensive testBus
* Remove extraneous debug statement
* Remove testChan in favour of using system time
As per
https://github.com/tendermint/tendermint/pull/3300#discussion_r255886586,
this removes the `testChan` WAL member and replaces the approach with a
system time-oriented one. In this new approach, we keep track of the
system time at which each flush and periodic flush successfully
occurred.
The naming of the various functions is also updated here to be more
consistent with "flushing" as opposed to "sync'ing".
* Update naming convention and ensure lock for timestamp update
* Add Flush method as part of WAL interface
Adds a `Flush` method as part of the WAL interface to enforce the idea
that we can manually trigger a WAL flush from outside of the WAL. This
is employed in the consensus state management to flush the WAL prior to
signing votes/proposals, as per https://github.com/tendermint/tendermint/issues/3043#issuecomment-453853630
* Update CHANGELOG_PENDING
* Remove mutex approach and replace with DI
The dependency injection approach to dealing with testing concerns could
allow similar effects to some kind of "testing bus"-based approach. This
commit introduces an example of this, where instead of relying on
(potentially fragile) timing of things between the code and the test, we
inject code into the function under test that can signal the test
through a channel.
This allows us to avoid the `time.Sleep()`-based approach previously
employed.
* Update comment on WAL flushing during vote signing
Co-Authored-By: thanethomson <connect@thanethomson.com>
* Simplify flush interval definition
Co-Authored-By: thanethomson <connect@thanethomson.com>
* Expand commentary on WAL disk flushing
Co-Authored-By: thanethomson <connect@thanethomson.com>
* Add broken test to illustrate WAL sync test problem
Removes test-related state (dependency injection code) from the WAL data
structure and adds test code to illustrate the problem with using
`WALGenerateNBlocks` and `wal.SearchForEndHeight` to test periodic
sync'ing.
* Fix test error messages
* Use WAL group buffer size to check for flush
A function is added to `libs/autofile/group.go#Group` in order to return
the size of the buffered data (i.e. data that has not yet been flushed
to disk). The test now checks that, prior to a `time.Sleep`, the group
buffer has data in it. After the `time.Sleep` (during which time the
periodic flush should have been called), the buffer should be empty.
* Remove config root dir removal from #3291
* Add godoc for NewWAL mentioning periodic sync
* docs: fix rpc Tx() method docs (#3331)
* update changelog
* cs: update wal comments (#3334)
* cs: update wal comments
Follow-up to https://github.com/tendermint/tendermint/pull/3300
* Update consensus/wal.go
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* refactor decideProposal in common_test (#3343)
* fix TestWALPeriodicSync (#3342)
The test was sometimes failing due to processFlushTicks being called too
early. The solution is to call wal#Start later in the test.
* pubsub 2.0 (#3227)
* green pubsub tests :OK:
* get rid of clientToQueryMap
* Subscribe and SubscribeUnbuffered
* start adapting other pkgs to new pubsub
* nope
* rename MsgAndTags to Message
* remove TagMap
it does not bring any additional benefits
* bring back EventSubscriber
* fix test
* fix data race in TestStartNextHeightCorrectly
```
Write at 0x00c0001c7418 by goroutine 796:
github.com/tendermint/tendermint/consensus.TestStartNextHeightCorrectly()
/go/src/github.com/tendermint/tendermint/consensus/state_test.go:1296 +0xad
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
Previous read at 0x00c0001c7418 by goroutine 858:
github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1631 +0x1366
github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1476 +0x8f
github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
/go/src/github.com/tendermint/tendermint/consensus/state.go:667 +0xa1e
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:628 +0x794
Goroutine 796 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:878 +0x659
testing.runTests.func1()
/usr/local/go/src/testing/testing.go:1119 +0xa8
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
testing.runTests()
/usr/local/go/src/testing/testing.go:1117 +0x4ee
testing.(*M).Run()
/usr/local/go/src/testing/testing.go:1034 +0x2ee
main.main()
_testmain.go:214 +0x332
Goroutine 858 (running) created at:
github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
/go/src/github.com/tendermint/tendermint/consensus/state.go:334 +0x221
github.com/tendermint/tendermint/consensus.startTestRound()
/go/src/github.com/tendermint/tendermint/consensus/common_test.go:122 +0x63
github.com/tendermint/tendermint/consensus.TestStateFullRound…
pinkiebell
added a commit
to leapdao/tendermint
that referenced
this pull request
Apr 14, 2019
commit 4695414393a3e0bc3836fb1cece7f1f3768d3311
Merge: 6cc3f4d8 def5c8cf
Author: Ethan Buchman <ethan@coinculture.info>
Date: Fri Apr 12 10:56:03 2019 -0400
Merge pull request #3548 from tendermint/release/v0.31.4
Release/v0.31.4
commit def5c8cf124ff58cc9c4a61ea017808a0c81c722
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Fri Apr 12 16:48:34 2019 +0200
address review comments: (#3550)
- mention ADR in release summary
- remove [p2p] api changes
- amend v0.31.3 log to contain note about breaking change
commit b6da8880c22202a7061ae2093ef01819438a136e
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Fri Apr 12 14:24:51 2019 +0200
prepare v0.31.4 release:
- prep changelog
- add missing changelog entries
- fix minor glitch in existing changelog (v0.31.2)
- bump versions
commit a453628c4e43c67c747d5078704479aa73716182
Author: Martin Dyring-Andersen <martin@dyring-andersen.dk>
Date: Fri Apr 12 13:25:14 2019 +0200
Fix a couple of typos (#3547)
Fix some typos in p2p/transport.go
commit 4e4224213f3eaa89abeac6686efe4ac970ad63d0
Author: Sean Braithwaite <brapse@gmail.com>
Date: Fri Apr 12 12:32:00 2019 +0200
adr: Peer Behaviour (#3539)
* [adr] ADR 037: Peer Behaviour inital draft
* Update docs/architecture/adr-037-peer-behaviour.md
Co-Authored-By: brapse <brapse@gmail.com>
* Update docs/architecture/adr-037-peer-behaviour.md
Co-Authored-By: brapse <brapse@gmail.com>
* [docs] adr-037 Better footnote styling
* [ADR] ADR-037 adjust Footnotes for github markdown
* [ADR] ADR-037 fix numbered list
commit b5b3b85697a9c29adaab0e1ab5d1380459e2c5eb
Author: Alexander Simmerl <a.simmerl@gmail.com>
Date: Fri Apr 12 12:31:02 2019 +0200
Bring back NodeInfo NetAddress form the dead (#3545)
A prior change to address accidental DNS lookups introduced the
SocketAddr on peer, which was then used to add it to the addressbook.
Which in turn swallowed the self reported port of the peer, which is
important on a reconnect. This change revives the NetAddress on NodeInfo
which the Peer carries, but now returns an error to avoid nil
dereferencing another issue observed in the past. Additionally we could
potentially address #3532, yet the original problem statemenf of that
issue stands.
As a drive-by optimisation `MarkAsGood` now takes only a `p2p.ID` which
makes it interface a bit stricter and leaner.
commit 18d2c45c334d9f22621ea0af0686e2e54937fb40
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Fri Apr 12 10:46:07 2019 +0200
rpc: Fix response time grow over time (#3537)
* rpc: store validator info periodly
* increase ValidatorSetStoreInterval
also
- unexpose it
- add a comment
- refactor code
- add a benchmark, which shows that 100000 results in ~ 100ms to get 100
validators
* make the change non-breaking
* expand comment
* rename valSetStoreInterval to valSetCheckpointInterval
* change the panic msg
* add a test and changelog entry
* update changelog entry
* update changelog entry
* add a link to PR
* fix test
* Update CHANGELOG_PENDING.md
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* update comment
* use MaxInt64 func
commit c3df21fe827aad0447bb59a9546c32db8406a5fd
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Apr 11 18:59:14 2019 +0300
add missing changelog entry (#3544)
* add missing changelog entry
commit bcec8be035a89b5d08ee00b8cc7115017a58c7e0
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Apr 11 15:32:16 2019 +0200
p2p: do not log err if peer is private (#3474)
* add actionable advice for ErrAddrBookNonRoutable err
Should replace https://github.com/tendermint/tendermint/pull/3463
* reorder checks in addrbook#addAddress so
ErrAddrBookPrivate is returned first
and do not log error in DialPeersAsync if the address is private
because it's not an error
commit 9a415b057238d118c1edf23d673ebf9ce4f8d2ae
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Tue Apr 9 18:21:35 2019 +0200
docs: abci#Commit: better explain the possible deadlock (#3536)
commit 40da355234bda9d510dca370e94b646450673879
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Wed Apr 3 14:56:51 2019 +0200
docs: fix block.Header.Time description (#3529)
It's not proposer local time anymore, but a weighted median
Fixes #3514
commit f965a4db15796dfca4e504d93eaa83760ced4af2
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Wed Apr 3 11:22:52 2019 +0200
p2p: seed mode refactoring (#3011)
ListOfKnownAddresses is removed
panic if addrbook size is less than zero
CrawlPeers does not attempt to connect to existing or peers we're currently dialing
various perf. fixes
improved tests (though not complete)
move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716)
* addrbook: preallocate memory when saving addrbook to file
* addrbook: remove oldestFirst struct and check for ID
* oldestFirst replaced with sort.Slice
* ID is now mandatory, so no need to check
* addrbook: remove ListOfKnownAddresses
GetSelection is used instead in seed mode.
* addrbook: panic if size is less than 0
* rewrite addrbook#saveToFile to not use a counter
* test AttemptDisconnects func
* move IsDialingOrExistingAddress check into DialPeerWithAddress
* save and cleanup crawl peer data
* get rid of DefaultSeedDisconnectWaitPeriod
* make linter happy
* fix TestPEXReactorSeedMode
* fix comment
* add a changelog entry
* Apply suggestions from code review
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress
* lowercase errors
* do not persist seed data
pros:
- no extra files
- less IO
cons:
- if the node crashes, seed might crawl a peer too soon
* fixes after Ethan's review
* add a changelog entry
* we should only consult Switch about peers
checking addrbook size does not make sense since only PEX reactor uses
it for dialing peers!
https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
commit 086d6cbe8c4432a1a03d7924ae8352c42dc9cc4f
Merge: e4a03f24 6cc3f4d8
Author: Ethan Buchman <ethan@coinculture.info>
Date: Tue Apr 2 16:49:44 2019 -0400
Merge pull request #3527 from tendermint/v0.31
Merge V0.31.3 back to develop
commit 6cc3f4d87cae6589aef0ac2b4707f1ebc8a669b2
Merge: 79e9f205 3cfd9757
Author: Ethan Buchman <ethan@coinculture.info>
Date: Tue Apr 2 16:45:04 2019 -0400
Merge pull request #3525 from tendermint/release/v0.31.3
Release/v0.31.3
commit 3cfd9757a78fc2398c6853aace2541338be8eaef
Author: Ethan Buchman <ethan@coinculture.info>
Date: Tue Apr 2 09:14:33 2019 -0400
changelog and version v0.31.3
commit 882622ec10ba3fdc9f360da65bfb2bbbd13193ed
Author: Ethan Buchman <ethan@coinculture.info>
Date: Mon Apr 1 19:59:57 2019 -0400
Fixes tendermint/tendermint#3522
* OriginalAddr -> SocketAddr
OriginalAddr records the originally dialed address for outbound peers,
rather than the peer's self reported address. For inbound peers, it was
nil. Here, we rename it to SocketAddr and for inbound peers, set it to
the RemoteAddr of the connection.
* use SocketAddr
Numerous places in the code call peer.NodeInfo().NetAddress().
However, this call to NetAddress() may perform a DNS lookup if the
reported NodeInfo.ListenAddr includes a name. Failure of this lookup
returns a nil address, which can lead to panics in the code.
Instead, call peer.SocketAddr() to return the static address of the
connection.
* remove nodeInfo.NetAddress()
Expose `transport.NetAddress()`, a static result determined
when the transport is created. Removing NetAddress() from the nodeInfo
prevents accidental DNS lookups.
* fixes from review
* linter
* fixes from review
commit 1ecf8148385692da971a46e14091270b23c6dff3
Author: Ethan Buchman <ethan@coinculture.info>
Date: Mon Apr 1 19:45:57 2019 -0400
Fixes tendermint/tendermint#3439
* make sure we create valid private keys:
- genPrivKey samples and rejects invalid fieldelems (like libsecp256k1)
- GenPrivKeySecp256k1 uses `(sha(secret) mod (n − 1)) + 1`
- fix typo, rename test file: s/secpk256k1/secp256k1/
* Update crypto/secp256k1/secp256k1.go
commit e4a03f249dcb7d2e833fc20add382b2a5e7f167e
Author: Greg Szabo <16846635+greg-szabo@users.noreply.github.com>
Date: Mon Apr 1 14:18:18 2019 -0400
Release message changelog link fix (#3519)
commit 56d8aa42b3ca628c6544cab1f902f8289738c97a
Merge: 0ae41cc6 79e9f205
Author: Ethan Buchman <ethan@coinculture.info>
Date: Mon Apr 1 14:17:58 2019 -0400
Merge pull request #3520 from tendermint/v0.31
Merge v0.31.2 release back to develop
commit 79e9f205782f1adb94b4150561e414280115cf4e
Merge: a0234aff ab24925c
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Mon Apr 1 17:58:28 2019 +0200
Merge pull request #3518 from tendermint/prepare-release-v0.31.2
Release v0.31.2
commit ab24925c9432e9eff06b34445b1760217436f87d
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Mon Apr 1 12:32:37 2019 +0200
prepare changelog and bump versions to v0.31.2
commit 0ae41cc663014454184c2d31328b53dd45dd34d1
Author: Greg Szabo <16846635+greg-szabo@users.noreply.github.com>
Date: Mon Apr 1 11:47:00 2019 -0400
Fix for wrong version tag (#3517)
* Fix for wrong version tag (tag on the release branch instead of master)
commit 422d04c8bae80e103474d37bd5a5afd0061e8d72
Author: Ethan Buchman <ethan@coinculture.info>
Date: Sun Mar 31 07:14:18 2019 -0400
Bucky/mempool txsmap (#3512)
* mempool: resCb -> globalCb
* reqResCb takes an externalCb
* failing test for #3509
* txsMap is sync.Map
* update changelog
commit 2233dd45bd774905b1ba545eeadea5ebe49d1e31
Author: zjubfd <baifudong@lancai.cn>
Date: Sat Mar 30 01:47:53 2019 +0800
libs: remove useless code in group (#3504)
* lib: remove useless code in group
* update change log
* Update CHANGELOG_PENDING.md
Co-Authored-By: guagualvcha <baifudong@lancai.cn>
commit 9199f3f6132ff24500b538068450eae7d11bef64
Author: Greg Szabo <16846635+greg-szabo@users.noreply.github.com>
Date: Fri Mar 29 07:57:16 2019 -0400
Release management using CircleCI (#3498)
* Release management using CircleCI
* Changelog updated
commit 6c1a4b513747fed5be06edffd16db4a1c8749041
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Thu Mar 28 17:39:09 2019 +0100
blockchain: comment out logger in test code that causes a race condition (#3500)
commit c7bb9984979b72e81a569b4540a2da1e73c1c470
Merge: d586945d 7b72436c
Author: Ethan Buchman <ethan@coinculture.info>
Date: Thu Mar 28 08:07:59 2019 -0400
Merge pull request #3502 from tendermint/bucky/merge-master
Bucky/merge master
commit 7b72436c751587141d8de605ccbcec3514dbea77
Merge: d586945d a0234aff
Author: Ethan Buchman <ethan@coinculture.info>
Date: Thu Mar 28 08:12:05 2019 -0400
Merge branch 'master' into bucky/merge-master
commit a0234affb6959a0aec285eebf3a3963251d2d186
Merge: 0d985ede 9390a810
Author: Ethan Buchman <ethan@coinculture.info>
Date: Thu Mar 28 07:57:42 2019 -0400
Merge pull request #3489 from tendermint/release/v0.31.1
Release/v0.31.1
commit 9390a810ebda0b1aebd90f6e8b8688562bdf7958
Author: Ethan Buchman <ethan@coinculture.info>
Date: Wed Mar 27 21:03:28 2019 -0400
minor changelog updates (#3499)
commit a49d80b89c9df3cb2497adb8175915c654479708
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 19:12:01 2019 +0100
catch up with develop and rebase on current release to include #3482
commit ccfe75ec4a701ad7dcaf1f26df772b0881ab1184
Author: Sean Braithwaite <brapse@gmail.com>
Date: Wed Mar 27 18:51:57 2019 +0100
docs: Fix broken links (#3482) (#3488)
* docs: fix broken links (#3482)
A bunch of links were broken in the documentation s they included the
`docs` prefix.
* Update CHANGELOG_PENDING
* docs: switch to relative links for github compatitibility (#3482)
commit d586945d6981262759e0e681e7bc92a74637185d
Author: Sean Braithwaite <brapse@gmail.com>
Date: Wed Mar 27 18:51:57 2019 +0100
docs: Fix broken links (#3482) (#3488)
* docs: fix broken links (#3482)
A bunch of links were broken in the documentation s they included the
`docs` prefix.
* Update CHANGELOG_PENDING
* docs: switch to relative links for github compatitibility (#3482)
commit ae88965ff6ddf174607a02eee83cfc5ed11c6cbb
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 17:46:29 2019 +0100
changelog: add summary & fix link & add external contributor (#3490)
commit 233813483642c2ac370110dd42c3291556c26039
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 16:52:19 2019 +0100
bump versions
commit 1b33a50e6d07e48f2377db64ce944a435f9f2ec4
Merge: 3c7bb6b5 5fa540bd
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 16:50:59 2019 +0100
Merge remote-tracking branch 'remotes/origin/develop' into release/v0.31.1
commit 3c7bb6b571ac47e050a47013e22dbdfd68f02745
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 16:48:00 2019 +0100
Add some numbers for #2778
commit 5fa540bdc9eac01cc3df6de01119550d1122114b
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Wed Mar 27 16:45:34 2019 +0100
mempool: add a safety check, write tests for mempoolIDs (#3487)
* mempool: add a safety check, write tests for mempoolIDs
and document 65536 limit in the mempool reactor spec
follow-up to https://github.com/tendermint/tendermint/pull/2778
* rename the test
* fixes after Ismail's review
commit 52727863e187e85901e7f2e7285920a701d2a8e6
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 16:07:03 2019 +0100
add external contributors
commit e3f840e6a6f81406c44e760e578e48fcd3fedf12
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 16:04:43 2019 +0100
reset CHANGELOG_PENDING.md
commit ed63e1f378eaf465b714084fa1edd6ac4761782f
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 16:03:25 2019 +0100
Add more entries to the Changelog, fix formatting, linkify
commit 55b7118c981e93b920cead4616b91ec58130e316
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 27 15:35:32 2019 +0100
Prep changelog: copy from pending & update version
commit 5a25b75b1d586c6d9c3fdfc940057f7b85089a90
Author: zjubfd <baifudong@lancai.cn>
Date: Wed Mar 27 00:13:14 2019 +0800
p2p: refactor GetSelectionWithBias for addressbook (#3475)
Why submit this pr:
we have suffered from infinite loop in addrbook bug which takes us a long time to find out why process become a zombie peer. It have been fixed in #3232. But the ADDRS_LOOP is still there, risk of infinite loop is still exist.
The algorithm that to random pick a bucket is not stable, which means the peer may unluckily always choose the wrong bucket for a long time, the time and cpu cost is meaningless.
A simple improvement:
shuffle bucketsNew and bucketsOld, and pick necessary number of address from them. A stable
algorithm.
commit a4d9539544ba4377f16e797fea01090bc974e1b5
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Tue Mar 26 09:44:49 2019 +0100
rpc/client: include NetworkClient interface into Client interface (#3473)
I think it's nice when the Client interface has all the methods. If someone does not need a particular method/set of methods, she can use individual interfaces (e.g. NetworkClient, MempoolClient) or write her own interface.
technically breaking
Fixes #3458
commit 1bb8e02a962a1ffd7ce4e7355f648b644105f649
Author: HaoyangLiu <lhy.0318@163.com>
Date: Tue Mar 26 16:29:06 2019 +0800
mempool: fix broadcastTxRoutine leak (#3478)
Refs #3306, irisnet@fdbb676
I ran an irishub validator. After the validator node ran several days, I dump the whole goroutine stack. I found that there were hundreds of broadcastTxRoutine. However, the connected peer quantity was less than 30. So I belive that there must be broadcastTxRoutine leakage issue.
According to my analysis, I think the root cause of this issue locate in below code:
select {
case <-next.NextWaitChan():
// see the start of the for loop for nil check
next = next.Next()
case <-peer.Quit():
return
case <-memR.Quit():
return
}
As we know, if multiple paths are avaliable in the same time, then a random path will be selected. Suppose that next.NextWaitChan() and peer.Quit() are both avaliable, and next.NextWaitChan() is chosen.
// send memTx
msg := &TxMessage{Tx: memTx.tx}
success := peer.Send(MempoolChannel, cdc.MustMarshalBinaryBare(msg))
if !success {
time.Sleep(peerCatchupSleepIntervalMS * time.Millisecond)
continue
}
Then next will be non-empty and the peer send operation won't be success. As a result, this go routine will be track into infinite loop and won't be released.
My proposal is to check peer.Quit() and memR.Quit() in every loop no matter whether next is nil.
commit 6de7effb05581f9bea2e8af06e4e74a85c34bc5f
Author: Dev Ojha <ValarDragon@users.noreply.github.com>
Date: Tue Mar 26 01:27:29 2019 -0700
mempool no gossip back (#2778)
Closes #1798
This is done by making every mempool tx maintain a list of peers who its received the tx from. Instead of using the 20byte peer ID, it instead uses a local map from peerID to uint16 counter, so every peer adds 2 bytes. (Word aligned to probably make it 8 bytes)
This also required resetting the callback function on every CheckTx. This likely has performance ramifications for instruction caching. The actual setting operation isn't costly with the removal of defers in this PR.
* Make the mempool not gossip txs back to peers its received it from
* Fix adversarial memleak
* Don't break interface
* Update changelog
* Forgot to add a mtx
* forgot a mutex
* Update mempool/reactor.go
Co-Authored-By: ValarDragon <ValarDragon@users.noreply.github.com>
* Update mempool/mempool.go
Co-Authored-By: ValarDragon <ValarDragon@users.noreply.github.com>
* Use unknown peer ID
Co-Authored-By: ValarDragon <ValarDragon@users.noreply.github.com>
* fix compilation
* use next wait chan logic when skipping
* Minor fixes
* Add TxInfo
* Add reverse map
* Make activeID's auto-reserve 0
* 0 -> UnknownPeerID
Co-Authored-By: ValarDragon <ValarDragon@users.noreply.github.com>
* Switch to making the normal case set a callback on the reqres object
The recheck case is still done via the global callback, and stats
are also set via global callback
* fix merge conflict
* Addres comments
* Add cache tests
* add cache tests
* minor fixes
* update metrics in reqResCb and reformat code
* goimport -w mempool/reactor.go
* mempool: update memTx senders
I had to introduce txsMap for quick mempoolTx lookups.
* change senders type from []uint16 to sync.Map
Fixes DATA RACE:
```
Read at 0x00c0013fcd3a by goroutine 183:
github.com/tendermint/tendermint/mempool.(*MempoolReactor).broadcastTxRoutine()
/go/src/github.com/tendermint/tendermint/mempool/reactor.go:195 +0x3c7
Previous write at 0x00c0013fcd3a by D[2019-02-27|10:10:49.058] Read PacketMsg switch=3 peer=35bc1e3558c182927b31987eeff3feb3d58a0fc5@127.0.0.1
:46552 conn=MConn{pipe} packet="PacketMsg{30:2B06579D0A143EB78F3D3299DE8213A51D4E11FB05ACE4D6A14F T:1}"
goroutine 190:
github.com/tendermint/tendermint/mempool.(*Mempool).CheckTxWithInfo()
/go/src/github.com/tendermint/tendermint/mempool/mempool.go:387 +0xdc1
github.com/tendermint/tendermint/mempool.(*MempoolReactor).Receive()
/go/src/github.com/tendermint/tendermint/mempool/reactor.go:134 +0xb04
github.com/tendermint/tendermint/p2p.createMConnection.func1()
/go/src/github.com/tendermint/tendermint/p2p/peer.go:374 +0x25b
github.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine()
/go/src/github.com/tendermint/tendermint/p2p/conn/connection.go:599 +0xcce
Goroutine 183 (running) created at:
D[2019-02-27|10:10:49.058] Send switch=2 peer=1efafad5443abeea4b7a8155218e4369525d987e@127.0.0.1:46193 channel=48 conn=MConn{pipe} m
sgBytes=2B06579D0A146194480ADAE00C2836ED7125FEE65C1D9DD51049
github.com/tendermint/tendermint/mempool.(*MempoolReactor).AddPeer()
/go/src/github.com/tendermint/tendermint/mempool/reactor.go:105 +0x1b1
github.com/tendermint/tendermint/p2p.(*Switch).startInitPeer()
/go/src/github.com/tendermint/tendermint/p2p/switch.go:683 +0x13b
github.com/tendermint/tendermint/p2p.(*Switch).addPeer()
/go/src/github.com/tendermint/tendermint/p2p/switch.go:650 +0x585
github.com/tendermint/tendermint/p2p.(*Switch).addPeerWithConnection()
/go/src/github.com/tendermint/tendermint/p2p/test_util.go:145 +0x939
github.com/tendermint/tendermint/p2p.Connect2Switches.func2()
/go/src/github.com/tendermint/tendermint/p2p/test_util.go:109 +0x50
I[2019-02-27|10:10:49.058] Added good transaction validator=0 tx=43B4D1F0F03460BD262835C4AA560DB860CFBBE85BD02386D83DAC38C67B3AD7 res="&{CheckTx:gas_w
anted:1 }" height=0 total=375
Goroutine 190 (running) created at:
github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart()
/go/src/github.com/tendermint/tendermint/p2p/conn/connection.go:210 +0x313
github.com/tendermint/tendermint/libs/common.(*BaseService).Start()
/go/src/github.com/tendermint/tendermint/libs/common/service.go:139 +0x4df
github.com/tendermint/tendermint/p2p.(*peer).OnStart()
/go/src/github.com/tendermint/tendermint/p2p/peer.go:179 +0x56
github.com/tendermint/tendermint/libs/common.(*BaseService).Start()
/go/src/github.com/tendermint/tendermint/libs/common/service.go:139 +0x4df
github.com/tendermint/tendermint/p2p.(*peer).Start()
<autogenerated>:1 +0x43
github.com/tendermint/tendermint/p2p.(*Switch).startInitPeer()
```
* explain the choice of a map DS for senders
* extract ids pool/mapper to a separate struct
* fix literal copies lock value from senders: sync.Map contains sync.Mutex
* use sync.Map#LoadOrStore instead of Load
* fixes after Ismail's review
* rename resCbNormal to resCbFirstTime
commit 25a3c8b1724c9611d6edc175b1b0d079f5ee28c1
Author: zjubfd <baifudong@lancai.cn>
Date: Sun Mar 24 01:08:15 2019 +0800
rpc: support tls rpc (#3469)
Refs #3419
commit 85be2a554e7e7752bed0b9409ab153bf04e05e7b
Author: Thane Thomson <connect@thanethomson.com>
Date: Fri Mar 22 09:16:38 2019 -0400
tools/tm-signer-harness: update height and round for test harness (#3466)
In order to re-enable the test harness for the KMS (see
tendermint/kms#227), we need some marginally more realistic proposals
and votes. This is because the KMS does some additional sanity checks
now to ensure the height and round are increasing over time.
commit 1d4afb179b9660bf13705c67b01e20838a4506eb
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Mar 21 11:05:39 2019 +0100
replace PB2TM.ConsensusParams with a call to params#Update (#3448)
Fixes #3444
commit 660bd4a53e0dd7642a473689c8686c3f83e3a0ca
Author: tracebundy <745403419@qq.com>
Date: Wed Mar 20 20:30:49 2019 +0800
fix comment (#3454)
commit 81b9bdf40010c6a4e336133ec53fe6e4e6089911
Author: Ethan Buchman <ethan@coinculture.info>
Date: Wed Mar 20 08:29:40 2019 -0400
comments on validator ordering (#3452)
* comments on validator ordering
* NextValidatorsHash
commit 926127c774a2c9110c4284938411818918ffecac
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Wed Mar 20 03:59:33 2019 +0300
blockchain: update the maxHeight when a peer is removed (#3350)
* blockchain: update the maxHeight when a peer is removed
Refs #2699
* add a changelog entry
* make linter pass
commit 03085c2da23b179c4a51f59a03cb40aa4e85a613
Author: zjubfd <baifudong@lancai.cn>
Date: Wed Mar 20 08:18:18 2019 +0800
rpc: client disable compression (#3430)
commit 7af4b5086af9268f7cc8b41f5a174ade675d8ab4
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Wed Mar 20 03:10:54 2019 +0300
Remove RepeatTimer and refactor Switch#Broadcast (#3429)
* p2p: refactor Switch#Broadcast func
- call wg.Add only once
- do not call peers.List twice!
* bad for perfomance
* peers list can change in between calls!
Refs #3306
* p2p: use time.Ticker instead of RepeatTimer
no need in RepeatTimer since we don't Reset them
Refs #3306
* libs/common: remove RepeatTimer (also TimerMaker and Ticker interface)
"ancient code that’s caused no end of trouble" Ethan
I believe there's much simplier way to write a ticker than can be reset
https://medium.com/@arpith/resetting-a-ticker-in-go-63858a2c17ec
commit 60b2ae5f5a3e16625af1342e012462448d565394
Author: needkane <604476380@qq.com>
Date: Wed Mar 20 08:00:53 2019 +0800
crypto: delete unused code (#3426)
commit a6349f50633a45755041b8230a0f7eb990383cfd
Author: Anca Zamfir <ancazamfir@users.noreply.github.com>
Date: Wed Mar 20 01:56:13 2019 +0200
Formalize proposer election algorithm properties (#3140)
* Update proposer-selection.md
* Fixed typos
* fixed typos
* Attempt to address some comments
* Update proposer-selection.md
* Update proposer-selection.md
* Update proposer-selection.md
Added the normalization step.
* Addressed review comments
* New example for normalization section
Added a new example to better show the need for normalization
Added requirement for changing validator set
Addressed review comments
* Fixed problem with R2
* fixed the math for new validator
* test
* more small updates
* Moved the centering above the round-robin election
- the centering is now done before the actual round-robin block
- updated examples
- cleanup
* change to reflect new implementation for new validator
commit 22bcfca87a987dc43ad111a226b072ce09bbeb8d
Merge: 4162ebe8 0d985ede
Author: Ethan Buchman <ethan@coinculture.info>
Date: Tue Mar 19 19:54:09 2019 -0400
Merge pull request #3450 from tendermint/master
Merge master back to develop
commit 0d985ede28bd6937fa9d3613618e42cab6fc871c
Merge: 97681953 1e346978
Author: Ethan Buchman <ethan@coinculture.info>
Date: Tue Mar 19 19:53:37 2019 -0400
Merge pull request #3417 from tendermint/release/v0.31.0
Release/v0.31.0
commit 1e3469789dce5a034a21b6e48288f1809a102595
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Wed Mar 20 00:45:51 2019 +0100
Ensure WriteTimeout > TimeoutBroadcastTxCommit (#3443)
* Make sure config.TimeoutBroadcastTxCommit < rpcserver.WriteTimeout()
* remove redundant comment
* libs/rpc/http_server: move Read/WriteTimeout into Config
* increase defaults for read/write timeouts
Based on this article
https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration
* WriteTimeout should be larger than TimeoutBroadcastTxCommit
* set a deadline for subscribing to txs
* extract duration into const
* add two changelog entries
* Update CHANGELOG_PENDING.md
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* Update CHANGELOG_PENDING.md
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* 12 -> 10
* changelog
* changelog
commit 5f68fbae37e11b01f4d2e5c33b182401a7283e4a
Merge: 551b6322 e276f35f
Author: Ethan Buchman <ethan@coinculture.info>
Date: Tue Mar 19 19:25:26 2019 -0400
Merge pull request #3449 from tendermint/ismail/merge_develop_into_release/0.31.0
Merge develop into release/0.31.0
commit e276f35f86b3980e088a95aa70c96dbddcdf658b
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Tue Mar 19 14:36:42 2019 +0100
remove 3421 from changelog
commit 8e62a3d62a5b04a5a7126d00d75925c990aec9c6
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Tue Mar 19 12:19:02 2019 +0100
Add #3421 to changelog and reorder alphabetically
commit 48aaccab8f3cc17032220a01a6d58acde695bbde
Merge: 551b6322 4162ebe8
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Tue Mar 19 12:09:26 2019 +0100
Merge in develop and update CHANGELOG.md
commit 4162ebe8b586deccc0e7476d8abafb75138bfe58
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Tue Mar 19 11:38:32 2019 +0400
types: refactor PB2TM.ConsensusParams to take BlockTimeIota as an arg (#3442)
See https://github.com/tendermint/tendermint/pull/3403/files#r266208947
In #3403 we unexposed BlockTimeIota from the ABCI, but it's still part
of the ConsensusParams struct, so we have to remember to add it back
after calling PB2TM.ConsensusParams. Instead, PB2TM.ConsensusParams
should take it as an argument
Fixes #3432
commit 551b6322f5a74f578a8487001e805e4e1da6394d
Author: Ethan Buchman <ethan@coinculture.info>
Date: Sat Mar 16 19:24:12 2019 -0400
Update v0.31.0 release notes (#3434)
* changelog: fix formatting
* update release notes
* update changelog
* linkify
* update UPGRADING
commit 52c4e15eb25e8a3d6e77f90239710de3655d4ec6
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Thu Mar 14 16:07:06 2019 +0100
changelog: more review fixes/release/v0.31.0 (#3427)
* Update release summary
* Add pubsub config changes
* Add link to issue for pubsub changes
commit 5483ac6b0a2a75bc140487e8405392533a3f1687
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Thu Mar 14 12:17:49 2019 +0100
minor changes / fixes to release 0.31.0 (#3422)
* bump ABCIVersion due to renaming BlockSizeParams -> BlockParams
(https://github.com/tendermint/tendermint/pull/3417#discussion_r264974791)
* Move changelog on consensus params entry to breaking
* Add @melekes' suggestion for breaking change in pubsub into upgrading.md
* Add changelog entry for #3351
* Add changelog entry for #3358 & #3359
* Add changelog entry for #3397
* remove changelog entry for #3397 (was already released in 0.30.2)
* move 3351 to improvements
* Update changelog comment
commit 745713330736c5c751450245d88b8037cbee3aa6
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Mar 14 15:00:58 2019 +0400
grpcdb: close Iterator/ReverseIterator after use (#3424)
Fixes #3402
commit a59930a327065f2d091840586da6c35b6fd07472
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Wed Mar 13 16:09:05 2019 +0400
localnet: fix $LOG variable (#3423)
Fixes #3421
Before: it was creating a file named ${LOG:-tendermint.log} in .build/nodeX
After: it creates a file named tendermint.log
commit 85c023db88a8517587f0e8499b1e89f0eea1112d
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Tue Mar 12 20:07:26 2019 +0100
Prep release v0.31.0:
- update changelog, reset pending
- bump versions
- add external contributors (partly manually)
commit 4cbd36f341c8fee55a4ea386aee42a2c7fe9c67d
Merge: e42f833f 97681953
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Tue Mar 12 21:22:03 2019 +0400
Merge pull request #3415 from tendermint/master
Merge master back to develop (do not squash)
commit e42f833fd4ce6cb886dbf594dcb2f3af61918325
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Tue Mar 12 16:20:59 2019 +0400
Merge master back to develop (#3412)
* libs/db: close batch (#3397)
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close
Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842
* update changelog and bump version to 0.30.2
commit ad3e990c6a424d939b181a2a7c911eb4592c0c79
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 11 23:59:00 2019 +0400
fix GO_VERSION in installation scripts (#3411)
there is no such file https://storage.googleapis.com/golang/go1.12.0.linux-amd64.tar.gz
Fixes #3405
commit 676212fa8fdd1ddf29e9c352129c97da3c113769
Author: srmo <srm@schakalacka.org>
Date: Mon Mar 11 20:06:03 2019 +0100
cmd: make sure to have 'testnet' create the data directory for nonvals (#3409)
Fixes #3408
commit 303557203402f0a1b6db4a55cb2c6562765058b1
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 11 22:52:09 2019 +0400
cs: comment out log.Error to avoid TestReactorValidatorSetChanges timing out (#3401)
commit d741c7b4785c36cef4b0912900b6193db21d00e6
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 11 22:45:58 2019 +0400
limit number of /subscribe clients and queries per client (#3269)
* limit number of /subscribe clients and queries per client
Add the following config variables (under [rpc] section):
* max_subscription_clients
* max_subscriptions_per_client
* timeout_broadcast_tx_commit
Fixes #2826
new HTTPClient interface for subscriptions
finalize HTTPClient events interface
remove EventSubscriber
fix data race
```
WARNING: DATA RACE
Read at 0x00c000a36060 by goroutine 129:
github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1()
/go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0
Previous write at 0x00c000a36060 by goroutine 132:
github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
/go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0
github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
/go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
/go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
Goroutine 129 (running) created at:
github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
/go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7
github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
/go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
/go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
Goroutine 132 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:878 +0x659
github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync()
/go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
==================
```
lite client works (tested manually)
godoc comments
httpclient: do not close the out channel
use TimeoutBroadcastTxCommit
no timeout for unsubscribe
but 1s Local (5s HTTP) timeout for resubscribe
format code
change Subscribe#out cap to 1
and replace config vars with RPCConfig
TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout
rpc: Context as first parameter to all functions
reformat code
fixes after my own review
fixes after Ethan's review
add test stubs
fix config.toml
* fixes after manual testing
- rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes
Tendermint resources (pubsub)
- rpc: better error in Subscribe and BroadcastTxCommit
- HTTPClient: do not resubscribe if err = ErrAlreadySubscribed
* fixes after Ismail's review
* Update rpc/grpc/grpc_test.go
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
commit 15f621141dce76d992f6c8dcfbd4c522878b6108
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 11 22:21:17 2019 +0400
remove TimeIotaMs from ABCI consensus params (#3403)
Also
- init substructures to avoid panic in pb2tm.ConsensusParams
Before: if csp.Block is nil and we later try to access/write to it,
we'll panic.
After: if csp.Block is nil and we later try to access/write to it,
there'll be no panic.
commit dc359bd3a51c52803a6af820a36ee41796284e87
Author: Anca Zamfir <ancazamfir@users.noreply.github.com>
Date: Mon Mar 11 16:17:25 2019 +0200
types: remove check for priority order of existing validators (#3407)
When scaling and averaging is invoked, it is possible to have validators
with close priorities ending up with same priority. With the current code,
this makes it impossible to verify the priority orders before and after updates.
Fixes #3383
commit 976819537d199aa43c6ad55ec4e8958323322bb0
Merge: e0f89364 f996b10f
Author: Ethan Buchman <ethan@coinculture.info>
Date: Mon Mar 11 08:17:14 2019 -0400
Merge pull request #3399 from tendermint/release/v0.30.2
Release/v0.30.2
commit 100ff08de93ff1907bf810f584ec5bdc7a2a5260
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 11 15:31:53 2019 +0400
p2p: do not panic when filter times out (#3384)
Fixes #3369
commit f996b10f479d7c9a6d81cca5a02c47b29a52b3f3
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Sun Mar 10 13:06:34 2019 +0400
update changelog and bump version to 0.30.2
commit 36d7180ca216f0d7ff62851fa441de2b0371d699
Author: Yumin Xia <stumblecra@gmail.com>
Date: Sun Mar 10 00:46:32 2019 -0800
libs/db: close batch (#3397)
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close
Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842
commit b021f1e505482fb34f8a8e57cd86b171e4a57344
Author: Yumin Xia <stumblecra@gmail.com>
Date: Sun Mar 10 00:46:32 2019 -0800
libs/db: close batch (#3397)
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close
Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842
commit 90794260bcd23989e4bd8530d596e39d9509e7ca
Author: mircea-c <mircea@tendermint.com>
Date: Sat Mar 9 10:13:36 2019 -0500
circleci: removed complexity from docs deployment job (#3396)
commit b6a510a3e7cefca56af4bdb009413e3950c6d59e
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Fri Mar 8 09:46:09 2019 +0400
make ineffassign linter pass (#3386)
Refs #3262
This fixes two small bugs:
1) lite/dbprovider: return `ok` instead of true in parse* functions. It's weird that we're ignoring `ok` value before.
2) consensus/state: previously because of the shadowing we almost never output "Error with msg". Now we declare both `added` and `err` in the beginning of the function, so there's no shadowing.
commit e415c326f93e5fa473d83a0ea6fa495ad0ea87f3
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Fri Mar 8 06:40:59 2019 +0100
update golang.org/x/crypto (#3392)
Update Gopkg.lock via dep ensure --update golang.org/x/crypto
see #3391 (comment) (nothing to review here really).
commit 28e9e9e7145b357cbd7078f4c83dcf66c13d4f7d
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Thu Mar 7 17:03:57 2019 +0100
update levigo to 1.0.0 (#3389)
Although the version we were pinning to is from Nov. 2016 there were no substantial changes:
jmhodges/levigo@2b8c778 added go-modules support (no code changes)
jmhodges/levigo@853d788 added a badge to the readme
closes #3381
commit 3ebfa99f2c2403c17348e41190406892abf21d2a
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Mar 7 19:35:04 2019 +0400
do not pin repos without releases to exact revisions (#3382)
We're pinning repos without releases because it's very easy to upgrade all the dependencies by executing dep ensure --upgrade. Instead, we should just never run this command directly, only dep ensure --upgrade <some repo>. And we can defend that in PRs.
Refs #3374
The problem with pinning to exact revisions: people who import Tendermint as a library (e.g. abci/types) are stuck with these revisions even though the code they import may not even use them.
commit 91b488f9a541f5dd69f0bb2db4249356733a2ed3
Author: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Date: Thu Mar 7 22:02:13 2019 +0900
docs: fix the reverse of meaning in spec (#3387)
https://tools.ietf.org/html/rfc6962#section-2.1
"The largest power of two less than the number of items" is actually correct!
For n > 1, let k be the largest power of two smaller than n (i.e.,
k < n <= 2k).
commit f25d727035f4615cbb4b43452191fd83218fc1e1
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Mar 7 09:10:34 2019 +0400
make dupl linter pass (#3385)
Refs #3262
commit 411bc5e49fcc3dda866ae799ba2ceda0084f80d4
Author: Anca Zamfir <ancazamfir@users.noreply.github.com>
Date: Wed Mar 6 09:54:49 2019 +0100
types: followup after validator set changes (#3301)
* fix failure in TestProposerFrequency
* Add test to check priority order after updates
* Changed applyRemovals() and removed Remove()
Changed applyRemovals() similar to applyUpdates()
Removed function Remove()
Updated comments
* review comments
* simplify applyRemovals and add more comments
* small correction in comment
* Fix check in test
* Fix priority check for centering, address review comments
* fix assert for priority centering
* review comments
* review comments
* cleanup and review comments
added upper limit check for validator voting power
moved check for empty validator set earlier
moved panic on potential negative set length in verifyRemovals
added more tests
* review comments
commit 858875fbb8b94debc4973fdf5923ced6564d3c39
Author: Silas Davis <silas@monax.io>
Date: Wed Mar 6 08:22:35 2019 +0000
Copy secp256k1 code from go-ethereum to avoid GPL vendoring issues in (#3371)
downstream
Signed-off-by: Silas Davis <silas@monax.io>
commit 1eaa42cd25f2355f4b30440bddae435d056d1f44
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Tue Mar 5 08:08:52 2019 +0100
golang 1.12.0 (#3376)
- update docker image on circleci
- remove GOCACHE=off from Makefile (see:
https://tip.golang.org/doc/go1.12#gocache)
- update badge in readme
- update in scripts/install
- update Vagrantfile
- update in networks/remote/integration.sh
- tools/build/Makefile
commit 8c9df30e28a3b95efe537afe9930de2150999441
Author: Jack Zampolin <jack.zampolin@gmail.com>
Date: Mon Mar 4 22:56:46 2019 -0800
libs/db: Add cleveldb.Stats() (#3379)
Fixes: #3378
* Add stats to cleveldb implementation
* update changelog
* remote TODO
also
- sort keys
- preallocate memory
* fix const initializer []string literal is not a constant
* add test
commit 52771e1287eb44eabae99e6d2781a57c577821bb
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 4 13:24:44 2019 +0400
make BlockTimeIota a consensus parameter, not a locally configurable … (#3048)
* make BlockTimeIota a consensus parameter, not a locally configurable option
Refs #2920
* make TimeIota int64 ms
Refs #2920
* update Gopkg.toml
* fixes after Ethan's review
* fix TestRemoteSignerProposalSigningFailed
* update changelog
commit f39138aa2e543438548a150bdad45304ccc3296b
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 4 12:18:32 2019 +0400
remove RoundState from EventDataRoundState (#3354)
Before we're using it to get a round state in tests. Now it can be done
by calling csX.GetRoundState. We will need to rewrite
TestStateSlashingPrevotes and TestStateSlashingPrecommits, which are
commented right now, to not rely on EventDataRoundState#RoundState
field.
Refs #1527
commit 8a962ffc4639eed95501114e0436ccffc52f8ef1
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Mar 4 12:17:38 2019 +0400
deps: update gogo/protobuf from 1.1.1 to 1.2.1 and golang/protobuf from 1.1.0 to 1.3.0 (#3357)
* deps: update gogo/proto from 1.1.1 to 1.2.1
- verified changes manually
git diff 636bf030~ ba06b47c --stat -- ':!*.pb.go' ':!test'
* deps: update golang/protobuf from 1.1.0 to 1.3.0
- verified changes manually
git diff b4deda0~ c823c79 -- ':!*.pb.go' ':!test'
commit 3421e4dcd7c6937e9ae8f72af17b38840840a781
Author: zjubfd <baifudong@lancai.cn>
Date: Sun Mar 3 05:36:52 2019 +0800
make lightd availbe (#3364)
1."abci_query": rpcserver.NewRPCFunc(c.ABCIQuery, "path,data,prove")
"validators": rpcserver.NewRPCFunc(c.Validators, "height"),
the parameters and function do not match, cause index out of range error.
2. the prove of query is forced to be true, while default option is false.
3. fix the wrong key of merkle
commit 976b1c2ef73e557a702ef17600e1eeaf4865f292
Author: zjubfd <baifudong@lancai.cn>
Date: Sun Mar 3 04:21:21 2019 +0800
fix pool timer leak bug, resolve#3353 (#3358)
When remove peer, block pool simple remove bpPeer,
but do not stop timer, that cause stopError for recorrected
peers. Stop timer when remove from pool.
commit d95894152beb3b25423cdb109af006340f0ac80b
Author: zjubfd <baifudong@lancai.cn>
Date: Sun Mar 3 04:17:37 2019 +0800
fix dirty data in peerset,resolve #3304 (#3359)
* fix dirty data in peerset
startInitPeer before PeerSet add the peer,
once mconnection start and Receive of one
Reactor faild, will try to remove it from PeerSet
while PeerSet still not contain the peer. Fix
this by change the order.
* fix test FilterDuplicate
* fix start/stop race
* fix err
commit 37a548414bbb5dc8dce9fe452ba5f933fad29684
Author: Sven-Hendrik Haase <svenstaro@gmail.com>
Date: Sat Mar 2 07:06:57 2019 +0100
docs: fix typo (#3373)
commit 853dd34d31bee6de593be94df87cbb508688249e
Author: Juan Leni <juan.leni@zondax.ch>
Date: Thu Feb 28 08:48:20 2019 +0100
privval: improve Remote Signer implementation (#3351)
This issue is related to #3107
This is a first renaming/refactoring step before reworking and removing heartbeats.
As discussed with @Liamsi , we preferred to go for a couple of independent and separate PRs to simplify review work.
The changes:
Help to clarify the relation between the validator and remote signer endpoints
Differentiate between timeouts and deadlines
Prepare to encapsulate networking related code behind RemoteSigner in the next PR
My intention is to separate and encapsulate the "network related" code from the actual signer.
SignerRemote ---(uses/contains)--> SignerValidatorEndpoint <--(connects to)--> SignerServiceEndpoint ---> SignerService (future.. not here yet but would like to decouple too)
All reconnection/heartbeat/whatever code goes in the endpoints. Signer[Remote/Service] do not need to know about that.
I agree Endpoint may not be the perfect name. I tried to find something "Go-ish" enough. It is a common name in go-kit, kubernetes, etc.
Right now:
SignerValidatorEndpoint:
handles the listener
contains SignerRemote
Implements the PrivValidator interface
connects and sets a connection object in a contained SignerRemote
delegates PrivValidator some calls to SignerRemote which in turn uses the conn object that was set externally
SignerRemote:
Implements the PrivValidator interface
read/writes from a connection object directly
handles heartbeats
SignerServiceEndpoint:
Does most things in a single place
delegates to a PrivValidator IIRC.
* cleanup
* Refactoring step 1
* Refactoring step 2
* move messages to another file
* mark for future work / next steps
* mark deprecated classes in docs
* Fix linter problems
* additional linter fixes
commit d6e2fb453d20841828cfbe1f6a7e8ca593c1b564
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Feb 28 11:31:59 2019 +0400
update docs (#3349)
* docs: explain create_empty_blocks configurations
Closes #3307
* Vagrantfile: install nodejs for docs
* update docs instructions
npm install does not make sense since there's no packages.json file
* explain broadcast_tx_* tx format
Closes #536
* docs: explain how transaction ordering works
Closes #2904
* bring in consensus parameters explained
* example for create_empty_blocks_interval
* bring in explanation from https://github.com/tendermint/tendermint/issues/2487#issuecomment-424899799
* link to formatting instead of duplicating info
commit ec9bff52346d9629f690147674326a41fcefd20f
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Feb 25 09:11:07 2019 +0400
rename WAL#Flush to WAL#FlushAndSync (#3345)
* rename WAL#Flush to WAL#FlushAndSync
- rename auto#Flush to auto#FlushAndSync
- cleanup WAL interface to not leak implementation details!
* remove Group()
* add WALReader interface and return it in SearchForEndHeight()
- add interface assertions
Refs #3337
* replace WALReader with io.ReadCloser
commit 6797d8585167a9dc1e51843420ae4ab7eaf848a0
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Mon Feb 25 06:06:21 2019 +0100
p2p: fix comment in secret connection (#3348)
Just a minor followup on the review if #3347: Fixes a comment. [#3347 (comment)](https://github.com/tendermint/tendermint/pull/3347#discussion_r259582330)
commit cdf3a74f4899b3c716703b932e682307752cd6b6
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Sat Feb 23 19:48:28 2019 +0400
Unclean shutdown on SIGINT / SIGTERM (#3308)
* libs/common: TrapSignal accepts logger as a first parameter
and does not block anymore
* previously it was dumping "captured ..." msg to os.Stdout
* TrapSignal should not be responsible for blocking thread of execution
Refs #3238
* exit with zero (0) code upon receiving SIGTERM/SIGINT
Refs #3238
* fix formatting in docs/app-dev/abci-cli.md
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* fix formatting in docs/app-dev/abci-cli.md
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
commit 41f91318e9e253a1b59cc1cc128ad412941b8e2a
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Sat Feb 23 19:32:31 2019 +0400
bound mempool memory usage (#3248)
* bound mempool memory usage
Closes #3079
* rename SizeBytes to TxsTotalBytes
and other small fixes after Zarko's review
* rename MaxBytes to MaxTxsTotalBytes
* make ErrMempoolIsFull more informative
* expose mempool's txs_total_bytes via RPC
* test full response
* fixes after Ethan's review
* config: rename mempool.size to mempool.max_txs
https://github.com/tendermint/tendermint/pull/3248#discussion_r254034004
* test more cases
https://github.com/tendermint/tendermint/pull/3248#discussion_r254036532
* simplify test
* Revert "config: rename mempool.size to mempool.max_txs"
This reverts commit 39bfa3696177aa46195000b90655419a975d6ff7.
* rename count back to n_txs
to make a change non-breaking
* rename max_txs_total_bytes to max_txs_bytes
* format code
* fix TestWALPeriodicSync
The test was sometimes failing due to processFlushTicks being called too
early. The solution is to call wal#Start later in the test.
* Apply suggestions from code review
commit e0adc5e807c8c2ea4ac03f8eeeb006b1a154d453
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Sat Feb 23 16:25:57 2019 +0100
secret connection check all zeroes (#3347)
* reject the shared secret if is all zeros in case the blacklist was not
sufficient
* Add test that verifies lower order pub-keys are rejected at the DH step
* Update changelog
* fix typo in test-comment
commit 2137ecc130874118cc23995befed370958c34448
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Sat Feb 23 08:11:27 2019 +0400
pubsub 2.0 (#3227)
* green pubsub tests :OK:
* get rid of clientToQueryMap
* Subscribe and SubscribeUnbuffered
* start adapting other pkgs to new pubsub
* nope
* rename MsgAndTags to Message
* remove TagMap
it does not bring any additional benefits
* bring back EventSubscriber
* fix test
* fix data race in TestStartNextHeightCorrectly
```
Write at 0x00c0001c7418 by goroutine 796:
github.com/tendermint/tendermint/consensus.TestStartNextHeightCorrectly()
/go/src/github.com/tendermint/tendermint/consensus/state_test.go:1296 +0xad
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
Previous read at 0x00c0001c7418 by goroutine 858:
github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1631 +0x1366
github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
/go/src/github.com/tendermint/tendermint/consensus/state.go:1476 +0x8f
github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
/go/src/github.com/tendermint/tendermint/consensus/state.go:667 +0xa1e
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
/go/src/github.com/tendermint/tendermint/consensus/state.go:628 +0x794
Goroutine 796 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:878 +0x659
testing.runTests.func1()
/usr/local/go/src/testing/testing.go:1119 +0xa8
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
testing.runTests()
/usr/local/go/src/testing/testing.go:1117 +0x4ee
testing.(*M).Run()
/usr/local/go/src/testing/testing.go:1034 +0x2ee
main.main()
_testmain.go:214 +0x332
Goroutine 858 (running) created at:
github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
/go/src/github.com/tendermint/tendermint/consensus/state.go:334 +0x221
github.com/tendermint/tendermint/consensus.startTestRound()
/go/src/github.com/tendermint/tendermint/consensus/common_test.go:122 +0x63
github.com/tendermint/tendermint/consensus.TestStateFullRound1()
/go/src/github.com/tendermint/tendermint/consensus/state_test.go:255 +0x397
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162
```
* fixes after my own review
* fix formatting
* wait 100ms before kicking a subscriber out
+ a test for indexer_service
* fixes after my second review
* no timeout
* add changelog entries
* fix merge conflicts
* fix typos after Thane's review
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* reformat code
* rewrite indexer service in the attempt to fix failing test
https://github.com/tendermint/tendermint/pull/3227/#issuecomment-462316527
* Revert "rewrite indexer service in the attempt to fix failing test"
This reverts commit 0d9107a098230de7138abb1c201877c246e89ed1.
* another attempt to fix indexer
* fixes after Ethan's review
* use unbuffered channel when indexing transactions
Refs https://github.com/tendermint/tendermint/pull/3227#discussion_r258786716
* add a comment for EventBus#SubscribeUnbuffered
* format code
commit 67fd42835448172d08616589f1473e7153cc0902
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Fri Feb 22 11:49:08 2019 +0400
fix TestWALPeriodicSync (#3342)
The test was sometimes failing due to processFlushTicks being called too
early. The solution is to call wal#Start later in the test.
commit f22ada442a249597dff765a59197bc57a8eae349
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Fri Feb 22 11:48:40 2019 +0400
refactor decideProposal in common_test (#3343)
commit ed1de135486608d8b4869760960e3eb35ade2c6d
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Thu Feb 21 18:28:02 2019 +0400
cs: update wal comments (#3334)
* cs: update wal comments
Follow-up to https://github.com/tendermint/tendermint/pull/3300
* Update consensus/wal.go
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
commit 4f83eec78251b9228f0edb76f261e3e387d21c12
Merge: db5d7602 e0f89364
Author: Ethan Buchman <ethan@coinculture.info>
Date: Wed Feb 20 10:11:33 2019 -0500
Merge pull request #3339 from tendermint/master
Merge master back to develop
commit e0f8936455029a40287a69d5b0e7baa4d5864da1
Merge: 28d75ec8 f2351dc7
Author: Ethan Buchman <ethan@coinculture.info>
Date: Wed Feb 20 10:10:49 2019 -0500
Merge pull request #3326 from tendermint/release/v0.30.1
Release/v0.30.1
commit f2351dc758e5c8cf6bc6c5c32931daa8ae6e49a9
Author: Ethan Buchman <ethan@coinculture.info>
Date: Wed Feb 20 10:05:57 2019 -0500
update changelog
commit db5d7602fea6a3418d4ae83511c1a35cf9e1c9ca
Author: Daniil Lashin <danil-lashin@yandex.ru>
Date: Wed Feb 20 14:34:52 2019 +0300
docs: fix rpc Tx() method docs (#3331)
commit dff3deb2a95474ee1a10fc6aa7fd0a0a9a6aee94
Author: Thane Thomson <connect@thanethomson.com>
Date: Wed Feb 20 07:45:18 2019 +0200
cs: sync WAL more frequently (#3300)
As per #3043, this adds a ticker to sync the WAL every 2s while the WAL is running.
* Flush WAL every 2s
This adds a ticker that flushes the WAL every 2s while the WAL is
running. This is related to #3043.
* Fix spelling
* Increase timeout to 2mins for slower build environments
* Make WAL sync interval configurable
* Add TODO to replace testChan with more comprehensive testBus
* Remove extraneous debug statement
* Remove testChan in favour of using system time
As per
https://github.com/tendermint/tendermint/pull/3300#discussion_r255886586,
this removes the `testChan` WAL member and replaces the approach with a
system time-oriented one. In this new approach, we keep track of the
system time at which each flush and periodic flush successfully
occurred.
The naming of the various functions is also updated here to be more
consistent with "flushing" as opposed to "sync'ing".
* Update naming convention and ensure lock for timestamp update
* Add Flush method as part of WAL interface
Adds a `Flush` method as part of the WAL interface to enforce the idea
that we can manually trigger a WAL flush from outside of the WAL. This
is employed in the consensus state management to flush the WAL prior to
signing votes/proposals, as per https://github.com/tendermint/tendermint/issues/3043#issuecomment-453853630
* Update CHANGELOG_PENDING
* Remove mutex approach and replace with DI
The dependency injection approach to dealing with testing concerns could
allow similar effects to some kind of "testing bus"-based approach. This
commit introduces an example of this, where instead of relying on
(potentially fragile) timing of things between the code and the test, we
inject code into the function under test that can signal the test
through a channel.
This allows us to avoid the `time.Sleep()`-based approach previously
employed.
* Update comment on WAL flushing during vote signing
Co-Authored-By: thanethomson <connect@thanethomson.com>
* Simplify flush interval definition
Co-Authored-By: thanethomson <connect@thanethomson.com>
* Expand commentary on WAL disk flushing
Co-Authored-By: thanethomson <connect@thanethomson.com>
* Add broken test to illustrate WAL sync test problem
Removes test-related state (dependency injection code) from the WAL data
structure and adds test code to illustrate the problem with using
`WALGenerateNBlocks` and `wal.SearchForEndHeight` to test periodic
sync'ing.
* Fix test error messages
* Use WAL group buffer size to check for flush
A function is added to `libs/autofile/group.go#Group` in order to return
the size of the buffered data (i.e. data that has not yet been flushed
to disk). The test now checks that, prior to a `time.Sleep`, the group
buffer has data in it. After the `time.Sleep` (during which time the
periodic flush should have been called), the buffer should be empty.
* Remove config root dir removal from #3291
* Add godoc for NewWAL mentioning periodic sync
commit 9d4f59b8366ad393410051e09672b76578545d9e
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Feb 18 15:27:07 2019 +0400
update changelog and bump version
commit d2c7f8dbcf238ccc3a2469e350523c7ed28be154
Author: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Date: Mon Feb 18 11:08:22 2019 +0100
p2p: check secret conn id matches dialed id (#3321)
ref: [#3010 (comment)](https://github.com/tendermint/tendermint/issues/3010#issuecomment-464287627)
> I tried searching for code where we authenticate a peer against its NetAddress.ID and couldn't find it. I don't see a reason to switch to Noise, but a need to ensure that the node's ID is authenticated e.g. after dialing from the address book.
* p2p: check secret conn id matches dialed id
* Fix all p2p tests & make code compile
* add simple test for dialing with wrong ID
* update changelog
* address review comments
* yet another place where to use IDAddressString and fix
testSetupMultiplexTransport
commit 8283ca7ddb5b56090aaa15e005b40782fa618ae7
Author: Anton Kaliaev <anton.kalyaev@gmail.com>
Date: Mon Feb 18 13:23:40 2019 +0400
3291 follow-up (#3323)
* changelog: use issue number instead of PR number
* follow up to #3291
- rpc/test/helpers.go add StopTendermint(node) func
- remove ensureDir(filepath.Dir(walFile), 0700)
- mempool/mempool_test.go add type cleanupFunc func()
* cmd/show_validator: wrap err to make it more clear
commit 59cc6d36c944012cd4ae39fbee698e8cf353005a
Author: Alessio Treglia <quadrispro@ubuntu.com>
Date: Mon Feb 18 08:45:27 2019 +0100
improve ResetTestRootWithChainID() concurrency safety (#3291)
* improve ResetTestRootWithChainID() concurrency safety
Rely on ioutil.TempDir() to create test root directories and ensure
multiple same-chain id test cases can run in parallel.
* Update config/toml.go
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* clean up test directories after completion
Closes: #1034
* Remove redundant EnsureDir call
* s/PanicSafety()/panic()/s
* Put create dir functionality back in ResetTestRootWithChainID
* Place test directories in OS's tempdir
In modern UNIX and UNIX-like systems /tmp is very often
mounted as tmpfs. This might speed test execution a bit.
* Set 0700 to a const
* rootsDirs -> configRootDirs
* Don't double remove directories
* Avoid global variables
* Fix consensus tests
* Reduce defer stack
* Address review comments
* Try to fix tests
* Update CHANGELOG_PENDING.md
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* Update consensus/common_test.go
Co-Authored-By: alessio <quadrispro@ubuntu.com>
* Update consensus/common_test.go
Co-Authored-By: alessio <quadrispro@ubuntu.com>
commit af8793c01a168f0f15d3147673c572c0f8bf7863
Author: Zarko Milosevic <zarko@interchain.io>
Date: Mon Feb 18 08:29:41 2019 +0100
cs: reset triggered timeout precommit (#3310)
* Reset TriggeredTimeoutPrecommit as part of updateToState
* Add failing test and fix
* fix DATA RACE in TestResetTimeoutPrecommitUponNewHeight
```
WARNING: DATA RACE
Read at 0x00c001691d28 by goroutine 691:
github.com/tendermint/tendermint/consensus.decideProposal()
/go/src/github.com/tendermint/tendermint/consensus/common_test.go:133 +0x121
github.com/tendermint/tendermint/consensus.TestResetTimeoutPrecommitUponNewHeight()
/go/src/github.com/tendermint/tendermint/consensus/state_test.go:1389 +0x958
testing.tRunner()
…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rely on ioutil.TempDir() to create test root directories and ensure
multiple same-chain id test cases can run in parallel.