Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

After build gaiad-5001 i cant sync the new node #34

Closed
iammelea opened this issue May 24, 2018 · 3 comments
Closed

After build gaiad-5001 i cant sync the new node #34

iammelea opened this issue May 24, 2018 · 3 comments

Comments

@iammelea
Copy link
Contributor

iammelea commented May 24, 2018 •
edited
Loading

I try to build 3 times, and in 3 machines the bad sync.
Ubuntu 16.04 disk SSD

gaiad version
0.17.0-20abeb3

The first time this appears in the block 341 or so and start bad sync

I[05-24|01:48:53.229] Evidence is not valid module=evidence evidence="[VoteA: Vote{25:C59779BF393E 107476/00/2(Precommit) 2CBE63AEA732 /1A278DA9296D.../ @ 2018-05-23T18:51:14.579Z}; VoteB: Vote{25:C59779BF393E 107476/00/2(Precommit) 000000000000 /0F6C5F926C96.../ @ 2018-05-23T18:51:17.089Z} VoteA: Vote{25:C59779BF393E 108591/00/2(Precommit) B776B6166752 /9BA8F5F08044.../ @ 2018-05-23T20:19:19.568Z}; VoteB: Vote{25:C59779BF393E 108591/00/2(Precommit) 000000000000 /22A07FFF3444.../ @ 2018-05-23T20:19:22.487Z} VoteA: Vote{25:C59779BF393E 108601/01/2(Precommit) 6F410AD98DB6 /5C7711852686.../ @ 2018-05-23T20:20:11.493Z}; VoteB: Vote{25:C59779BF393E 108601/01/2(Precommit) 000000000000 /E1AB07FA4651.../ @ 2018-05-23T20:20:12.499Z} VoteA: Vote{25:C59779BF393E 109574/00/2(Precommit) 6F69FE39A6E5 /1BD3A92DC6BE.../ @ 2018-05-23T21:31:55.156Z}; VoteB: Vote{25:C59779BF393E 109574/00/2(Precommit) 000000000000 /118D8C7A5765.../ @ 2018-05-23T21:31:57.435Z} VoteA: Vote{25:C59779BF393E 109591/00/2(Precommit) 76A10486E80E /F4196A8340BF.../ @ 2018-05-23T21:33:11.215Z}; VoteB: Vote{25:C59779BF393E 109591/00/2(Precommit) 000000000000 /9B1EBA17DA67.../ @ 2018-05-23T21:33:13.902Z} VoteA: Vote{25:C59779BF393E 109723/00/2(Precommit) 76216ADED7C6 /6BEB8BA7A030.../ @ 2018-05-23T21:42:47.990Z}; VoteB: Vote{25:C59779BF393E 109723/00/2(Precommit) 000000000000 /8BF7E1FAF98D.../ @ 2018-05-23T21:42:50.590Z} VoteA: Vote{25:C59779BF393E 109947/00/2(Precommit) 58BF0B3A2159 /E24CA71C7938.../ @ 2018-05-23T22:00:47.547Z}; VoteB: Vote{25:C59779BF393E 109947/00/2(Precommit) 000000000000 /DD31AF715B15.../ @ 2018-05-23T22:00:49.224Z} VoteA: Vote{25:C59779BF393E 110042/00/2(Precommit) 4D3A789FBB70 /31F8BF402CC9.../ @ 2018-05-23T22:08:45.926Z}; VoteB: Vote{25:C59779BF393E 110042/00/2(Precommit) 000000000000 /594A6A18DC8B.../ @ 2018-05-23T22:08:48.121Z} VoteA: Vote{25:C59779BF393E 110083/00/2(Precommit) 201AE1C8C3B6 /B7D7851D096C.../ @ 2018-05-23T22:12:04.295Z}; VoteB: Vote{25:C59779BF393E 110083/00/2(Precommit) 000000000000 /4D845DF22474.../ @ 2018-05-23T22:12:07.101Z} VoteA: Vote{25:C59779BF393E 111003/00/1(Prevote) 76569E269401 /F130065D284D.../ @ 2018-05-23T23:27:47.973Z}; VoteB: Vote{25:C59779BF393E 111003/00/1(Prevote) E03F3B5F87D9 /B512B38E4F28.../ @ 2018-05-23T23:27:50.672Z} VoteA: Vote{25:C59779BF393E 111032/00/2(Precommit) B42554A70463 /ED69D63A9024.../ @ 2018-05-23T23:30:12.747Z}; VoteB: Vote{25:C59779BF393E 111032/00/2(Precommit) 000000000000 /3FD4FFCF4153.../ @ 2018-05-23T23:30:15.270Z} VoteA: Vote{25:C59779BF393E 111045/00/2(Precommit) 05FF655CFC1F /CFA05458F4A1.../ @ 2018-05-23T23:31:10.340Z}; VoteB: Vote{25:C59779BF393E 111045/00/2(Precommit) 000000000000 /EE3C5E2BB15F.../ @ 2018-05-23T23:31:12.724Z} VoteA: Vote{25:C59779BF393E 111491/00/2(Precommit) 277756E99441 /E0FDF74BFEBE.../ @ 2018-05-24T00:08:12.605Z}; VoteB: Vote{25:C59779BF393E 111491/00/2(Precommit) 000000000000 /D32BA81F0B85.../ @ 2018-05-24T00:08:14.646Z} VoteA: Vote{25:C59779BF393E 111493/00/2(Precommit) 6DDE42FF282A /DF767CF23148.../ @ 2018-05-24T00:08:19.678Z}; VoteB: Vote{25:C59779BF393E 111493/00/2(Precommit) 000000000000 /151EA8D712D1.../ @ 2018-05-24T00:08:22.376Z} VoteA: Vote{25:C59779BF393E 111846/00/2(Precommit) A196849789FB /B7E6317519DA.../ @ 2018-05-24T00:35:39.031Z}; VoteB: Vote{25:C59779BF393E 111846/00/2(Precommit) 000000000000 /B33F2C8FA63D.../ @ 2018-05-24T00:35:40.792Z} VoteA: Vote{25:C59779BF393E 111961/02/2(Precommit) 3BFE85C5E1FB /9D7C46E8E2B0.../ @ 2018-05-24T00:44:16.814Z}; VoteB: Vote{25:C59779BF393E 111961/02/2(Precommit) 000000000000 /9535DDA8B225.../ @ 2018-05-24T00:44:18.100Z} VoteA: Vote{25:C59779BF393E 112013/02/2(Precommit) D7ACD4941F33 /F5E0C43F34A7.../ @ 2018-05-24T00:48:15.674Z}; VoteB: Vote{25:C59779BF393E 112013/02/2(Precommit) 000000000000 /9E98B71CDBC7.../ @ 2018-05-24T00:48:17.507Z} VoteA: Vote{25:C59779BF393E 112477/00/2(Precommit) 8294ECF4D285 /5BC285E09B80.../ @ 2018-05-24T01:23:50.384Z}; VoteB: Vote{25:C59779BF393E 112477/00/2(Precommit) 000000000000 /AE08F510374C.../ @ 2018-05-24T01:23:52.902Z} VoteA: Vote{25:C59779BF393E 112549/00/2(Precommit) D15564E799A1 /4458947BF2AC.../ @ 2018-05-24T01:29:11.898Z}; VoteB: Vote{25:C59779BF393E 112549/00/2(Precommit) 000000000000 /6DAC6AA7CBCD.../ @ 2018-05-24T01:29:13.998Z} VoteA: Vote{25:C59779BF393E 112589/00/2(Precommit) B26EEB51E3E6 /A94DC43F4071.../ @ 2018-05-24T01:32:15.977Z}; VoteB: Vote{25:C59779BF393E 112589/00/2(Precommit) 000000000000 /1CA0311B4EE7.../ @ 2018-05-24T01:32:17.816Z} VoteA: Vote{25:C59779BF393E 112601/00/2(Precommit) 4561C7EB47B9 /9A06E6793207.../ @ 2018-05-24T01:33:11.280Z}; VoteB: Vote{25:C59779BF393E 112601/00/2(Precommit) 000000000000 /463B77FCDE73.../ @ 2018-05-24T01:33:12.843Z} VoteA: Vote{25:C59779BF393E 112635/00/2(Precommit) D9EC0885D2E6 /DB964ED67AFE.../ @ 2018-05-24T01:35:49.436Z}; VoteB: Vote{25:C59779BF393E 112635/00/2(Precommit) 000000000000 /9EDAE34B010C.../ @ 2018-05-24T01:35:51.702Z} VoteA: Vote{25:C59779BF393E 112641/00/2(Precommit) 1E760B8126E1 /E221A8901FD4.../ @ 2018-05-24T01:36:11.264Z}; VoteB: Vote{25:C59779BF393E 112641/00/2(Precommit) 000000000000 /416B91DEE52D.../ @ 2018-05-24T01:36:13.501Z} VoteA: Vote{25:C59779BF393E 112677/00/2(Precommit) B11611639695 /4A98FD245F22.../ @ 2018-05-24T01:38:59.073Z}; VoteB: Vote{25:C59779BF393E 112677/00/2(Precommit) 000000000000 /088F7B3D9467.../ @ 2018-05-24T01:39:01.495Z}]" err="Could not find validator set for height #112677"

after this red lines

E[05-24|13:42:06.381] Stopping peer for error module=p2p peer="Peer{MConn{138.197.187.76:46656} 6e2b7984aa49cfe8efa63f90ec0c878fe58f9028 out}" err="Could not find validator set for height #123965"

and red lines

E[05-24|13:42:30.917] Dialing failed module=p2p addr=3519f05985394107e0b2e285361b7e012adb1113@54.209.118.64:46656 err="Error{Error creating peer (cause: dial tcp 54.209.118.64:46656: i/o timeout)}" attempts=6
E[05-24|13:42:30.917] Dialing failed module=p2p addr=a67e0ec6f46b7e50d09a12ee38cc110aa408ac74@54.37.235.164:46656 err="Error{Error creating peer (cause: dial tcp 54.37.235.164:46656: i/o timeout)}" attempts=2
E[05-24|13:42:30.918] Dialing failed module=p2p addr=f654b04ec94e7fb8e9262ecd769ef689bdad6e57@193.70.41.36:46656 err="Error{Error creating peer (cause: dial tcp 193.70.41.36:46656: i/o timeout)}" attempts=2

and

E[05-24|13:42:52.431] Stopping peer for error module=p2p peer="Peer{MConn{45.77.53.208:46656} aef085c4bfed0c1ffc6705f2e1e3bf85e5164600 out}" err="Could not find validator set for height #123965"
I[05-24|13:42:57.748] Ensure peers module=p2p numOutPeers=2 numInPeers=3 numDialing=0 numToDial=8
I[05-24|13:42:57.749] Will dial address module=p2p addr=5baa2e6b95ef62e47d09abeb1277c0c798b1a069@207.154.227.84:46656
I[05-24|13:42:57.750] Will dial address module=p2p addr=29147732ea935a326dc2e366b0e4a23ef1d1bac8@163.172.47.92:46656
I[05-24|13:42:57.750] Will dial address module=p2p addr=10fcf8e293e83ef9ea45ac53311d4716d4acce74@96.230.239.222:46656
I[05-24|13:42:57.750] Will dial address module=p2p addr=a6242ea2092d20d70e95a225acdd87d19c2569e8@206.189.62.166:46656
I[05-24|13:42:57.750] Will dial address module=p2p addr=b6021a815f31e7a3228e0c5404e132b40c975232@194.102.32.89:46656
I[05-24|13:42:57.751] Will dial address module=p2p addr=75f013981d13fe6a8d8caba77eebe6746aff5db8@46.101.203.248:46656
I[05-24|13:42:57.751] Will dial address module=p2p addr=b70f3ca576591d84dc49f442399c856340be603e@31.187.71.49:46656
I[05-24|13:42:57.751] Will dial address module=p2p addr=3d0113930b91cf7f28c412ced211c69657070a2c@54.37.235.164:46656
I[05-24|13:42:57.752] We need more addresses. Sending pexRequest to random peer module=p2p peer="Peer{MConn{108.48.116.174:64529} 3fb1d71fc907717e2573fcf4b20235545ee570a1 in}"
I[05-24|13:42:57.752] Dialing peer module=p2p address=3d0113930b91cf7f28c412ced211c69657070a2c@54.37.235.164:46656
I[05-24|13:42:57.752] Dialing peer module=p2p address=5baa2e6b95ef62e47d09abeb1277c0c798b1a069@207.154.227.84:46656
I[05-24|13:42:57.752] Dialing peer module=p2p address=29147732ea935a326dc2e366b0e4a23ef1d1bac8@163.172.47.92:46656
I[05-24|13:42:57.752] Dialing peer module=p2p address=10fcf8e293e83ef9ea45ac53311d4716d4acce74@96.230.239.222:46656
I[05-24|13:42:57.752] Dialing peer module=p2p address=a6242ea2092d20d70e95a225acdd87d19c2569e8@206.189.62.166:46656
I[05-24|13:42:57.752] Dialing peer module=p2p address=b6021a815f31e7a3228e0c5404e132b40c975232@194.102.32.89:46656
I[05-24|13:42:57.752] Dialing peer module=p2p address=75f013981d13fe6a8d8caba77eebe6746aff5db8@46.101.203.248:46656
I[05-24|13:42:57.752] Dialing peer module=p2p address=b70f3ca576591d84dc49f442399c856340be603e@31.187.71.49:46656
E[05-24|13:42:57.833] Dialing failed module=p2p addr=b70f3ca576591d84dc49f442399c856340be603e@31.187.71.49:46656 err="Error{Error creating peer (cause: dial tcp 31.187.71.49:46656: connect: connection refused)}" attempts=4
E[05-24|13:42:57.841] Dialing failed module=p2p addr=29147732ea935a326dc2e366b0e4a23ef1d1bac8@163.172.47.92:46656 err="Error{Error creating peer (cause: dial tcp 163.172.47.92:46656: connect: connection refused)}" attempts=3
E[05-24|13:42:57.843] Dialing failed module=p2p addr=5baa2e6b95ef62e47d09abeb1277c0c798b1a069@207.154.227.84:46656 err="Error{Error creating peer (cause: dial tcp 207.154.227.84:46656: connect: connection refused)}" attempts=2
E[05-24|13:42:57.846] Dialing failed module=p2p addr=a6242ea2092d20d70e95a225acdd87d19c2569e8@206.189.62.166:46656 err="Error{Error creating peer (cause: dial tcp 206.189.62.166:46656: connect: connection refused)}" attempts=3
E[05-24|13:42:57.890] Dialing failed module=p2p addr=b6021a815f31e7a3228e0c5404e132b40c975232@194.102.32.89:46656 err="Error{Error creating peer (cause: dial tcp 194.102.32.89:46656: connect: connection refused)}" attempts=0
E[05-24|13:43:00.753] Dialing failed module=p2p addr=3d0113930b91cf7f28c412ced211c69657070a2c@54.37.235.164:46656 err="Error{Error creating peer (cause: dial tcp 54.37.235.164:46656: i/o timeout)}" attempts=3
E[05-24|13:43:00.756] Dialing failed module=p2p addr=10fcf8e293e83ef9ea45ac53311d4716d4acce74@96.230.239.222:46656 err="Error{Error creating peer (cause: dial tcp 96.230.239.222:46656: i/o timeout)}" attempts=3
E[05-24|13:43:00.757] Dialing failed module=p2p addr=75f013981d13fe6a8d8caba77eebe6746aff5db8@46.101.203.248:46656 err="Error{Error creating peer (cause: dial tcp 46.101.203.248:46656: i/o timeout)}" attempts=2

this repeat and repeat and the node is working but in the same buckle and stop in the same block for long time

gaiacli status
{"node_info":{"id":"44631a3e7518e972ad4bb6375fa098796a2e3dbb","listen_addr":"142.44.240.204:46656","network":"gaia-5001","version":"0.19.3","channels":"4020212223303800","moniker":"trust melea #3","other":["amino_version=0.9.9","p2p_version=0.5.0","consensus_version=v1/0.2.2","rpc_version=0.7.0/3","tx_index=on","rpc_addr=tcp://0.0.0.0:46657"]},"sync_info":{"latest_block_hash":"D0F59AD17E87E891C2BAEC7E73DA37EFF27C91C3","latest_app_hash":"2F09A979AD22E6B39CD0113244032C5E109CAACB","latest_block_height":1362,"latest_block_time":"2018-05-15T16:10:20.316525749-04:00","syncing":true},"validator_info":{"address":"009485AED0333CF552E4560E20C2DB5BF8B59852","pub_key":{"type":"AC26791624DE60","value":"DH8h0D5zVWurEZPNo/6p4SHih4HEujh2zF3NNtx6zjE="},"voting_power":0}}

if i stop the node and start again, the node up some blocks and start the same error and red lines and don't catch more new blocks.

after gaiad unsafe_reset_all

neither work, the same.
@iammelea
Copy link
Contributor Author

iammelea commented May 24, 2018 •
edited
Loading

Zaki
14:50
I would reccomend disabling fast_sync in the config.toml

This work but the same error
Evidence is not valid module=evidence evidence="[VoteA:
appears time to time. but the node sync. now is sync very slow (disabling fast_sync) unless stop in the same block. fast_sync true. don't work.

@ebuchman
Copy link
Member

Thanks for reporting this. Looks like theres an issue with how we handle evidence of double-signing before we're synced, causing disconnections from the network: tendermint/tendermint#1624

Will get a fix pushed today

@ebuchman
Copy link
Member

This should be fixed with gaia 0.17.3 (ie. using Tendermint v0.19.6)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ebuchman @iammelea