Failed to prevent double sigin when one vote is nil and another is not.

[liangping]

I was jailed on kava-testnet by double sign
The reason is that Vote A is nil.

"evidence": [
          {
            "type": "tendermint/DuplicateVoteEvidence",
            "value": {
              "PubKey": {
                "type": "tendermint/PubKeyEd25519",
                "value": "XrY084jXI0WSlR3iNxLzCB7VdX+j08RJtz9DJ3vC5OM="
              },
              "VoteA": {
                "type": 1,
                "height": "34499",
                "round": "0",
                "block_id": {
                  "hash": "",
                  "parts": {
                    "total": "0",
                    "hash": ""
                  }
                },
                "timestamp": "2019-08-03T04:46:58.802135141Z",
                "validator_address": "E17320EDF8FE7A397B798E06B7612E05083AE0C4",
                "validator_index": "66",
                "signature": "wbCbdDJfceRJiNEgJisJCXPvoUFWNEFYAQAcuOrVkS7PJgKzXcaF0OR1ieQ2OXYObmCOKjqrnaJF2wZjRtJKBg=="
              },
              "VoteB": {
                "type": 1,
                "height": "34499",
                "round": "0",
                "block_id": {
                  "hash": "2FC0C142C517DF851D7268366FF95DF2DF357485E849669603C5212A052F84A2",
                  "parts": {
                    "total": "1",
                    "hash": "748E79B35B07CBE94B0E4BB70ACCBCC0D0489C53008477F3B412D045C61097AC"
                  }
                },
                "timestamp": "2019-08-03T04:46:58.949364761Z",
                "validator_address": "E17320EDF8FE7A397B798E06B7612E05083AE0C4",
                "validator_index": "66",
                "signature": "kuwaOGZb5aJA/7D1GYymOmFAizFv0HYx6hZKyYbp3mTihCvyo9T+XHzvhMtluS7eLsvPKqcqo709U1w4Z6/QCA=="
              }
            }
          }
        ]

Here are signature logs on tmkms server:

04:46:58 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://47.101.10.160:26658] signed PreVote:<nil> at h/r/s 34499/0/6 (102 ms)
04:46:59 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://kava-test.ping.pub:26658] signed PreVote:2FC0C142C5 at h/r/s 34499/0/6 (123 ms)
04:46:59 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://kava-test.ping.pub:26658] signed PreCommit:<nil> at h/r/s 34499/0/6 (102 ms)
04:46:59 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://47.101.10.160:26658] signed PreCommit:<nil> at h/r/s 34499/0/6 (199 ms)
04:47:00 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://kava-test.ping.pub:26658] signed PreVote:F4F042F8EB at h/r/s 34499/1/6 (123 ms)
04:47:01 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://47.101.10.160:26658] signed PreVote:F4F042F8EB at h/r/s 34499/1/6 (123 ms)
04:47:01 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://kava-test.ping.pub:26658] signed PreCommit:F4F042F8EB at h/r/s 34499/1/6 (156 ms)
04:47:01 ESC[0mESC[1mESC[34m[info] ESC[0m[kava-testnet-2000@tcp://47.101.10.160:26658] signed PreCommit:F4F042F8EB at h/r/s 34499/1/6 (212ms)

It does work on following situation:

02:42:35 [error] [kava-testnet-2000@tcp://kava-test.ping.pub:26658] attempted double sign: height regression: last height:33249 new height:33246
04:10:39 [error] [kava-testnet-2000:tcp://47.101.10.160:26658] attempted double sign at h/r/s: 34135/0/3 (8EDD442783 != 7A4FF7ED52)

[tarcieri]

This definitely does look like double signing. I think the issue is here:

https://github.com/tendermint/kms/blob/master/src/chain/state.rs#L102

I believe removing the is_some() checks should prevent this, although I'm uncertain if there are cases where a vote for a block ID is allowed at the same h/r/s as one where it previously voted <nil> (e.g. a PreCommit with an ID, when the PreVote had no ID, which also occurred above).

[liangping]

@tarcieri Yep. that could fix it. I will rebuild and test it.

[mdyring]

@liangping can you provide any additional info on what you did (nothing, restart service, etc.) to trigger the double sign?

[liangping]

@mdyring I did not do anything. I just run two validators with diffierent nodeid connected to one tmkms.

[mdyring]

Thank for your clarifying, I've also been testing same setup for gaia-13004 and 5 validators without incident, but was worrying this happened in a single validator/kms setup. :-)

[tarcieri]

Can you both try upgrading to tmkms v0.6.1 and see if you're able to reproduce issues like these?

[liangping]

@tarcieri Sure, I will do it a couple days later. I am testing stablibity of remoting sign.

[liangping]

@tarcieri @mdyring don't see double sigining on latest version (v0.6.3).

01:21:05 [error] [kava-testnet-2000:tcp://kava-test.ping.pub:26659] attempted double sign at h/r/s: 302390/0/6 (33B8F7DB24 != <nil>)