Validator ID keep changing on restart

[JoeKash]

I'm not sure if this is a Tendermint issue or tmkms issue but if I try to associate the validator node id
with tmkms the id keep changing randomly everytime I restart the validator.

This is testend on the latest Kava node (0.11.0-alpha.1) with Tendermint v0.33.6

If I run tmkms without the peer id.
I get the log -

Sep 17 07:20:54.056 WARN tmkms::session: [kava-testnet-9000@tcp://172.17.0.2:26658]: unverified validator peer ID! 
(D61B22B4673E5174B663DA6F7398702985A6AFA7)

I than set the peer id to my toml file and restart tmkms -
addr = "tcp://D61B22B4673E5174B663DA6F7398702985A6AFA7@172.17.0.2:26658"

This will work as long as the validator is running but if I restart the validator tmkms will start showing the following error and refuse to connect.

Sep 17 07:05:17.139 ERROR tmkms::client: [kava-testnet-9000@tcp://172.17.0.2:26658] verification failed: 172.17.0.2:26658: 
validator peer ID mismatch! (expected D61B22B4673E5174B663DA6F7398702985A6AFA7, 
got FD4AB758EFA25547110D651D44E748133A55718A)

Also I couldn't find how to do the opposite (get the validator verifying tmkms ID)

When I try to set validator config,json with priv_validator_laddr as follows
priv_validator_laddr = "tcp://5E56C6946AD644795E3EE948FA224C1A10A51A61@172.17.0.2:26658"
where 5E56C6946AD644795E3EE948FA224C1A10A51A61 is the node ID I'm getting from the log for KMS node ID

I get the following error when starting the validator -

ERROR: error with private validator socket client: failed to start private validator: listen tcp: 
lookup 5E56C6946AD633795E3EF948FA214C1A10A51A61@172.17.0.2: no such host

I guess I'm just not setting the ID at the right place but I couldn't find where I should set it

[tony-iqlusion]

This is an issue with Tendermint.

Here's the Tendermint-side issue: tendermint/tendermint#3105

It was closed as a planned gRPC migration was supposed to address the issue, however that migration was delayed.

[tony-iqlusion]

Closing this out as a Tendermint issue, not a TMKMS issue.

Please follow up on tendermint/tendermint#3105 or wait for gRPC support.