New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Keep getting "rafthttp: the clock difference against peer 5bd0394181f126dd is too high [10.449391606s > 1s]" #7051
Comments
The system clock is different on you machines. Enable NTP can fix this. Or you have to manually sync the clocks on different machines. |
@xiang90 The system date/time differences is in seconds but still it is coming and i observe this only while running with large data as i have mentioned. Will it impact performance..? its taking 31m for the fastest client for putting 1.6M data, (1.6M * 5 clients with 5 conncurrent ops are happening continuously). Also i observe snapshot and purge happening too frequently. |
@heyrakeshsingh the time difference will cause v2 TTLs to expire too soon / too late. I don't believe it will make a performance difference so long as there's no TTL keys. |
I am observing the same issue but though i have NTP installed on all Master vms and all have same time. any hint> am runing k8s 1.10.9 2019-03-05 19:22:56.134642 W | rafthttp: the clock difference against peer 6c8a781291acb5f2 is too high [33.951034466s > 1s] |
Did you restart ETCD after installing NTP? |
I have setup 5node+2 proxy cluster (each on node on separate machine- 12 core, 32 GB RAM ) with db size 4gb. Running 5 clients where each client spawning 5 threads. The operation performed is
The operation is performed for 1.6 M keys from each client (ie total 25 threads sending request to etcd cluster server for 8 M keys)
I am getting below logs:-
2016-12-22 17:00:29.475936 W | rafthttp: the clock difference against peer 5bd0394181f126dd is too high [10.449391606s > 1s]
2016-12-22 17:00:59.476151 W | rafthttp: the clock difference against peer 5bd0394181f126dd is too high [10.449443811s > 1s]
2016-12-22 17:01:29.476315 W | rafthttp: the clock difference against peer 5bd0394181f126dd is too high [10.449597395s > 1s]
2016-12-22 17:01:59.476514 W | rafthttp: the clock difference against peer 5bd0394181f126dd is too high [10.449581152s > 1s]
The text was updated successfully, but these errors were encountered: