Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Long Replay times - CPU recommendations #2987

Closed
isnochys opened this issue Sep 28, 2018 · 4 comments
Closed

Long Replay times - CPU recommendations #2987

isnochys opened this issue Sep 28, 2018 · 4 comments

Comments

@isnochys
Copy link

With upcoming patches, witness nodes need to replay for each new version.
What is the recommended CPU hardware for a fast replay?
My specs are:

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 79
model name : Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
stepping : 1
microcode : 0x1
cpu MHz : 2199.998
cache size : 16384 KB
physical id : 0
siblings : 10
core id : 0
cpu cores : 10
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single kaiser fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4399.99
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual

Currently a replay takes 24 hours with this processor.
Other witnesses have i7 CPUs and it takes the 3-5 hours for a complete replay.
Disc is fast enough,non spinning media, SSDs.
And nmon doesn't show any IO waiting times, so the 1 core is always at 100%, that leads to no other bottleneck but the CPU.
wn01

Why the big difference from Xeon to i7 processor?
Is there any recommendation to improve that?

@Gandalf-the-Grey
Copy link
Contributor

When it comes to just CPU, due to the nature of the blockchain, single core performance does matter.

@corkyk
Copy link

corkyk commented Sep 29, 2018

To specify, steemd is single threaded, hence gtg's comment above. RAM is also a factor. I do not see a ram size in your spec above, but the current recommendation for a witness node is at least 64-128gb for faster replays, with a properly sized shm and once replayed, rumor is that it is possible to run on just 32gb (but not recommended). I have had 128gb machines for many months and our replays tend to allow us to be earlier than most to report replay completion when starting at about the same time as others as we mention it anecdotally in witness chat.

So, at least a 2.5-3ghz cpu and 64gb ram seems like a solid baseline for acceptable performance from own observation and discussion with others. Leaning higher will improve your performance as would seem natural but I suspect my 128gb servers are more than is needed.

@isnochys
Copy link
Author

isnochys commented Oct 1, 2018

Witness node has 50 GB RAM, pushed with zram to 90 GB.

Thanks for the answer

@iamsmooth
Copy link

If you are using rocksdb there might be some multithreaded component, but I'm not sure, just a guess here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants