geth 1.15.8 is slower than previous versions

[islishude]

On 1.15.5, It takes about 300ms to import 2048 blocks

INFO [04-16|02:50:40.312] Block synchronisation started
INFO [04-16|02:50:40.348] Imported new chain segment               number=108 hash=bc14dc..70c573 blocks=108 txs=0 mgas=0.000 elapsed=20.469ms    mgasps=0.000 age=3mo3w5d triedirty=0.00B
INFO [04-16|02:50:40.348] Indexed transactions                     blocks=109 txs=0 tail=0 elapsed="502.203µs"
INFO [04-16|02:50:40.537] Imported new chain segment               number=117 hash=1d58b1..c91b11 blocks=9   txs=9 mgas=1.774 elapsed=4.794ms     mgasps=370.090 age=3mo3w5d snapdiffs=2.96KiB triediffs=15.72KiB triedirty=0.00B
INFO [04-16|02:50:40.584] Imported new chain segment               number=124 hash=c66f94..74d89e blocks=7   txs=7 mgas=12.349 elapsed=4.711ms     mgasps=2621.090 age=3mo3w5d snapdiffs=5.82KiB triediffs=30.60KiB triedirty=0.00B
INFO [04-16|02:50:41.003] Imported new chain segment               number=2172 hash=bd6db8..7ce356 blocks=2048 txs=165 mgas=39.510 elapsed=408.758ms   mgasps=96.658   age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:41.281] Imported new chain segment               number=4220 hash=d5c863..c9933e blocks=2048 txs=0   mgas=0.000  elapsed=270.992ms   mgasps=0.000    age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:41.550] Imported new chain segment               number=6268 hash=0c4edb..f11eef blocks=2048 txs=0   mgas=0.000  elapsed=261.918ms   mgasps=0.000    age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:41.850] Imported new chain segment               number=8316 hash=d9dc2d..40978a blocks=2048 txs=0   mgas=0.000  elapsed=285.162ms   mgasps=0.000    age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:42.130] Imported new chain segment               number=10364 hash=299e78..08e16d blocks=2048 txs=0   mgas=0.000  elapsed=266.551ms   mgasps=0.000    age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:42.422] Imported new chain segment               number=12412 hash=96ac04..c77e65 blocks=2048 txs=0   mgas=0.000  elapsed=285.682ms   mgasps=0.000    age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:42.720] Imported new chain segment               number=14460 hash=09427a..0135c0 blocks=2048 txs=0   mgas=0.000  elapsed=285.528ms   mgasps=0.000    age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:42.907] Looking for peers                        peercount=24 tried=0 static=0
INFO [04-16|02:50:43.001] Imported new chain segment               number=16508 hash=e8222e..291a73 blocks=2048 txs=0   mgas=0.000  elapsed=273.398ms   mgasps=0.000    age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:43.276] Imported new chain segment               number=18556 hash=bec76f..f6e583 blocks=2048 txs=0   mgas=0.000  elapsed=268.369ms   mgasps=0.000    age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:43.563] Imported new chain segment               number=20604 hash=63467b..808737 blocks=2048 txs=0   mgas=0.000  elapsed=280.040ms   mgasps=0.000    age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:43.861] Imported new chain segment               number=22652 hash=2fed1a..a4da96 blocks=2048 txs=0   mgas=0.000  elapsed=285.008ms   mgasps=0.000    age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:50:44.304] Imported new chain segment               number=24700 hash=c376d5..df149a blocks=2048 txs=307 mgas=0.000  elapsed=435.593ms   mgasps=0.000    age=3mo3w4d snapdiffs=157.52KiB triediffs=436.52KiB triedirty=122.31KiB

but on 1.15.8, it takes 7s

INFO [04-16|02:47:09.964] Imported new chain segment               number=108 hash=bc14dc..70c573 blocks=108 txs=0 mgas=0.000 elapsed=212.658ms mgasps=0.000 age=3mo3w5d triedirty=0.00B
INFO [04-16|02:47:09.965] Indexed transactions                     blocks=109 txs=0 tail=0 elapsed=1.374ms
INFO [04-16|02:47:09.985] Imported new chain segment               number=116 hash=a2fd57..41a1e3 blocks=8   txs=8 mgas=1.232 elapsed=21.340ms  mgasps=57.719 age=3mo3w5d snapdiffs=2.57KiB triediffs=13.66KiB triedirty=0.00B
INFO [04-16|02:47:17.468] Imported new chain segment               number=2164 hash=c04d96..49371b blocks=2048 txs=173 mgas=52.401 elapsed=7.442s    mgasps=7.041  age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:47:25.217] Imported new chain segment               number=4212 hash=2bc567..27d505 blocks=2048 txs=0   mgas=0.000  elapsed=7.741s    mgasps=0.000  age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:47:32.956] Imported new chain segment               number=6260 hash=7e6682..fe8b93 blocks=2048 txs=0   mgas=0.000  elapsed=7.731s    mgasps=0.000  age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:47:40.698] Imported new chain segment               number=8308 hash=69bef0..2a3a53 blocks=2048 txs=0   mgas=0.000  elapsed=7.734s    mgasps=0.000  age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:47:48.460] Imported new chain segment               number=10356 hash=8a6d12..992686 blocks=2048 txs=0   mgas=0.000  elapsed=7.753s    mgasps=0.000  age=3mo3w5d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:47:56.211] Imported new chain segment               number=12404 hash=899768..68be7c blocks=2048 txs=0   mgas=0.000  elapsed=7.743s    mgasps=0.000  age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:48:03.960] Imported new chain segment               number=14452 hash=deca33..282862 blocks=2048 txs=0   mgas=0.000  elapsed=7.741s    mgasps=0.000  age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:48:11.713] Imported new chain segment               number=16500 hash=48d68d..627279 blocks=2048 txs=0   mgas=0.000  elapsed=7.744s    mgasps=0.000  age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:48:19.456] Imported new chain segment               number=18548 hash=d67611..b172d1 blocks=2048 txs=0   mgas=0.000  elapsed=7.735s    mgasps=0.000  age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:48:27.206] Imported new chain segment               number=20596 hash=cb202a..04acaf blocks=2048 txs=0   mgas=0.000  elapsed=7.742s    mgasps=0.000  age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:48:34.948] Imported new chain segment               number=22644 hash=c52068..16d6ef blocks=2048 txs=0   mgas=0.000  elapsed=7.734s    mgasps=0.000  age=3mo3w4d snapdiffs=85.35KiB triediffs=416.18KiB triedirty=20.40KiB
INFO [04-16|02:48:42.965] Imported new chain segment               number=24569 hash=0cc3fe..b253b9 blocks=1925 txs=307 mgas=0.000  elapsed=8.000s    mgasps=0.000  age=3mo3w4d snapdiffs=157.52KiB triediffs=436.52KiB triedirty=122.31KiB
INFO [04-16|02:48:43.434] Imported new chain segment               number=24692 hash=eee97c..01274f blocks=123  txs=0   mgas=0.000  elapsed=468.721ms mgasps=0.000  age=3mo3w4d snapdiffs=157.52KiB triediffs=436.52KiB triedirty=122.31KiB
INFO [04-16|02:48:51.224] Imported new chain segment               number=26740 hash=cf1cd6..17b4c1 blocks=2048 txs=18  mgas=0.185  elapsed=7.783s    mgasps=0.024  age=3mo3w4d snapdiffs=162.25KiB triediffs=440.46KiB triedirty=126.74KiB
INFO [04-16|02:48:59.011] Imported new chain segment               number=28788 hash=b72a3f..c48e92 blocks=2048 txs=12  mgas=0.000  elapsed=7.775s    mgasps=0.000  age=3mo3w4d snapdiffs=165.08KiB triediffs=442.81KiB triedirty=129.62KiB
INFO [04-16|02:49:06.783] Imported new chain segment               number=30836 hash=f9fc91..07260c blocks=2048 txs=11  mgas=0.000  elapsed=7.764s    mgasps=0.000  age=3mo3w4d snapdiffs=167.68KiB triediffs=444.48KiB triedirty=132.61KiB

[jwasinger]

Hmm that's weird. I did compare the beginning of a full-sync of 1.15.8 and 1.15.7 and there's not a measurable difference between the two:

[islishude]

I've tested on hoodi on aws c5.2xlarge and ebs-gp3, it's very slow as well, you can see the elapsed is 8.044s to import 100 blocks.

docker run --name geth --network host --rm -v /data/geth:/root/.ethereum ethereum/client-go:v1.15.8 --hoodi --synctarget 0xc40f122e4bc5ce023aadfe2bbec1f93d5e9c5797b3e53fd7b85f15bfda1581ef

INFO [04-16|07:45:25.669] Waiting for peers to retrieve sync target
INFO [04-16|07:45:26.670] Attempting to retrieve sync target       peer=65df594551865eed651f2630e8c22487dee68d2ebc1aeb34268b31f05c20edf1
INFO [04-16|07:45:26.886] Syncing beacon headers                   downloaded=512 left=196,235 eta=1m7.575s
INFO [04-16|07:45:27.804] New local node record                    seq=1,744,789,525,666 id=fce99f435fd7e530 ip=18.141.44.40 udp=30303 tcp=30303
INFO [04-16|07:45:34.977] Syncing beacon headers                   downloaded=126,976 left=69771   eta=4.542s
INFO [04-16|07:45:39.291] Syncing beacon headers                   downloaded=196,747 left=0       eta=0s
ERROR[04-16|07:45:39.291] Latest filled block is not available
INFO [04-16|07:45:39.291] Block synchronisation started
INFO [04-16|07:45:39.308] Imported new chain segment               number=1 hash=c62962..c47996 blocks=1 txs=0 mgas=0.000 elapsed=4.149ms mgasps=0.000 age=4w1d19h snapdiffs=204.00B triediffs=1.88KiB triedirty=0.00B
ERROR[04-16|07:45:39.309] Nil finalized block cannot evict old blobs
INFO [04-16|07:45:39.313] Indexed transactions                     blocks=2 txs=0 tail=0 elapsed=4.180ms
INFO [04-16|07:45:39.722] Imported new chain segment               number=60 hash=5a68d0..f2b9e7 blocks=59 txs=4 mgas=0.085 elapsed=250.662ms mgasps=0.339 age=4w1d19h snapdiffs=12.27KiB triediffs=166.92KiB triedirty=0.00B
ERROR[04-16|07:45:39.723] Nil finalized block cannot evict old blobs
INFO [04-16|07:45:42.398] Imported new chain segment               number=218 hash=15daf1..4bf746 blocks=158 txs=1698 mgas=2969.393 elapsed=2.674s    mgasps=1110.191 age=4w1d18h snapdiffs=3.72MiB  triediffs=53.50MiB  triedirty=45.84KiB
ERROR[04-16|07:45:42.398] Nil finalized block cannot evict old blobs
INFO [04-16|07:45:50.429] Imported new chain segment               number=521 hash=5f1969..920293 blocks=303 txs=3846 mgas=6780.148 elapsed=8.022s    mgasps=845.097  age=4w1d17h snapdiffs=4.07MiB  triediffs=57.62MiB  triedirty=44.84MiB
INFO [04-16|07:45:58.445] Imported new chain segment               number=876 hash=eb4c70..4ffa28 blocks=355 txs=3253 mgas=5620.841 elapsed=8.015s    mgasps=701.280  age=4w1d16h snapdiffs=2.98MiB  triediffs=61.55MiB  triedirty=76.49MiB
WARN [04-16|07:46:00.657] Post-merge network, but no beacon client seen. Please launch one to follow the chain!
INFO [04-16|07:46:06.461] Imported new chain segment               number=1565 hash=54e2ac..70ea35 blocks=689 txs=2120 mgas=3678.703 elapsed=8.016s    mgasps=458.920  age=4w1d13h snapdiffs=3.72MiB  triediffs=54.37MiB  triedirty=99.57MiB
INFO [04-16|07:46:14.466] Imported new chain segment               number=1797 hash=6150c0..690556 blocks=232 txs=3368 mgas=5946.802 elapsed=8.005s    mgasps=742.821  age=4w1d13h snapdiffs=7.12MiB  triediffs=108.86MiB triedirty=124.88MiB
INFO [04-16|07:46:19.868] Imported new chain segment               number=2266 hash=017588..d282ca blocks=469 txs=819  mgas=1395.962 elapsed=5.401s    mgasps=258.451  age=4w1d11h snapdiffs=971.55KiB triediffs=1.04MiB   triedirty=152.56MiB
ERROR[04-16|07:46:19.868] Nil finalized block cannot evict old blobs
INFO [04-16|07:46:27.888] Imported new chain segment               number=2777 hash=d8b724..523c53 blocks=511 txs=798  mgas=1594.151 elapsed=8.012s    mgasps=198.954  age=4w1d9h  snapdiffs=1.17MiB   triediffs=2.82MiB   triedirty=152.84MiB
INFO [04-16|07:46:35.966] Imported new chain segment               number=2875 hash=be72d4..58f44f blocks=98  txs=991  mgas=2577.124 elapsed=8.078s    mgasps=318.998  age=4w1d9h  snapdiffs=1.51MiB   triediffs=6.53MiB   triedirty=152.90MiB

[islishude]

sepolia

1.15.5, 700ms

INFO [04-16|07:56:19.962] Indexed transactions                     blocks=2049 txs=0 tail=0 elapsed=8.137ms
INFO [04-16|07:56:20.676] Imported new chain segment               number=4096 hash=9d9262..170328 blocks=2048 txs=0 mgas=0.000 elapsed=712.704ms   mgasps=0.000 age=3y6mo1w  snapdiffs=185.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:21.390] Imported new chain segment               number=6144 hash=be87ba..7a4a94 blocks=2048 txs=0 mgas=0.000 elapsed=707.623ms   mgasps=0.000 age=3y6mo1w  snapdiffs=279.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:22.114] Imported new chain segment               number=8192 hash=6f299b..cdcc4b blocks=2048 txs=0 mgas=0.000 elapsed=712.781ms   mgasps=0.000 age=3y6mo1w  snapdiffs=373.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:22.837] Imported new chain segment               number=10240 hash=6d4ce6..d3b6cd blocks=2048 txs=0 mgas=0.000 elapsed=716.829ms   mgasps=0.000 age=3y6mo1w  snapdiffs=467.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:23.572] Imported new chain segment               number=12288 hash=5f4fd5..65e4c2 blocks=2048 txs=0 mgas=0.000 elapsed=724.546ms   mgasps=0.000 age=3y6mo1w  snapdiffs=561.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:24.290] Imported new chain segment               number=14336 hash=1e2cb7..1a01b0 blocks=2048 txs=0 mgas=0.000 elapsed=711.882ms   mgasps=0.000 age=3y6mo1w  snapdiffs=655.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:25.024] Imported new chain segment               number=16384 hash=0be16c..262c53 blocks=2048 txs=0 mgas=0.000 elapsed=728.830ms   mgasps=0.000 age=3y6mo1w  snapdiffs=749.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:25.746] Imported new chain segment               number=18432 hash=697620..6d6548 blocks=2048 txs=0 mgas=0.000 elapsed=715.902ms   mgasps=0.000 age=3y6mo1w  snapdiffs=843.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:26.481] Imported new chain segment               number=20480 hash=af48bd..621a30 blocks=2048 txs=0 mgas=0.000 elapsed=724.665ms   mgasps=0.000 age=3y6mo1w  snapdiffs=937.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:27.203] Imported new chain segment               number=22528 hash=d55d69..1c1eb6 blocks=2048 txs=0 mgas=0.000 elapsed=716.438ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.01MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:27.939] Imported new chain segment               number=24576 hash=cc173e..ff7f68 blocks=2048 txs=0 mgas=0.000 elapsed=726.612ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.10MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:28.682] Imported new chain segment               number=26624 hash=3a5134..f8f7cb blocks=2048 txs=0 mgas=0.000 elapsed=732.807ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.19MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:29.413] Imported new chain segment               number=28672 hash=e8a52a..432683 blocks=2048 txs=0 mgas=0.000 elapsed=724.601ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.28MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:30.137] Imported new chain segment               number=30720 hash=be5f1a..1e7cd8 blocks=2048 txs=0 mgas=0.000 elapsed=717.107ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.37MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:30.879] Imported new chain segment               number=32768 hash=dcdb91..da1426 blocks=2048 txs=0 mgas=0.000 elapsed=732.277ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.47MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:31.619] Imported new chain segment               number=34816 hash=f32465..f9b01d blocks=2048 txs=1 mgas=0.021 elapsed=734.688ms   mgasps=0.029 age=3y6mo1w  snapdiffs=1.56MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:32.363] Imported new chain segment               number=36864 hash=d3c763..0a699d blocks=2048 txs=0 mgas=0.000 elapsed=737.760ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.65MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:33.103] Imported new chain segment               number=38912 hash=53f54e..299756 blocks=2048 txs=0 mgas=0.000 elapsed=727.079ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.74MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|07:56:33.850] Imported new chain segment               number=40960 hash=83ca5a..685c00 blocks=2048 txs=0 mgas=0.000 elapsed=740.647ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.83MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:34.593] Imported new chain segment               number=43008 hash=4c9d9e..8ad646 blocks=2048 txs=0 mgas=0.000 elapsed=737.899ms   mgasps=0.000 age=3y6mo1w  snapdiffs=1.93MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:35.356] Imported new chain segment               number=45056 hash=a769a0..4b8335 blocks=2048 txs=0 mgas=0.000 elapsed=753.028ms   mgasps=0.000 age=3y6mo6d  snapdiffs=2.02MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:36.117] Imported new chain segment               number=47104 hash=942f69..ad9524 blocks=2048 txs=0 mgas=0.000 elapsed=754.955ms   mgasps=0.000 age=3y6mo6d  snapdiffs=2.11MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:36.886] Imported new chain segment               number=49152 hash=b021ee..94b4e6 blocks=2048 txs=0 mgas=0.000 elapsed=762.885ms   mgasps=0.000 age=3y6mo6d  snapdiffs=2.20MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:37.635] Imported new chain segment               number=51200 hash=9bc46a..8f48aa blocks=2048 txs=0 mgas=0.000 elapsed=743.579ms   mgasps=0.000 age=3y6mo1d  snapdiffs=2.29MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:38.387] Imported new chain segment               number=53248 hash=2bc5a7..183d53 blocks=2048 txs=0 mgas=0.000 elapsed=746.067ms   mgasps=0.000 age=3y6mo1d  snapdiffs=2.38MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:39.148] Imported new chain segment               number=55296 hash=1d3990..2eca09 blocks=2048 txs=0 mgas=0.000 elapsed=750.592ms   mgasps=0.000 age=3y6mo22h snapdiffs=2.48MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:39.899] Imported new chain segment               number=57344 hash=23a7f5..b1065f blocks=2048 txs=0 mgas=0.000 elapsed=745.105ms   mgasps=0.000 age=3y6mo15h snapdiffs=2.57MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:40.645] Imported new chain segment               number=59392 hash=ff625b..7890b2 blocks=2048 txs=0 mgas=0.000 elapsed=740.501ms   mgasps=0.000 age=3y6mo8h  snapdiffs=2.66MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:41.402] Imported new chain segment               number=61440 hash=eac8a0..2d5bc3 blocks=2048 txs=0 mgas=0.000 elapsed=750.264ms   mgasps=0.000 age=3y6mo1h  snapdiffs=2.75MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:42.165] Imported new chain segment               number=63488 hash=47ed34..2b092e blocks=2048 txs=0 mgas=0.000 elapsed=757.257ms   mgasps=0.000 age=3y5mo4w  snapdiffs=2.84MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:43.078] Imported new chain segment               number=65536 hash=0ac9db..5a8531 blocks=2048 txs=0 mgas=0.000 elapsed=901.984ms   mgasps=0.000 age=3y5mo4w  snapdiffs=2.94MiB   triediffs=83.62KiB triedirty=991.00B
INFO [04-16|07:56:43.838] Imported new chain segment               number=67584 hash=4196dd..c65f8d blocks=2048 txs=0 mgas=0.000 elapsed=750.131ms   mgasps=0.000 age=3y5mo4w  snapdiffs=3.03MiB   triediffs=83.62KiB triedirty=991.00B

1.15.8, 6s

INFO [04-16|08:01:11.210] Imported new chain segment               number=4096 hash=9d9262..170328 blocks=2048 txs=0 mgas=0.000 elapsed=6.323s  mgasps=0.000 age=3y6mo1w  snapdiffs=185.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:01:17.551] Imported new chain segment               number=6144 hash=be87ba..7a4a94 blocks=2048 txs=0 mgas=0.000 elapsed=6.335s  mgasps=0.000 age=3y6mo1w  snapdiffs=279.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:01:23.910] Imported new chain segment               number=8192 hash=6f299b..cdcc4b blocks=2048 txs=0 mgas=0.000 elapsed=6.348s  mgasps=0.000 age=3y6mo1w  snapdiffs=373.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:01:30.315] Imported new chain segment               number=10240 hash=6d4ce6..d3b6cd blocks=2048 txs=0 mgas=0.000 elapsed=6.398s  mgasps=0.000 age=3y6mo1w  snapdiffs=467.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:01:36.681] Imported new chain segment               number=12288 hash=5f4fd5..65e4c2 blocks=2048 txs=0 mgas=0.000 elapsed=6.360s  mgasps=0.000 age=3y6mo1w  snapdiffs=561.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:01:43.030] Imported new chain segment               number=14336 hash=1e2cb7..1a01b0 blocks=2048 txs=0 mgas=0.000 elapsed=6.339s  mgasps=0.000 age=3y6mo1w  snapdiffs=655.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:01:49.401] Imported new chain segment               number=16384 hash=0be16c..262c53 blocks=2048 txs=0 mgas=0.000 elapsed=6.363s  mgasps=0.000 age=3y6mo1w  snapdiffs=749.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:01:55.778] Imported new chain segment               number=18432 hash=697620..6d6548 blocks=2048 txs=0 mgas=0.000 elapsed=6.367s  mgasps=0.000 age=3y6mo1w  snapdiffs=843.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:02:02.189] Imported new chain segment               number=20480 hash=af48bd..621a30 blocks=2048 txs=0 mgas=0.000 elapsed=6.404s  mgasps=0.000 age=3y6mo1w  snapdiffs=937.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:02:08.560] Imported new chain segment               number=22528 hash=d55d69..1c1eb6 blocks=2048 txs=0 mgas=0.000 elapsed=6.364s  mgasps=0.000 age=3y6mo1w  snapdiffs=1.01MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:02:14.903] Imported new chain segment               number=24576 hash=cc173e..ff7f68 blocks=2048 txs=0 mgas=0.000 elapsed=6.336s  mgasps=0.000 age=3y6mo1w  snapdiffs=1.10MiB   triediffs=83.62KiB triedirty=741.00B
INFO [04-16|08:02:21.269] Imported new chain segment               number=26624 hash=3a5134..f8f7cb blocks=2048 txs=0 mgas=0.000 elapsed=6.360s  mgasps=0.000 age=3y6mo1w  snapdiffs=1.19MiB   triediffs=83.62KiB triedirty=741.00B

[jwasinger]

Interesting. So the regressions you've posted appear when Geth is importing close-to-empty blocks.

[jwasinger]

I had posted the results of some benchmarks that I did. However, I wasn't on the same basline version (v1.15.5) that's apparently before the regression. Running them again with v.15.8 and v1.15.5 to confirm this.

[s1na]

My immediate feeling was it's because the new log indexer is doing its job and it should become fast once that is done. But the indexer should be disabled during catch-up sync AFAIK which is the case in all of the logs above. But just to rule it out can you confirm that the logs are not indicating indexer activity?

[islishude]

My immediate feeling was it's because the new log indexer

blocks=2048 txs=0 mgas=0.000 elapsed=6.360s mgasps=0.000

It's slow even there's no transactions.

[rjl493456442]

Can you try to run the Pebble with Async mode? https://github.com/ethereum/go-ethereum/blob/master/ethdb/pebble/pebble.go#L188

[islishude]

1.15.7 is faster, it uses async mode.

INFO [04-16|11:52:15.273] Imported new chain segment               number=4096 hash=9d9262..170328 blocks=2048 txs=0 mgas=0.000 elapsed=712.738ms   mgasps=0.000 age=3y6mo1w  snapdiffs=185.64KiB triediffs=83.62KiB triedirty=741.00B
INFO [04-16|11:52:15.986] Imported new chain segment               number=6144 hash=be87ba..7a4a94 blocks=2048 txs=0 mgas=0.000 elapsed=707.850ms   mgasps=0.000 age=3y6mo1w  snapdiffs=279.64KiB triediffs=83.62KiB triedirty=741.00B

[islishude]

Okay, that's the cause of importing slowly.

I've tried on master branch with below code change.

index 969e67af5a..b26d08cacd 100644
--- a/ethdb/pebble/pebble.go
+++ b/ethdb/pebble/pebble.go
@@ -185,7 +185,7 @@ func New(file string, cache int, handles int, namespace string, readonly bool, e
                fn:           file,
                log:          logger,
                quitChan:     make(chan chan error),
-               writeOptions: &pebble.WriteOptions{Sync: !ephemeral},
+               writeOptions: &pebble.WriteOptions{Sync: false},
        }
        opt := &pebble.Options{
                // Pebble has a single combined cache area and the write

[rjl493456442]

INFO [04-16|02:50:44.304] Imported new chain segment number=24700 hash=c376d5..df149a blocks=2048 txs=307 mgas=0.000

Are you running Geth for ethereum or some other things? It's weird the reported mgas is 0

[islishude]

It's sepolia

docker run --name geth --network host --rm -v /data/sepolia7:/root/.ethereum ethereum/client-go:v1.15.7 --sepolia --synctarget 0xa6eec61b83b971015a137c69c78d6c2df20a10060dd459499470948a8a65b364

[s1na]

I think this will be fixed via #31703