Restart and remap are too slow

[zhaojkun]

prolems

We use bolt options like etcd

const initialMmapSize = 10 * 1024 * 1024 * 1024 // 10G
db, err := bolt.Open("my.db", 0600, &bolt.Options{
		MmapFlags:       syscall.MAP_POPULATE,
		InitialMmapSize: initialMmapSize,
})

Restart boltdb is very slow.From the system metrics there are lots of disk reads duration db restart. When open returns, all the db file will be in the page cache. From this, I guess all bytes of the db file may be read into memory.

When the memory cannot hold all the db file and some pages of the file has been swapped out, boltdb's remap will be also very slow.

Is there some methods to speed up boltdb's restart and remap

system

• System memory : 96G
• Disk: SATA HDD
• DB File Size: >= 10G

[heyitsanthony]

@zhaojkun try it without MAP_POPULATE...

[C-Otto]

I don't know a lot about bbolt, but I know that in my setup reading the DB file into memory (using cat, basically) improves startup time a lot (factor >10x). Please have a look at the lnd issue linked above.

[C-Otto]

@zhaojkun could you please reopen (or explain why this is closed)?