bitcoin-qt out-of-virtual-address-space problem

[ralphtheninja]

---

EXCEPTION: St9bad_alloc
std::bad_alloc
bitcoin in ProcessMessages()

---

EXCEPTION: St9bad_alloc
std::bad_alloc
bitcoin in ThreadMessageHandler()

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted

The only thing I did was to wipe everything clean and start version 0.8.0. It started to download the block chain for a while and when I got back home it was dead. Is there anything I can do to prepare myself so I can gather more information if it happens again? Thinking of crash dumps, logs etc.

[gavinandresen]

bad_alloc means it is running out of memory. How much memory do you have? What OS?

[ralphtheninja]

magnus@desktop|21:59|~/src/bitcoin ((v0.8.0)) $ cat /proc/meminfo
MemTotal: 16445204 kB
MemFree: 4247964 kB
Buffers: 341888 kB
Cached: 8300268 kB
SwapCached: 0 kB
Active: 10045068 kB
Inactive: 1818152 kB
Active(anon): 2951020 kB
Inactive(anon): 303216 kB
Active(file): 7094048 kB
Inactive(file): 1514936 kB
Unevictable: 80 kB
Mlocked: 80 kB
HighTotal: 15751096 kB
HighFree: 4179560 kB
LowTotal: 694108 kB
LowFree: 68404 kB
SwapTotal: 16659452 kB
SwapFree: 16659452 kB
Dirty: 3440 kB
Writeback: 0 kB
AnonPages: 3221360 kB
Mapped: 233656 kB
Shmem: 33172 kB
Slab: 236588 kB
SReclaimable: 200160 kB
SUnreclaim: 36428 kB
KernelStack: 5144 kB
PageTables: 31576 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 24882052 kB
Committed_AS: 7121976 kB
VmallocTotal: 122880 kB
VmallocUsed: 63688 kB
VmallocChunk: 52276 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 98296 kB
DirectMap2M: 815104 kB

I'm running Lubuntu 12.04, 32 bit.

[jeremysawicki]

This is how bitcoin runs out of memory (virtual address space) while downloading blocks:

Whenever a node receives a block that it cannot link to the chain, it stores it in memory in mapOrphanBlocks. There is no limit to the size of that data. When a node is downloading blocks, it may download and keep in memory many blocks that it can't currently link to the chain. In particular, if your node connects to another node which is also downloading blocks and which is further along in the download, the other node will spam your node with an "inv" message for every new block that it downloads (no, the logic in CBlock::AcceptBlock does not prevent it), and your node will download and store those blocks in mapOrphanBlocks. This can occur with multiple connected nodes simultaneously which are at different points in the download.

I encountered this problem several times with 0.8.0rc1. With the current size of the blockchain, it is not hard to exhaust the virtual address space (at most 4GiB) in 32-bit builds, which includes all Windows builds.

The fix could proceed in one of two directions: (1) store orphan blocks on disk instead of in memory, or (2) don't keep an unbounded set of orphan blocks. For (2), the ideal would be to prevent downloading of too many orphans rather than discarding orphans that have already been downloaded, which is wasteful. The goal would be to download blocks roughly in order. One idea is to make use of "getheaders" during block download. Headers would be downloaded and linked together, and blocks would only be downloaded if we already know based on headers that they are not too far away from the chain of blocks that we already have.

[sipa]

Headers-first synchronization (being worked on) removes the concept of orphan blocks entirely, as a block is only downloaded when its ancestry is already known, so it can always be saved on disk.

[laanwj]

Since 0.9.0 the number of orphan blocks that is stored has been restricted, so this immediate issue should be fixed.