Always gets 'Corrupted block database detected' with DB cache set to 1024MB #6502
Comments
|
I gave it another try. Deleted all files from BitcoinCore data directory, but encrypted wallet (wallet.dat). The issue reproduced - CPU/HDD/NET utilization went to 0%, number of processed blocks got stuck, after graceful shutdown of BitcoinCore the block database is corrupted. Abnormality found in logfile: Database corruption reported during subsequent BitcoinCore start (same command line used as above for starting BitcoinCore): I can make the complete logfile available via AWS S3, uncompressed size ~380MB. Also, I can make available the corrupted DB files. If it was Linux, I would attach gdb and produce backtrace and coredump of the process. However, it is Windows and I do not know what more information to collect. I can reproduce the issue again and follow instructions for further information gathering, if Bitcoin developers want me to do so. As it seems as a memory allocation issue, maybe it cannot handle that large cache, I will try to make the cache of half size (512MB instead of 1024MB) and run again. UPDATE: With DB cache set to 512MB the issue does not happen. |
GoodMirek
changed the title
GoodMirek
changed the title
|
bad_alloc is an out of memory error. dbcache of 1 GB apparently uses too much memory for your system.
|
|
Wladimir,
I still believe it is a software issue, as my system has a plenty of physical RAM available prior and during the occurence of the issue. As mentioned, let me highlight that RAM was not exhausted, computer has 4GB (usable 3.5GB as of 32bit Windows) + swap and less than 50% of physical RAM was utilized. |
|
bad_alloc means out of virtual memory, which is not directly related to physical RAM, and is limited to 3 GB on 32bit Windows, I believe. |
|
Right. And some virtual memory is used for other purposes, such as mmap'ing databases. |
|
Windows task manager reported that the process was utilizing around 970MB at the time of issue. The memory allocation was growing steadily since start of the blockchain fetch. Reference: [1] https://msdn.microsoft.com/en-us/library/aa366778.aspx |
|
Any reason for not just using the 64-bit version? |
|
No. |
|
I have been challenged with this problem now for about six weeks. I am on Ubuntu 12.04 32-Bit with 1.7 gigs of RAM. Problem has been experienced with 10.2 and 11.0. Based on the discussion, I suspect Bitcoin Core does not work any longer for typical 32-Bit machines. I kept getting the "Do you want to rebuild the block database now?" message. I tried -reindex with no solution. I have an old Windows XP Box for which I have ran a node for years. Now, it always fails. Since it was not relevant to my concerns, I basically ignored it without diagnosis. |
|
With the default dbcache size it works fine on 32-bit machines (I have several nodes on 32-bit). It should even work better with 0.11 as dbcache now correctly estimates the memory usage, whereas it used to underestimate it. |
|
That isn't the case for me. Private bytes on a Win32 node rises over time from a startup value of ~230,000 K and eventually leads to a crash after a day or so of uptime, with something like this: or an "Error reading from database: Database I/O error". I suspect it doesn't have sufficient contiguous VM to allocate new objects or buffers. In the first case above there was also an exception in the debug log: I use Windows Vista x86 with the increaseuserva 2624 boot setting (i.e. VM is ~2624MB, not 2GB). I have downloaded the complete blockchain; I'm not using any parameters, just setting idle priority: bitcoin-qt 0.11.0 manages to get up to over 2,000,000 K private bytes within 16 hours, and rises a further 300,000 K overnight, maintaining the same number of connections (50-60). By this time I have sent 12GB, seen 15,000+ peers, and received 1GB. Outbound traffic is highly variable, it is often around 1-200KByte/sec, but it can spike up to the full ~750KByte/sec line speed. At this point bitcoin-qt goes into irregular periods of high CPU usage (one full core) for up to half a minute. When I break the process in a debugger, the thread concerned has the following on the top of the stack, which suggests to me that it's having trouble allocating memory: In the debug log I get regular "free transaction rejected by rate limiter" and "inputs already spent", but I think that's usual. Perhaps this is useful: "UpdateTip: new best=00000000000000000e87ff685ac893b1c7b4a7159a365bbff0d35b18f3e1691f height=378137 log2_work=83.450831 tx=87128200 date=2015-10-09 13:46:19 progress=0.999999 cache=50.2MiB(7407tx)" [cache varies, this is about as high as it gets, and was just before a crash] There's often "receive version message" coming every few seconds, e.g.: "/BitCoinJ:0.12SNAPSHOT/Satoshi:0.11.0/: version 70001, blocks=378123, us=127.0.0.1:8333, peer=14753" There is always a "database Error" sandwiched between some other log messages on startup: However, this does not seem to stop Bitcoin working. The system is on a private address with router DMZ and UPnP both available and working for IPv4, and a working IPv6 tunnel via SIxXS (used mostly by bitnodes.21.co, it seems). |
bitcoin-qt version: 0.11.0
DB cache set from GUI: 1024MB (the issue does not happen with DB cache set to 512MB)
OS: Win8.1 32bit
Scenario: Upgrade from an older 0.9.x version, with latest block about 32 weeks old. After start of the new 0.11.0 it started to fetch newer blocks, but get stuck after while. Graceful shutdown initiated manually and completed properly. During next start there is a message "Corrupted block database detected", rebuild chosen, causing to start fetch whole blockchain from the very beginning (6yrs 28wks). While it get to 2yrs and a few weeks it got stuck again. Graceful shutdown initiated and completed. During next start there is a message "Corrupted block database detected".
There is a locked wallet. It is running on PC behind NAT. The configuration change is DB cache size set to 1024MB.
Other programs incl. Litecoin client run well, no overheating, no suspected HW memory issues, enough free memory.
There is a lot (hundreds) of messages like following in debug.log prior the graceful shutdown:
Everything goes wrong after this exception occures:
What further information can I provide?
The text was updated successfully, but these errors were encountered: