Corrupted database?

[Lohoris]

Launching bitcoind I get the following error:

************************
EXCEPTION: St13runtime_error       
CLevelDB(): error opening database environment Corruption: missing start of fragmented record(2)       
bitcoin in AppInit()       

Using version v0.8.0rc1

[gavinandresen]

What OS?

Clean shutdown the run before, or did your machine or bitcoin crash?

This might be a case of "your disk drive was flaky for some unknowable reason."

[Lohoris]

OSX 10.6.8, HFS+ journaled disk.
Unfortunately I don't know if bitcoind was running or not while the machine crashed (we can assume it was).
On SE Nick pointed out the exactly line I'm hitting, so if you want we could add there some more debugging infos to track it down.

i.e. it is not very good if you have to re-download the whole blockchain for a single corrupted element, especially since not it is made of many small files, if would make more sense if only that file was re-downloaded...

[sipa]

@Lohoris The problem is not a corrupted downloaded file - that's easy. In your case, the database is corrupted, despite several systems which try to prevent ever getting an inconsistent state on disk. You also don't need to redownload anything. Starting with -reindex will rebuild the database using the block chain data you already have.

[Lohoris]

Oh! I'll try, thanks. Of course if this works it could be considered fixed...
(it's unlikely I'll be able to try before tomorrow, sorry)

[Lohoris]

reindex at block ~100000 started re-downloading, so I can't go on testing it right now (I've got a limited bandwidth).
Maybe I'll go on testing it on monday, sorry for the delay.

[qubez]

I'll spam this bug, for interesting reading:

http://www.codeofhonor.com/blog/whose-bug-is-this-anyway

On a properly functioning computer this stress test should never fail, but surprisingly we discovered that on about 1% of the computers being used to play Guild Wars it did fail! One percent might not sound like a big deal, but when one million gamers play the game on any given day that means 10,000 would have at least one crash bug. Our programming team could spend weeks researching the bugs for just one day at that rate!

A lot of bug hunting on something as processor and ram intensive and requiring 100% math accuracy like Bitcoin, is likely to be unrepeatable - it's going to be Bitcoin revealing to users their hardware problems. A similar hardware fitness test like the article describes, running before initial-download or during idle cpu, to certify a system's error-free crypto and 8GB of it's storage before attempting to assemble and verify a bit-accurate database, may prevent tears later.

[Lohoris]

Didn't have time to test it properly so far, sorry.
I'll close it for now, and I guess if someone else encounters this same problem upgrading, he'll be able to re-open it...

[Lohoris]

Ok this happened again right now.

I'll reindex and see what happens.

[Lohoris]

Reindex worked, as expected.
Next time this bug happens, what do I need to do before reindexing to debug it?

[zainmw91]

I am having this error i dunno what to do i have bitcoin wallet on Windows Vista when i open it i get Bitcoin-QT: System Error: Database Corrupted what shall i do to fix it?

[shamoons]

Try running it from command line: bitcoind -reindex

[zainmw91]

do you have Skype we can chat on sir?
I backed up the enitre bitcoin folder in app data and all removed it re-install re-inserting data to see if it works
please add me on skype zainmw91

Date: Thu, 18 Apr 2013 10:18:02 -0700
From: notifications@github.com
To: bitcoin@noreply.github.com
CC: zainmw91@hotmail.com
Subject: Re: [bitcoin] Corrupted database? (#2305)

Try running it from command line: bitcoind -reindex

—
Reply to this email directly or view it on GitHub.

[h0jeZvgoxFepBQ2C]

I think this functionality should be added to the GUI too.. Not everybody is a computer pro, and knows how to handle the command shell.. if the index is broken, a window should pop up "it crashed - should we reindex blabla... yes/no?"

[gavinandresen]

I'm going to close this as "corruption probably due to the run-out-of-file-descriptors" bug that was fixed.

[Lohoris]

Good, I'll update and let you know if anything happens.

[thebalaa]

This keeps happening to me with version v0.8.3-beta on OS X 10.8.3 Bitcoin-qt spends a few hours "Reindexing blocks on disk.." and then all is well for a day or so, until I gracefully shutdown bitcoin-qt and upon opening I am greeted with the Corrupt database error dialog. Any updates on this?

[woeisme]

I'm getting this on ubuntu 12.04. it re-indexed once when upgrading to 0.8.4 and a few days later its doing it again. I'm running on a SSD for another possible data point that might be relevant. Will running -reindex on the cli compete faster?

[nikolajsheller]

I've just stated seeing this issue on ubuntu 13.04 on a laptop with an SSD. I saw the error earlier today, re-indexed successfully and got the error again now.
Edit: I'm running v0.8.3.0-g40809ae-beta.

[gmaxwell]

See https://bitcointalk.org/index.php?topic=290922.0 or just update to 0.8.5

[nikolajsheller]

Thank you for the information.
There does not seem to be a PPA version of 0.8.5 for Ubuntu yet. I assume the correct place to clone and build from is: https://github.com/bitcoin/bitcoin/tree/0.8.5

[gmaxwell]

@nikolajsheller Yes, though the official Linux binaries in tar form are for ubuntu.

[chevdor]

I confirm the issue on MacOS. I got my database corrupted a few times already. I always recover using bitcoin-qt -reindex. That works fine but takes hours.
The last time I paid attention to close the application in a clean way. I did and re-opened the application several times without any issue. Nonetheless, after rebooting, the database is corrupted again. I am using 0.8.5-beta, the executable has been built on Friday, September 13, 2013 2:18 AM.

Is there a version that does not have this issue?

[raamdev]

On Mac OS X 10.9 (Mavericks) with Bitcoin-Qt 0.8.5, I was getting the "System Error: Database Corrupted" error every time I started the Bitcoin-Qt client. I tried deleting everything in my Bitcoin directory except peers.dat and wallet.dat, but the error persisted.

What fixed it for me was deleting everything including peers.dat.

For the record, here's what debug.log was showing:

Bitcoin version v0.8.5-beta (2013-09-12 13:35:18 +1000)
Using OpenSSL version OpenSSL 1.0.1c 10 May 2012
Startup time: 2013-11-10 14:58:06
Default data directory /Users/raam/Library/Application Support/Bitcoin
Using data directory /Users/raam/Library/Application Support/Bitcoin
Using at most 125 connections (2560 file descriptors available)
Using 4 threads for script verification
init message: Verifying wallet...
dbenv.open LogDir=/Users/raam/Library/Application Support/Bitcoin/database ErrorFile=/Users/raam/Library/Application Support/Bitcoin/db.log
Bound to [::]:8333
Bound to 0.0.0.0:8333
init message: Loading block index...
Opening LevelDB in /Users/raam/Library/Application Support/Bitcoin/blocks/index
Opened LevelDB successfully
Opening LevelDB in /Users/raam/Library/Application Support/Bitcoin/chainstate
Opened LevelDB successfully
LoadBlockIndexDB(): last block file = 3
LoadBlockIndexDB(): last block file info: CBlockFileInfo(blocks=1563, size=37123020, heights=136929...138491, time=2011-07-18...2011-07-28)
LoadBlockIndexDB(): transaction index disabled
LoadBlockIndexDB(): hashBestChain=00000000000008ef809c70a6ca5da21a03e3e69c105ed29cf285d816ac8a1504  height=138490 date=2011-07-28 14:18:58
init message: Verifying blocks...
Verifying last 288 blocks at level 3
No coin database inconsistencies in last 289 blocks (12767 transactions)
 block index            3730ms
init message: Loading wallet...
nFileVersion = 80500
 wallet                   38ms
LevelDB read failure: Corruption: block checksum mismatch
*** System error: Database corrupted
init message: Loading addresses...
Loaded 18043 addresses from peers.dat  66ms
Flush(false)
wallet.dat refcount=0
wallet.dat checkpoint
wallet.dat detach
wallet.dat closed
DBFlush(false) ended              29ms
StopNode()
Flushed 18043 addresses to peers.dat  34ms
Committing 12178 changed transactions to coin database...
Flush(true)
wallet.dat refcount=0
wallet.dat checkpoint
wallet.dat detach
wallet.dat closed
DBFlush(true) ended              40ms

So, it appears to have been an issue with peers.dat. If anyone needs the peers.dat file to debug this issue, let me know. I saved a copy of the bad version.

[sipa]

peers.dat is not a LevelDB file. Only the blocks/index/ and chainstate/ directories contain LevelDB databases, so the problem must have been with either of those.

[raamdev]

@sipa The only two files that I kept were peers.dat and wallet.dat. I deleted everything else and every time I started the Bitcoin-Qt client I received the "System Error: Database Corrupted" message. It wasn't until I deleted everything including the peers.dat file that the problem resolved itself.

[tahoeboarder909]

Macbook just crashed trying to sync the blockchain for the first time from Bitcoin QT. Now I'm getting the "System error: Database corrupted" I didn't have any coins in the wallet. Does anyone know how I can reset and trying to sync again? The data directory library file -> ~/Library/Application Support/Bitcoin/ isn't found in terminal when I enter it. Thanks.

[chevdor]

I am getting those db corruptions again those days. I tried restoring my
db and blocks folders from backups but it does not do the trick.
Only thing that works:reindex
That takes very long so this is really painful.
On Nov 24, 2013 10:14 AM, "tahoeboarder909" notifications@github.com
wrote:

Macbook just crashed trying to sync the blockchain for the first time from
Bitcoin QT. Now I'm getting the "System error: Database corrupted. I didn't
have any coins in the wallet. Does anyone know how I can reset and trying
to sync again? The data directory library file -> ~/Library/Application
Support/Bitcoin/ isn't found in terminal when I enter it. Thanks.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2305#issuecomment-29151681
.

[tahoeboarder909]

How does one reindex the blockchain??

[chevdor]

Bitcoin-qt -reindex
On Nov 27, 2013 1:27 AM, "Kyle Hess" notifications@github.com wrote:

How does one reindex the blockchain??

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2305#issuecomment-29350410
.

[fanquake]

@tahoeboarder909 Using Terminal

open path/to/Bitcoin-Qt.app --args -reindex

[mapalmalemus]

2017-08-25 19:26:16 Bitcoin version v0.14.2
2017-08-25 19:26:16 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
2017-08-25 19:26:17 Assuming ancestors of block 00000000000000000013176bf8d7dfeab4e1db31dc93bc311b436e82ab226b90 have valid signatures.
2017-08-25 19:26:17 GUI: "registerShutdownBlockReason: Successfully registered: Bitcoin Core no se ha cerrado de forma segura todavía..."
2017-08-25 19:26:17 Default data directory C:\Users\Miguel\AppData\Roaming\Bitcoin
2017-08-25 19:26:17 Using data directory C:\Users\Miguel\AppData\Roaming\Bitcoin
2017-08-25 19:26:17 Using config file C:\Users\Miguel\AppData\Roaming\Bitcoin\bitcoin.conf
2017-08-25 19:26:17 Using at most 125 automatic connections (2048 file descriptors available)
2017-08-25 19:26:17 Using 32 MiB out of 32 requested for signature cache, able to store 1048576 elements
2017-08-25 19:26:17 Using 0 threads for script verification
2017-08-25 19:26:17 Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2017-08-25 19:26:17 Using wallet wallet.dat
2017-08-25 19:26:17 scheduler thread start
2017-08-25 19:26:17 init message: Verificando cartera...
2017-08-25 19:26:17 CDBEnv::Open: LogDir=C:\Users\Miguel\AppData\Roaming\Bitcoin\database ErrorFile=C:\Users\Miguel\AppData\Roaming\Bitcoin\db.log
2017-08-25 19:26:17 Bound to [::]:8333
2017-08-25 19:26:17 Bound to 0.0.0.0:8333
2017-08-25 19:26:17 Cache configuration:
2017-08-25 19:26:17 * Using 2.0MiB for block index database
2017-08-25 19:26:17 * Using 8.0MiB for chain state database
2017-08-25 19:26:17 * Using 440.0MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)
2017-08-25 19:26:17 init message: Cargando indice de bloques...
2017-08-25 19:26:17 Opening LevelDB in C:\Users\Miguel\AppData\Roaming\Bitcoin\blocks\index
2017-08-25 19:26:17 Opened LevelDB successfully
2017-08-25 19:26:17 Using obfuscation key for C:\Users\Miguel\AppData\Roaming\Bitcoin\blocks\index: 0000000000000000
2017-08-25 19:26:17 Opening LevelDB in C:\Users\Miguel\AppData\Roaming\Bitcoin\chainstate
2017-08-25 19:26:18 Corruption: 924 missing files; e.g.: C:\Users\Miguel\AppData\Roaming\Bitcoin\chainstate/499631.ldb
2017-08-25 19:26:20 init message: Cargando indice de bloques...
2017-08-25 19:26:20 Wiping LevelDB in C:\Users\Miguel\AppData\Roaming\Bitcoin\blocks\index
2017-08-25 19:26:20 Opening LevelDB in C:\Users\Miguel\AppData\Roaming\Bitcoin\blocks\index
2017-08-25 19:26:20 Opened LevelDB successfully
2017-08-25 19:26:20 Using obfuscation key for C:\Users\Miguel\AppData\Roaming\Bitcoin\blocks\index: 0000000000000000
2017-08-25 19:26:20 Wiping LevelDB in C:\Users\Miguel\AppData\Roaming\Bitcoin\chainstate
2017-08-25 19:26:20 Opening LevelDB in C:\Users\Miguel\AppData\Roaming\Bitcoin\chainstate
2017-08-25 19:26:20 Corruption: 924 missing files; e.g.: C:\Users\Miguel\AppData\Roaming\Bitcoin\chainstate/499631.ldb
2017-08-25 19:26:21 scheduler thread interrupt
2017-08-25 19:26:21 Shutdown: In progress...
2017-08-25 19:26:21 Shutdown: done
2017-08-25 19:28:47

bitcoin-qt.exe -reindex -chainstate, it doesn´´t work.

I have removed chainstate folder, because it was corrupted and I get to access again and recovery (C:\Users\Miguel\AppData\Roaming\Bitcoin).

How can I repair original chainstate folder?

[TheBlueMatt]

@mapalmalemus hmm, I cant reproduce, but if you simply delete the files in C:\Users\Miguel\AppData\Roaming\Bitcoin\chainstate and then run with bitcoin-qt.exe -reinded-chainstate (note its not separate), that should do it.

[mapalmalemus]

Yes, I have done, thanks!