New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New block database files platform-dependent? #2293

Closed
Belkaar opened this Issue Feb 10, 2013 · 23 comments

Comments

Projects
None yet
6 participants
@Belkaar

Belkaar commented Feb 10, 2013

I compiled the content of the git tag 0.8-rc1 on my raspberry pi (ARM) and downloaded the blockchain (well most of it) into the new format.
When I try to copy the contents of blocks and chainstate an x86 machine and start the precompiled 0.8-rc1 bitcoind it crashes. Same happens with pi-created block chain files if you try to start the 0.8-rc1 windows.
(Copying the datafiles between x86 and ARM was no problem with 0.7)

Bitcoin version v0.8.0rc1-beta (2013-02-06 16:06:43 -0500)
Using OpenSSL version OpenSSL 0.9.8k 25 Mar 2009
Startup time: 2013-02-10 14:07:00
Default data directory /root/.bitcoin
Used data directory /mnt/store_0/bitcoin/bitcoincodes
init message: Verifying wallet integrity...
dbenv.open LogDir=/mnt/store_0/bitcoin/bitcoincodes/database ErrorFile=/mnt/store_0/bitcoin/bitcoincodes/db.log
Bound to [::]:8333
Bound to 0.0.0.0:8333
init message: Loading block index...
Opening LevelDB in /mnt/store_0/bitcoin/bitcoincodes/blocks/index
Opened LevelDB successfully
Opening LevelDB in /mnt/store_0/bitcoin/bitcoincodes/chainstate
Opened LevelDB successfully
LoadBlockIndex(): last block file = 37
LoadBlockIndex(): last block file: CBlockFileInfo(blocks=133, size=23791072, heights=216794..216926, time=2013-01-16..2013-01-17)
LoadBlockIndex(): transaction index disabled
LoadBlockIndex(): hashBestChain=00000000000004e3fa816b4c834a13e581e446c2b6881c83b28f9864d615a663 height=216926 date=2013-01-17 19:45:03
init message: Verifying block database integrity...
Verifying last 288 blocks at level 3
ERROR: DisconnectBlock() : outputs still spent? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : outputs still spent? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : undo data adding output to missing transaction
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
ERROR: DisconnectBlock() : added transaction mismatch? database corrupted
Unable to open file /mnt/store_0/bitcoin/bitcoincodes/blocks/rev00037.dat
ERROR: CBlockUndo::ReadFromDisk() : OpenBlockFile failed
ERROR: VerifyDB() : *** found bad undo data at 216819, hash=0000000000000333189f2bc4a671ed10abedb5c9fb3f6d69a2498c8c2317eeaa

Error: Corrupted block database detected. Please restart the client with -reindex.
Flush(false)
DBFlush(false) ended 0ms
StopNode()
Flushed 0 addresses to peers.dat 467ms
Committing 27465 changed transactions to coin database...
Flush(true)
DBFlush(true) ended 0ms
Bitcoin exited

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Feb 10, 2013

Member

Did you copy the files while the application was running?

Member

sipa commented Feb 10, 2013

Did you copy the files while the application was running?

@Belkaar

This comment has been minimized.

Show comment
Hide comment
@Belkaar

Belkaar Feb 10, 2013

No, I shut it down and it was running with --detach-db=1 (if that matters)
Also: I tried this twice.

Belkaar commented Feb 10, 2013

No, I shut it down and it was running with --detach-db=1 (if that matters)
Also: I tried this twice.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Feb 10, 2013

Member

Ok, interesting. Can you try:

  • If you're able to compile on/for the RPi, can you try running the unit tests on it? (cd src; make -f makefile.unix test_bitcoin && ./test_bitcoin).
  • Are you able to sync (even if it's very slow) on the RPi directly?
  • Can you copy the files from an RPi to a desktop system, and continue the sync?
Member

sipa commented Feb 10, 2013

Ok, interesting. Can you try:

  • If you're able to compile on/for the RPi, can you try running the unit tests on it? (cd src; make -f makefile.unix test_bitcoin && ./test_bitcoin).
  • Are you able to sync (even if it's very slow) on the RPi directly?
  • Can you copy the files from an RPi to a desktop system, and continue the sync?
@Belkaar

This comment has been minimized.

Show comment
Hide comment
@Belkaar

Belkaar Feb 10, 2013

  1. Will do
  2. It syncs and runs fine on the rpi and on the x86 machine. You just can't copy the db from one to the other (both ways will give errors)
  3. See 2

Belkaar commented Feb 10, 2013

  1. Will do
  2. It syncs and runs fine on the rpi and on the x86 machine. You just can't copy the db from one to the other (both ways will give errors)
  3. See 2
@Belkaar

This comment has been minimized.

Show comment
Hide comment
@Belkaar

Belkaar Feb 13, 2013

The test build said "Running 89 test cases, no errors"

Belkaar commented Feb 13, 2013

The test build said "Running 89 test cases, no errors"

@rebroad

This comment has been minimized.

Show comment
Hide comment
@rebroad

rebroad Mar 20, 2013

Contributor

AFAIK the database files on linux and MS windows are also incompatible with each other. Are there any plans to make it so that the files can be used for the client irrespective of platform/architecture?

Contributor

rebroad commented Mar 20, 2013

AFAIK the database files on linux and MS windows are also incompatible with each other. Are there any plans to make it so that the files can be used for the client irrespective of platform/architecture?

@jgarzik

This comment has been minimized.

Show comment
Hide comment
@jgarzik

jgarzik Mar 20, 2013

Contributor

This is low priority, given that the database may be regenerated from publicly available data at any time.

Contributor

jgarzik commented Mar 20, 2013

This is low priority, given that the database may be regenerated from publicly available data at any time.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 8, 2014

Member

I just checked with master on my cubox-i (4 core, armv7l-linux-gnueabihf, debian jessie). Tried two scenarios:

  1. On ARM box
  • bootstrap the first ? blocks using blk00000.dat file from other install
  • shut down bitcoind
  • copy resulting bitcoin directory to AMD64 box
  • run bitcoind on AMD64 box
  1. On AMD64 box
  • bootstrap the first 142647 blocks using bootstrap.dat file
  • shut down bitcoind
  • copy resulting bitcoin directory to ARM box
  • run bitcoind on ARM box

Curiously, (1) seemed to work, whereas (2) manages to get past the leveldb validation but then starts to behave very erratic. It accepts blocks but never does anything with them, ie

2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
....

Starting with -checkblocks=0 (which should check the entire database) gives

2014-03-08 07:47:38 Verifying last 142647 blocks at level 3
2014-03-08 07:51:22 No coin database inconsistencies in last 4083 blocks (216726 transactions)
2014-03-08 07:51:23  block index          233301ms

After a -reindex it works fine.

Edit: re-trying with the copied block index and -checkblocks=0 -checklevel=3 gives the same kind of errors OP has:

2014-03-08 08:21:54 Verifying last 142647 blocks at level 3
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
....

The daemon hangs after this.

Member

laanwj commented Mar 8, 2014

I just checked with master on my cubox-i (4 core, armv7l-linux-gnueabihf, debian jessie). Tried two scenarios:

  1. On ARM box
  • bootstrap the first ? blocks using blk00000.dat file from other install
  • shut down bitcoind
  • copy resulting bitcoin directory to AMD64 box
  • run bitcoind on AMD64 box
  1. On AMD64 box
  • bootstrap the first 142647 blocks using bootstrap.dat file
  • shut down bitcoind
  • copy resulting bitcoin directory to ARM box
  • run bitcoind on ARM box

Curiously, (1) seemed to work, whereas (2) manages to get past the leveldb validation but then starts to behave very erratic. It accepts blocks but never does anything with them, ie

2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
2014-03-08 07:51:31 ProcessBlock: ACCEPTED
....

Starting with -checkblocks=0 (which should check the entire database) gives

2014-03-08 07:47:38 Verifying last 142647 blocks at level 3
2014-03-08 07:51:22 No coin database inconsistencies in last 4083 blocks (216726 transactions)
2014-03-08 07:51:23  block index          233301ms

After a -reindex it works fine.

Edit: re-trying with the copied block index and -checkblocks=0 -checklevel=3 gives the same kind of errors OP has:

2014-03-08 08:21:54 Verifying last 142647 blocks at level 3
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
2014-03-08 08:22:57 ERROR: DisconnectBlock() : undo data adding output to missing transaction
....

The daemon hangs after this.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 8, 2014

Member

OK this is curious. It it non-deterministic. Sometimes the verify passes, as pindexState gets 'stuck'

2014-03-08 08:45:20 Verifying pindexState=138564 pIndex=1
2014-03-08 08:45:20 No coin database inconsistencies in last 4083 blocks (216726 transactions)
2014-03-08 08:45:21  block index          244825ms

(this doesn't imply that the resulting index actually works, it doesn't)

Member

laanwj commented Mar 8, 2014

OK this is curious. It it non-deterministic. Sometimes the verify passes, as pindexState gets 'stuck'

2014-03-08 08:45:20 Verifying pindexState=138564 pIndex=1
2014-03-08 08:45:20 No coin database inconsistencies in last 4083 blocks (216726 transactions)
2014-03-08 08:45:21  block index          244825ms

(this doesn't imply that the resulting index actually works, it doesn't)

@whitslack

This comment has been minimized.

Show comment
Hide comment
@whitslack

whitslack Mar 24, 2014

I too tried to initialize a block index and chainstate database on my x86 system and then copy the files over to a Raspberry Pi. Since both systems are Linux, 32-bit, little-endian, I figured there would be no differences in the on-disk format. However, this is apparently not so. When Bitcoind starts verifying blocks, it immediately starts complaining about missing transactions. However, using the exact same files on the x86 machine (i.e., everything inside blocks/ and chainstate/ identical to what it was when starting on the RPi), Bitcoind starts up and verifies the blocks just fine.

Very disappointing that the on-disk format of these LevelDB databases differs between two architectures that are both 32-bit, little-endian.

whitslack commented Mar 24, 2014

I too tried to initialize a block index and chainstate database on my x86 system and then copy the files over to a Raspberry Pi. Since both systems are Linux, 32-bit, little-endian, I figured there would be no differences in the on-disk format. However, this is apparently not so. When Bitcoind starts verifying blocks, it immediately starts complaining about missing transactions. However, using the exact same files on the x86 machine (i.e., everything inside blocks/ and chainstate/ identical to what it was when starting on the RPi), Bitcoind starts up and verifies the blocks just fine.

Very disappointing that the on-disk format of these LevelDB databases differs between two architectures that are both 32-bit, little-endian.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Mar 24, 2014

Member

I think this is unintentional, as the LevelDB on-disk format is very well described (up to the byte level). My guess is a LevelDB bug that only occurs on ARM.

We should probably report this upstream.

Member

sipa commented Mar 24, 2014

I think this is unintentional, as the LevelDB on-disk format is very well described (up to the byte level). My guess is a LevelDB bug that only occurs on ARM.

We should probably report this upstream.

@whitslack

This comment has been minimized.

Show comment
Hide comment
@whitslack

whitslack Mar 24, 2014

I guess we would need a minimal test case that produces different data files on ARM than on x86.

whitslack commented Mar 24, 2014

I guess we would need a minimal test case that produces different data files on ARM than on x86.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Mar 24, 2014

Member

It may just be a bug in Bitcoin's LevelDB glue code or serialization code too...

Member

sipa commented Mar 24, 2014

It may just be a bug in Bitcoin's LevelDB glue code or serialization code too...

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj May 7, 2014

Member

This is really strange. I've indexed the first four block files both on ARM and AMD64.
To compare the resulting leveldb files I've built a py-leveldb against bitcoin's leveldb and started poking at the database files, comparing them in various ways.

# Compare two bitcoind's leveldbs
import leveldb

dbdira = 'armtest'
dbdirb = 'amd64test'

databases = ['/chainstate', '/blocks/index']

def compare_databases(cs1, cs2):
    ai = cs1.RangeIter()
    bi = cs2.RangeIter()

    n = 0
    err = 0
    aend = bend = False
    while True:
        try:
            a = ai.next()
        except StopIteration:
            aend = True
        try:
            b = bi.next()
        except StopIteration:
            bend = True
        if aend and bend:
            break # both at end of iteration
        if aend or bend or a != b:
            err += 1
        n += 1

    print "Compared %i records" % n
    print "Found %i differences" % err

def compare_databases2(cs1, cs2):
    '''
    Check that all keys of cs1 are in cs2
    and that the values match.
    '''
    ai = cs1.RangeIter()

    n = 0
    err = 0
    for (key,value) in ai:
        try:
            if cs2.Get(key) != value:
                err += 1
        except KeyError:
            err += 1
        n += 1

    print "Compared %i records" % n
    print "Found %i differences" % err

for db in databases:
    print "* Comparing %s" % db
    cs1 = leveldb.LevelDB(dbdira + db)
    cs2 = leveldb.LevelDB(dbdirb + db)
    print "** Check 1"
    compare_databases(cs1, cs2)
    print "** Check 2"
    compare_databases2(cs1, cs2)
    print "** Check 3"
    compare_databases2(cs2, cs1)
    print

Output:

* Comparing /chainstate
** Check 1
Compared 390331 records
Found 0 differences
** Check 2
Compared 390331 records
Found 0 differences
** Check 3
Compared 390331 records
Found 0 differences

* Comparing /blocks/index
** Check 1
Compared 142654 records
Found 0 differences
** Check 2
Compared 142654 records
Found 0 differences
** Check 3
Compared 142654 records
Found 0 differences

Seemingly, there are no differences at all in the data. I've also checked the blk* and rev*.dat files. They match to the byte. However, bitcoin on AMD64 refuses to work with the ARM files. I'm baffled.

Of course, bitcoin may have some other access pattern that brings to light a bug/difference in a lower level, but this is turning out to be elusive.

Also: Re-silvering the ARM-created databases by creating a new database and simply copying all keys/values, makes a database that can be used on AMD64.

Member

laanwj commented May 7, 2014

This is really strange. I've indexed the first four block files both on ARM and AMD64.
To compare the resulting leveldb files I've built a py-leveldb against bitcoin's leveldb and started poking at the database files, comparing them in various ways.

# Compare two bitcoind's leveldbs
import leveldb

dbdira = 'armtest'
dbdirb = 'amd64test'

databases = ['/chainstate', '/blocks/index']

def compare_databases(cs1, cs2):
    ai = cs1.RangeIter()
    bi = cs2.RangeIter()

    n = 0
    err = 0
    aend = bend = False
    while True:
        try:
            a = ai.next()
        except StopIteration:
            aend = True
        try:
            b = bi.next()
        except StopIteration:
            bend = True
        if aend and bend:
            break # both at end of iteration
        if aend or bend or a != b:
            err += 1
        n += 1

    print "Compared %i records" % n
    print "Found %i differences" % err

def compare_databases2(cs1, cs2):
    '''
    Check that all keys of cs1 are in cs2
    and that the values match.
    '''
    ai = cs1.RangeIter()

    n = 0
    err = 0
    for (key,value) in ai:
        try:
            if cs2.Get(key) != value:
                err += 1
        except KeyError:
            err += 1
        n += 1

    print "Compared %i records" % n
    print "Found %i differences" % err

for db in databases:
    print "* Comparing %s" % db
    cs1 = leveldb.LevelDB(dbdira + db)
    cs2 = leveldb.LevelDB(dbdirb + db)
    print "** Check 1"
    compare_databases(cs1, cs2)
    print "** Check 2"
    compare_databases2(cs1, cs2)
    print "** Check 3"
    compare_databases2(cs2, cs1)
    print

Output:

* Comparing /chainstate
** Check 1
Compared 390331 records
Found 0 differences
** Check 2
Compared 390331 records
Found 0 differences
** Check 3
Compared 390331 records
Found 0 differences

* Comparing /blocks/index
** Check 1
Compared 142654 records
Found 0 differences
** Check 2
Compared 142654 records
Found 0 differences
** Check 3
Compared 142654 records
Found 0 differences

Seemingly, there are no differences at all in the data. I've also checked the blk* and rev*.dat files. They match to the byte. However, bitcoin on AMD64 refuses to work with the ARM files. I'm baffled.

Of course, bitcoin may have some other access pattern that brings to light a bug/difference in a lower level, but this is turning out to be elusive.

Also: Re-silvering the ARM-created databases by creating a new database and simply copying all keys/values, makes a database that can be used on AMD64.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa May 31, 2014

Member

@laanwj Feel like trying again with #4161 and #4177 merged?

Member

sipa commented May 31, 2014

@laanwj Feel like trying again with #4161 and #4177 merged?

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 5, 2014

Member

OK, going to try

Member

laanwj commented Jun 5, 2014

OK, going to try

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 5, 2014

Member

I tried using the leveldb files from ARM on AMD64 and vice-versa. Same problem as before - alas, the paranoid checks do not catch this.

Member

laanwj commented Jun 5, 2014

I tried using the leveldb files from ARM on AMD64 and vice-versa. Same problem as before - alas, the paranoid checks do not catch this.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Jun 5, 2014

Member

Thanks for testing!

I wonder: do we get any kind of warning/error in chainstate/LOG ?

Member

sipa commented Jun 5, 2014

Thanks for testing!

I wonder: do we get any kind of warning/error in chainstate/LOG ?

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 6, 2014

Member

I'll check next time.

I'm working on recording a trace of bitcoind's access pattern, then replaying that against the two database instance and seeing where returned values start to diverge.

Member

laanwj commented Jun 6, 2014

I'll check next time.

I'm working on recording a trace of bitcoind's access pattern, then replaying that against the two database instance and seeing where returned values start to diverge.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 6, 2014

Member

I found the bug! Well at least I've narrowed it down a lot. The problem is in the filter_policy, which is set to leveldb::NewBloomFilterPolicy(10). Seemingly this filter policy data is not portable.
Commenting out that line makes the same block index files work on ARM and AMD64.

This explains why the database works fine after resilvering in my above post (and in general in py_leveldb, which doesn't use a filter policy...).

Member

laanwj commented Jun 6, 2014

I found the bug! Well at least I've narrowed it down a lot. The problem is in the filter_policy, which is set to leveldb::NewBloomFilterPolicy(10). Seemingly this filter policy data is not portable.
Commenting out that line makes the same block index files work on ARM and AMD64.

This explains why the database works fine after resilvering in my above post (and in general in py_leveldb, which doesn't use a filter policy...).

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 7, 2014

Member

The Hash function used for the bloom filter was returning different values based on character signedness in some edge cases. See bitcoin-core/leveldb-old#5 for fix.
Upstream issue: https://code.google.com/p/leveldb/issues/detail?id=237

Member

laanwj commented Jun 7, 2014

The Hash function used for the bloom filter was returning different values based on character signedness in some edge cases. See bitcoin-core/leveldb-old#5 for fix.
Upstream issue: https://code.google.com/p/leveldb/issues/detail?id=237

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jul 1, 2014

Member

It doesn't seem that the upstream issue is being picked up. Platform compatibility of the databases is likely very low-priority for them.

We could work around this completely on the bitcoind side by providing our own FilterPolicy() implementation and passing it in with the options. That's just a small class that hashes keys and tests against them: https://github.com/bitcoin/bitcoin/blob/master/src/leveldb/util/bloom.cc#L17 .

Member

laanwj commented Jul 1, 2014

It doesn't seem that the upstream issue is being picked up. Platform compatibility of the databases is likely very low-priority for them.

We could work around this completely on the bitcoind side by providing our own FilterPolicy() implementation and passing it in with the options. That's just a small class that hashes keys and tests against them: https://github.com/bitcoin/bitcoin/blob/master/src/leveldb/util/bloom.cc#L17 .

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj
Member

laanwj commented Oct 21, 2014

See #5093

@laanwj laanwj closed this Oct 27, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment