Switch down from mdbx v0.10.5 to v0.10.1 caused error

[AskAlexSharov]

In next scenario:

• create db with v0.10.5, do some writes, close db
• open db with v0.10.1, do some writes, close db
• open db with v0.10.1 (same for v0.10.5 here), it will crush, and mdbx_chk shows error:

mdbx_chk -vvvvv
mdbx_chk v0.10.1-5-g18bc28be (2021-06-10T02:47:40+03:00, T-7a0b5ff57eb84895080d5e117d9d56ec9704fa9a)
Running for /Users/alex.sharov/data/test/nodes/eth66 in 'read-only' mode...
   readahead ON 0..10
 - monopolistic mode
 - current boot-id 8e25c6cb9a9bd29d-2f1735a090e242dd
 - pagesize 4096 (4096 system), max keysize 1980..2022, max readers 116
 - mapsize 268435456 (256.00 Mb)
 - dynamic datafile: 12288 (12.00 Kb) .. 268435456 (256.00 Mb), +2147483648 (2.00 Gb), -4096 (4.00 Kb)
 - current datafile: 40960 (40.00 Kb), 10 pages
 - meta-0: steady txn#7, stay
 - meta-1: steady txn#8, head
 - meta-2: steady txn#6, tail
 - performs check for meta-pages clashes
 - performs full check recent-txn-id with meta-pages
 - transactions: recent 8, latter reader 8, lag 0
Traversal b-tree by txn#8...
     meta-span 0[3] of @META: header 60, entries 3, payload 624, unused 11604, deep 0
     leaf-page 5 of @GC: header 24, entries 2, payload 56, unused 4016, deep 0
     leaf-page 3 of @MAIN: header 22, entries 1, payload 61, unused 4013, deep 0
 - found 'Inode' area
     leaf-page 4 of Inode: header 66, entries 23, payload 1615, unused 2415, deep 1
 - pages: walked 6, left/unused 4
     @GC: subtotal 1, leaf 1
     @MAIN: subtotal 1, leaf 1
     @META: subtotal 3
     Inode: subtotal 1, leaf 1
 - usage: total 24576 bytes, payload 2528 (10.3%), unused 22048 (89.7%)
     @GC: subtotal 4096 bytes (16.7%), payload 80 (2.0%), unused 4016 (98.0%)
     @MAIN: subtotal 4096 bytes (16.7%), payload 83 (2.0%), unused 4013 (98.0%)
     @META: subtotal 12288 bytes (50.0%), payload 684 (5.6%), unused 11604 (94.4%)
     Inode: subtotal 4096 bytes (16.7%), payload 1681 (41.0%), unused 2415 (59.0%)
 - summary: average fill 10.3%, 0 problems
Processing '@MAIN'...
 - key-value kind: usual-key => single-value, flags: none (0x00), dbi-id 1
 - last modification txn#6
 - page size 4096, entries 1
 - b-tree depth 1, pages: branch 0, leaf 1, overflow 0
 ! corrupted leaf-page #3, mod-txnid 8
 ! invalid page txnid (8) for parent-page' txnid (6)
 ! mdbx_cursor_get() failed, error 1 Operation not permitted
     entry #0: different number of entries (0 != 1)
 - problems: different number of entries (1)
 - summary: 0 records, 0 dups, 0 key's bytes, 0 data's bytes, 1 problems
 ! abort processing '@GC' due to a previous error
 - space: 65536 total pages, backed 10 (0.0%), allocated 10 (0.0%), remained 65526 (100.0%), used 6 (0.0%), gc 0 (0.0%), detained 0 (0.0%), reclaimable 0 (0.0%), available 65526 (100.0%)
Total 4 errors are detected, elapsed 0.001 seconds.

db 40Kb: db.zip

I can reproduce it easily.

(FYI: i used a bit patched v0.10.1 - to increase MDBX_READERS_LIMIT: erigontech/mdbx-go@v0.19.0...v0.19.2)

[erthink]

This is not a bug, but the feature...

Unfortunately, the v0.10.5 accidentally comes not full-compatible with previous releases:

• v0.10.5 can read/processing DBs created by previous releases, i.e. the backward-compatibility is provided;
• but previous releases are unable read/processing DBs created by v0.10.5, i.e. the forward-compatibility is broken for previous releases.

This cannot be fixed, as it requires fixing past versions, which as a result we will get already existing v0.10.5.
however, there are problems with the numbering of version 0.10.5, because if compatibility is broken, then it should be designated as v0.11.0 - I'll think about what to do.

[erthink]

At a whole this gave a false-positive diagnosis of database corruption - I'll think about what to do.

@AskAlexSharov, any ideas are welcome.

[AskAlexSharov]

1. If it’s not corruption - Is it possible to auto-recover such db if v0.10.5 used? (I mean: create db by v0.10.5, switch to v0.10.1 will fail, switch back to v0.10.5 will recover)?
2. Can I do this from APP side?
3. Why v0.10.1 fail only on second opening of db (works after first opening)?
4. My app has logic of db schema versioning - i can increase major version there and downgraded APP will reject to start. But, it also storing version inside db, and will open db to read it :-) probably I need store mdbx schema version outside of mdbx.
5. Maybe mdbx can do same logic: If db created/last-used by major version X, then mdbx of major version < X can reject to open/update it?
6. I have db which can't recover by v0.10.5 mdbx_chk. I did mdbx_chk -v -0 -t -w and got = Performing turn to the specified meta-page (0) due to successful check!, but can't open this db by v0.10.5. Kind of "false-positive".

[erthink]

Basically, seems just the single solution - don't mix new and old versions since shit bugs happens.

[AskAlexSharov]

How? (I mean - we don't control our installations, users can downgrade us). By storing last used MDBX version outside of mdbx and check it before open db?

[erthink]

I will change the DB format' signature and release version 0.11:

• new version will be able to read DB with old signature, but will silently upgrade ones by first commit.
• previously versions will be unable to read/write DB with the new signature (and will be unable to made it false-corrupted).

[AskAlexSharov]

love it

[erthink]

https://github.com/erthink/libmdbx/tree/v0.11.0

[AskAlexSharov]

• If create db by v0.11.0 and switch to earlier version, got expected: MDBX_VERSION_MISMATCH
• If create db by v0.10.1 and switch to v0.11.0 (do some write work here), then switch back to v0.10.1, got: corruption on second open (same as before).

Looks like signature update doesn't happen when switching to new version

[erthink]

Sorry, I didn't finish the work but I closed this issue by mistake.

[AskAlexSharov]

it works