Remove txindex migration code #22626

maflcko · 2021-08-04T20:14:41Z

No supported version of Bitcoin Core used the legacy txindex, so all relevant nodes can be assumed to have upgraded. Thus, there is no need to keep this code any longer.

As a temporary courtesy, provide a one-time warning on how to free the disk space used by the legacy txindex.

Fixes #22615

Zero-1729 · 2021-08-04T20:33:43Z

Concept ACK

I'd suggest maybe adding a release note to inform users about this change, as was done in #13033 when txindex was moved. Maybe let them know about the removal and needing to use -reindex now. It could be added to this PR or a follow-up.

DrahtBot · 2021-08-04T21:12:10Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#22951 (consensus: move amount.h into consensus by fanquake)
#22242 (Move CBlockTreeDB to node/blockstorage by MarcoFalke)
#15606 (assumeutxo by jamesob)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

practicalswift · 2021-08-04T21:13:37Z

Concept ACK

jnewbery · 2021-08-04T21:59:24Z

Concept ACK

maflcko · 2021-08-05T07:07:15Z

I don't think this needs a release note because txindex is off by default and the number of txindex users upgrading from 0.16 (or earlier) to 23.x (or later) while reading the release notes, should be non-existent. Also for them, the targeted error message will explain what is going on.

theStack · 2021-08-05T11:55:16Z

Concept ACK

jamesob · 2021-08-05T14:16:13Z

Concept ACK

src/txdb.cpp

hebasto · 2021-08-20T13:31:35Z

Concept ACK.

Zero-1729 · 2021-08-20T14:06:49Z

Approach ACK

hebasto

Approach ACK fa78a79.

src/txdb.h

src/validation.h

hebasto

ACK fa20f81, tested on Linux Mint 20.2 (x86_64).

The old-fashion txindex was created by Bitcoin Core 0.16.3.

This PR, the first run:

...
2021-08-20T22:20:22Z init message: Verifying blocks…
2021-08-20T22:20:23Z Verifying last 6 blocks at level 3
2021-08-20T22:20:23Z [0%]...[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].
2021-08-20T22:20:23Z No coin database inconsistencies in last 6 blocks (288 transactions)
2021-08-20T22:20:23Z  block index           20085ms
2021-08-20T22:20:23Z Error: The block index db contains a legacy 'txindex'. To clear the occupied disk space, run a full -reindex, otherwise ignore this error. This error message will not be displayed again.
Error: The block index db contains a legacy 'txindex'. To clear the occupied disk space, run a full -reindex, otherwise ignore this error. This error message will not be displayed again.
2021-08-20T22:20:23Z Shutdown: In progress...
2021-08-20T22:20:23Z scheduler thread exit
2021-08-20T22:20:27Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) started
2021-08-20T22:20:27Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) completed (0.00s)
2021-08-20T22:20:30Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) started
2021-08-20T22:20:30Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) completed (0.00s)
2021-08-20T22:20:31Z Shutdown: done

No txindex related error messages during the following runs.

Zero-1729

crACK fa20f81

LGTM 🧉

theStack

Approach ACK fa20f81

Found some more includes / forward declarations in src/validation.h that seem unused and hence could be removed in the first commit:

#include <crypto/common.h> // for ReadLE64
#include <protocol.h> // For CMessageHeader::MessageStartChars
class CBlockUndo;
class CInv;
class CConnman;

maflcko · 2021-09-03T08:01:14Z

Found some more includes

Might do in the next force push or a new pull, whichever happens earlier.

luke-jr · 2021-09-12T16:32:31Z

I'm not sure we care, but what if a user upgrades to 23.0, syncs further a bit, then downgrades back to an older version that uses this txindex? Will it silently be broken for txindex purposes? Will it fail to sync properly (perhaps if an expected txindex entry is missing)?

maflcko · 2021-09-13T11:10:48Z

The txindex will be disabled for older version as soon as you start with a new version. See also the comment:

         // Disable legacy txindex and warn once about occupied disk space

laanwj · 2021-09-16T17:44:09Z

Code review ACK fa20f81

fadc4c7 test: Add txindex migration test (MarcoFalke) Pull request description: Test for bitcoin#22626 ACKs for top commit: theStack: Tested ACK fadc4c7 🌁 Tree-SHA512: fc7133ef52826bf0d4fa2ac72c3f1bed4a185ff7492396552ff2cbf6531b053238039211a710cbb949379c56875cd7715f1ed49a514dd3b3f1b46554e3d4bef5

It was put there thinking it would be needed by the test harness and suites, which work fine without it, and it must be removed before API test harness can run against bitcoin-core v23. See bitcoin/bitcoin#22626 (comment)

This change needs thorough review before being merged. See bitcoin-core Issue "Remove txindex migration code #22626": bitcoin/bitcoin#22626 And comment about change not having a release note: bitcoin/bitcoin#22626 (comment) See src: https://github.com/bitcoin/bitcoin/blob/v23.0/src/txdb.cpp#L35 For API test suites: This change is backward compatible for bitcoin-core v22. This change has not been tested against bitcoin-core v21, v20, v19. Change needs to be thoroughly tested with desktop UI.

This method hasn't been used since the txindex migration code has been removed (PR bitcoin#22626, commit fa20f81). Co-authored-by: furszy <matiasfurszyfer@protonmail.com>

…es `CDBIterator,CDBWrapper,CCoinsViewDBCursor` e4b4db5 refactor: remove unused method `CDBWrapper::CompactRange` (Sebastian Falbesoner) fb38c6e refactor: remove unused methods `{CDBIterator,CCoinsViewDBCursor}::GetValueSize()` (Sebastian Falbesoner) Pull request description: The `GetValueSize` methods haven't been used since the chainstate db cache has been switched from per-tx to per-txout model years ago (PR #10195, commit d342424). The `CompactRange` is unused since the txindex migration code was removed (PR bitcoin/bitcoin#22626, commit bitcoin/bitcoin@fa20f81). ACKs for top commit: fanquake: ACK e4b4db5 furszy: re-ACK e4b4db5 laanwj: Code review ACK e4b4db5 Tree-SHA512: 77da445fb70c744046263c6f2ddb05782b68e3d4b2ea604dd7c7dc73ce7c1f2d2b48ec68db4dcb03e35fc27488b99b0a420f6fa3d5b83d325c1708ed68e99e0a

…terator,CDBWrapper,CCoinsViewDBCursor` e4b4db5 refactor: remove unused method `CDBWrapper::CompactRange` (Sebastian Falbesoner) fb38c6e refactor: remove unused methods `{CDBIterator,CCoinsViewDBCursor}::GetValueSize()` (Sebastian Falbesoner) Pull request description: The `GetValueSize` methods haven't been used since the chainstate db cache has been switched from per-tx to per-txout model years ago (PR bitcoin#10195, commit d342424). The `CompactRange` is unused since the txindex migration code was removed (PR bitcoin#22626, commit bitcoin@fa20f81). ACKs for top commit: fanquake: ACK e4b4db5 furszy: re-ACK e4b4db5 laanwj: Code review ACK e4b4db5 Tree-SHA512: 77da445fb70c744046263c6f2ddb05782b68e3d4b2ea604dd7c7dc73ce7c1f2d2b48ec68db4dcb03e35fc27488b99b0a420f6fa3d5b83d325c1708ed68e99e0a

Summary: > Add missing includes and forward declarations, remove unused ones > doc: Fix validation typo > Remove txindex migration code PR description: > No supported version of Bitcoin used the legacy txindex, so all relevant nodes can be assumed to have upgraded. Thus, there is no need to keep this code any longer. > > As a temporary courtesy, provide a one-time warning on how to free the disk space used by the legacy txindex. This is a backport of [[bitcoin/bitcoin#22626 | core#22626]] Depends on D12242 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D12244

DrahtBot added UTXO Db and Indexes Validation labels Aug 4, 2021

DrahtBot mentioned this pull request Aug 5, 2021

Move CBlockTreeDB to node/blockstorage #22242

Closed

maflcko force-pushed the 2108-noTxindexMigrate branch from faa605b to fa78a79 Compare August 5, 2021 06:56

maflcko removed the Validation label Aug 5, 2021

DrahtBot mentioned this pull request Aug 7, 2021

assumeutxo #15606

Closed

18 tasks

fanquake added this to the 23.0 milestone Aug 12, 2021

laanwj reviewed Aug 18, 2021

View reviewed changes

src/txdb.cpp Show resolved Hide resolved

laanwj reviewed Aug 18, 2021

View reviewed changes

src/txdb.cpp Outdated Show resolved Hide resolved

hebasto reviewed Aug 20, 2021

View reviewed changes

src/txdb.h Show resolved Hide resolved

src/validation.h Show resolved Hide resolved

src/validation.h Show resolved Hide resolved

MarcoFalke added 3 commits August 20, 2021 16:55

Add missing includes and forward declarations, remove unused ones

fab8900

doc: Fix validation typo

fae8786

Remove txindex migration code

fa20f81

maflcko force-pushed the 2108-noTxindexMigrate branch from fa78a79 to fa20f81 Compare August 20, 2021 14:59

hebasto approved these changes Aug 20, 2021

View reviewed changes

Zero-1729 reviewed Aug 21, 2021

View reviewed changes

fanquake requested review from jnewbery, theStack and jamesob September 2, 2021 06:05

theStack approved these changes Sep 2, 2021

View reviewed changes

DrahtBot mentioned this pull request Sep 11, 2021

consensus: move amount.h into consensus #22951

Merged

laanwj merged commit 71bdf0b into bitcoin:master Sep 16, 2021

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Sep 19, 2021

Merge bitcoin#22626: Remove txindex migration code

9577a98

maflcko deleted the 2108-noTxindexMigrate branch September 20, 2021 10:19

This was referenced Sep 20, 2021

refactor: Remove unused validation includes #23044

Merged

test: Add txindex migration test #23046

Merged

ghubstan mentioned this pull request May 19, 2022

Remove legacy bitcoind txindex param from apitest harness bisq-network/bisq#6214

Merged

laanwj mentioned this pull request Jun 23, 2022

refactor: remove unused methods in classes CDBIterator,CDBWrapper,CCoinsViewDBCursor #25438

Merged

ghubstan mentioned this pull request Jul 3, 2022

Prepare Bisq for bitcoin-core v23 compatiblity bisq-network/bisq#6219

Merged

bitcoin locked and limited conversation to collaborators Oct 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove txindex migration code #22626

Remove txindex migration code #22626

maflcko commented Aug 4, 2021 •

edited

Loading

Zero-1729 commented Aug 4, 2021 •

edited

Loading

DrahtBot commented Aug 4, 2021 •

edited

Loading

practicalswift commented Aug 4, 2021

jnewbery commented Aug 4, 2021

maflcko commented Aug 5, 2021

theStack commented Aug 5, 2021

jamesob commented Aug 5, 2021

hebasto commented Aug 20, 2021

Zero-1729 commented Aug 20, 2021

hebasto left a comment

hebasto left a comment •

edited

Loading

Zero-1729 left a comment

theStack left a comment

maflcko commented Sep 3, 2021

luke-jr commented Sep 12, 2021

maflcko commented Sep 13, 2021

laanwj commented Sep 16, 2021

Remove txindex migration code #22626

Remove txindex migration code #22626

Conversation

maflcko commented Aug 4, 2021 • edited Loading

Zero-1729 commented Aug 4, 2021 • edited Loading

DrahtBot commented Aug 4, 2021 • edited Loading

Conflicts

practicalswift commented Aug 4, 2021

jnewbery commented Aug 4, 2021

maflcko commented Aug 5, 2021

theStack commented Aug 5, 2021

jamesob commented Aug 5, 2021

hebasto commented Aug 20, 2021

Zero-1729 commented Aug 20, 2021

hebasto left a comment

Choose a reason for hiding this comment

hebasto left a comment • edited Loading

Choose a reason for hiding this comment

Zero-1729 left a comment

Choose a reason for hiding this comment

theStack left a comment

Choose a reason for hiding this comment

maflcko commented Sep 3, 2021

luke-jr commented Sep 12, 2021

maflcko commented Sep 13, 2021

laanwj commented Sep 16, 2021

maflcko commented Aug 4, 2021 •

edited

Loading

Zero-1729 commented Aug 4, 2021 •

edited

Loading

DrahtBot commented Aug 4, 2021 •

edited

Loading

hebasto left a comment •

edited

Loading