Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blockstorage: Drop legacy -txindex check #28195

Merged
merged 5 commits into from Sep 5, 2023

Conversation

maflcko
Copy link
Member

@maflcko maflcko commented Aug 1, 2023

The only reason for the check was to print a warning about an increase in storage use. Now that 22.x is EOL and everyone should have migrated (or decided to not care about storage use), remove the check.

Also, a move-only commit is included. (Rebased from #22242)

@DrahtBot
Copy link
Contributor

DrahtBot commented Aug 1, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK TheCharlatan, stickies-v
Concept ACK theStack

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #28392 (test: Use pathlib over os path by ns-xvrn)
  • #28311 ([WIP] BIP300 (Drivechains) consensus-level logic by luke-jr)
  • #28226 (util/blockstorage: Add TRACE_RAII, slightly faster -reindex-chainstate with CBufferedFile by martinus)
  • #28051 (Get rid of shutdown.cpp/shutdown.h, use SignalInterrupt directly by ryanofsky)
  • #24230 (indexes: Stop using node internal types and locking cs_main, improve sync logic by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot DrahtBot changed the title blockstorage: Drop legacy -txindex check blockstorage: Drop legacy -txindex check Aug 1, 2023
Copy link
Contributor

@theStack theStack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

There are still two instances where blocks_path can be used (sometimes it's quite annoying that python also allows single quote strings):

test/functional/feature_pruning.py:        self.prunedir = os.path.join(self.nodes[2].chain_path, 'blocks', '')
test/functional/wallet_backup.py:            shutil.rmtree(os.path.join(self.nodes[2].chain_path, 'blocks'))

With my very limited sed skills, I'd just add another line
sed -i "s|].chain_path, 'blocks'|].blocks_path|g" $(git grep -l chain_path)
to the scripted-diff (probably also possible with only one).

MarcoFalke added 4 commits August 1, 2023 15:26
-BEGIN VERIFY SCRIPT-
  sed -i 's|].chain_path, .blocks.|].blocks_path|g' $(git grep -l chain_path)
-END VERIFY SCRIPT-
The block index (CBlockTreeDB) is required to write and read blocks, so
move it to blockstorage. This allows to drop the txdb.h include from
`node/blockstorage.h`.

Can be reviewed with:

--color-moved=dimmed-zebra  --color-moved-ws=ignore-all-space
Can be reviewed with --word-diff-regex=.
@maflcko
Copy link
Member Author

maflcko commented Aug 1, 2023

Thanks, fixed up the scripted-diff

@fanquake
Copy link
Member

fanquake commented Aug 1, 2023

cc @TheCharlatan

-BEGIN VERIFY SCRIPT-
 sed -i 's|CBlockTreeDB|BlockTreeDB|g' $( git grep -l CBlockTreeDB )
-END VERIFY SCRIPT-
@TheCharlatan
Copy link
Contributor

ACK fae4055, though I lack historical context to really judge the second commit fa86855.

I was curious if it might be possible to easily make one of the CBlockIndex* in BlockTreeDB a CBlockIndex&, but the change became a bit sprawling.

@maflcko
Copy link
Member Author

maflcko commented Aug 15, 2023

though I lack historical context to really judge the second commit

The basic idea is that the legacy txindex was stored inside the block index db. Then a new src/index/ infrastructure was added and there was migration code. Then, the migration code was removed and a warning was added that a previous version is needed (or a full -reindex) to do the migration. Now that all those previous versions are EOL, and thus there should no one be left, who needs a migration, the warning can be removed. Even if someone is left behind, the worst that can happen is they'll be left with "dangling" db entries (using storage). And they can at any time clear the storage with a full -reindex.

Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK fae4055

The upgrade path when coming from legacy txindex is straightforward and clear enough, no need to keep this extra check around for such an old version imo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we at some point sunset this test completely? Given that the txindex construction is completely separate from BlockTreeDB, I don't think there's really any compatibility testing happening anymore? It's good to have the test around for this pull, but I'm not sure when it'll be useful in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Seems fine to remove it in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean it is still checking that the dangling BlockTreeDB entries like t T, or txindex do not cause any issues, but at some point we can drop this test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thx. I quite like adding "# TODO" comments to make it easier to find future cleanup items, but I don't really know how I'd phrase the "when" clause so no concrete suggestion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe create an issue (after merge) to remove it after +1 release of this being merged?

@maflcko
Copy link
Member Author

maflcko commented Sep 5, 2023

rfm, or is anything left to be done here?

@fanquake fanquake merged commit ecab855 into bitcoin:master Sep 5, 2023
15 checks passed
@maflcko maflcko deleted the 2307-test-blocks- branch September 5, 2023 10:41
Frank-GER pushed a commit to syscoin/syscoin that referenced this pull request Sep 8, 2023
fanquake added a commit that referenced this pull request Oct 30, 2023
897d6dd Remove feature_txindex_compatibility.py in V27 (Brandon Odiwuor)

Pull request description:

  Fixes #28421, see [#28195 (comment)](fa86855#r1311494362)

  Remove feature_txindex_compatibility.py in V27, follow up to #28195 being merged which is included in v26

ACKs for top commit:
  maflcko:
    lgtm ACK 897d6dd
  theStack:
    ACK 897d6dd
  stickies-v:
    ACK 897d6dd

Tree-SHA512: 53102d39f6fdbdcf1bb13b6feb2f446b0e9e8e3fe294c0e6fe37e7731713fb9fd5b048e19b6edf80579f5edbcf762b51d56d57bdcda67ec3527706891dc3572b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants