New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cursor to iterate over utxo set, use this in `gettxoutsetinfo` #7756

Merged
merged 1 commit into from Apr 15, 2016

Conversation

Projects
None yet
2 participants
@laanwj
Member

laanwj commented Mar 28, 2016

I need this for something else, but I think this is simply a better API. I also vaguely remember this has been discussed before.

  • Add a method Cursor() to CCoinsView that returns a cursor which can be used to iterate over the whole UTXO set.
  • Implement GetUTXOStats in terms of this, remove GetStats() method on CCoinsView.
  • Change gettxoutsetinfo RPC to use new GetUTXOStats function.

This is not yet enough to break the dependency of txdb.cpp/h on main (there's CDiskTxPos to cope with, for example), but it is a step toward that (see #7815).

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 31, 2016

Member

Travis fail is unrelated (#7470 (comment)), respinning.

Member

laanwj commented Mar 31, 2016

Travis fail is unrelated (#7470 (comment)), respinning.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Apr 6, 2016

Member

Rebased

Member

laanwj commented Apr 6, 2016

Rebased

@@ -45,7 +45,7 @@ bool CCoinsView::GetCoins(const uint256 &txid, CCoins &coins) const { return fal
bool CCoinsView::HaveCoins(const uint256 &txid) const { return false; }
uint256 CCoinsView::GetBestBlock() const { return uint256(); }
bool CCoinsView::BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock) { return false; }
bool CCoinsView::GetStats(CCoinsStats &stats) const { return false; }
CCoinsViewCursor *CCoinsView::Cursor() const { return 0; }

This comment has been minimized.

@sipa

sipa Apr 6, 2016

Member

Using NULL would be clearer as it's a pointer.

@sipa

sipa Apr 6, 2016

Member

Using NULL would be clearer as it's a pointer.

This comment has been minimized.

@laanwj

laanwj Apr 6, 2016

Member

Bjarne Stroustrup taught me not to use NULL in c++ code: http://www.stroustrup.com/bs_faq2.html#null
I don't care deeply about this though.

@laanwj

laanwj Apr 6, 2016

Member

Bjarne Stroustrup taught me not to use NULL in c++ code: http://www.stroustrup.com/bs_faq2.html#null
I don't care deeply about this though.

This comment has been minimized.

@sipa

sipa Apr 6, 2016

Member

Interesting. Comment retracted.

@sipa

sipa Apr 6, 2016

Member

Interesting. Comment retracted.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Apr 6, 2016

Member
  • rpc: Add method gettxoutsetinfo2 that uses new Cursor method

That seems outdated.

Member

sipa commented Apr 6, 2016

  • rpc: Add method gettxoutsetinfo2 that uses new Cursor method

That seems outdated.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Apr 6, 2016

Member

That seems outdated.

Yes, I've removed that in the rebase, I had multiple commits with two getutxosetinfo in an intermediate step but that was annoying to rebase over #7766.

Member

laanwj commented Apr 6, 2016

That seems outdated.

Yes, I've removed that in the rebase, I had multiple commits with two getutxosetinfo in an intermediate step but that was annoying to rebase over #7766.

@sipa

View changes

Show outdated Hide outdated src/txdb.cpp
@sipa

View changes

Show outdated Hide outdated src/txdb.cpp
@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Apr 9, 2016

Member

Pushed a commit that should address @sipa's comment: get rid of loop, no more chance of hiding failures.

Member

laanwj commented Apr 9, 2016

Pushed a commit that should address @sipa's comment: get rid of loop, no more chance of hiding failures.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Apr 12, 2016

Member

utACK 299fad9

Member

sipa commented Apr 12, 2016

utACK 299fad9

txdb: Add Cursor() method to CCoinsView to iterate over UTXO set
Add a method Cursor() to CCoinsView that returns a cursor which can be
used to iterate over the whole UTXO set.

- rpc: Change gettxoutsetinfo to use new Cursor method

- txdb: Remove GetStats method - Now that GetStats is implemented in
  terms of Cursor, remove it.
@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Apr 15, 2016

Member

Squashed eacf28c 299fad9 into 509cb00

Member

laanwj commented Apr 15, 2016

Squashed eacf28c 299fad9 into 509cb00

@laanwj laanwj merged commit 509cb00 into bitcoin:master Apr 15, 2016

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

laanwj added a commit that referenced this pull request Apr 15, 2016

Merge #7756: Add cursor to iterate over utxo set, use this in `gettxo…
…utsetinfo`

509cb00 txdb: Add Cursor() method to CCoinsView to iterate over UTXO set (Wladimir J. van der Laan)

laanwj added a commit to laanwj/bitcoin that referenced this pull request Apr 18, 2016

txdb: Fix assert crash in new UTXO set cursor
Remove the mistaken assumption that GetKey returning false signifies
an internal database issue. It will return false when the key cannot
be deserialized into the (char,uint256) stanza, which indicates
that the cursor has reached a different kind of key.

Fixes bug #7890 introduced in #7756.

laanwj added a commit to laanwj/bitcoin that referenced this pull request Sep 28, 2016

rest: Stream entire utxo set
This builds on #7756 and

- Adds a streaming API to the HTTP server. This allows streaming data to
  the client chunk by chunk, which is useful when not the entire data is
  available at once or it is huge and wouldn't fit (efficiently) in
  memory.

- Allows downloading the entire UTXO set through `/rest/utxoset`. This
  is a raw dump of all outputs, the state normally hashed by
  `gettxoutsetinfo`. The dump is performed in the background by making
  use of leveldb snapshotting, so without keeping cs_main locked.

    - This can be useful for analysis purposes if you don't want to mess
      with bitcoin core's database

    - Filename (via content-disposition) is
      `utxoset-<height>-<bestblockhash>.dat`. Also a custom
      `X-Best-Block` and `X-Block-Height` header is added.

codablock added a commit to codablock/dash that referenced this pull request Sep 16, 2017

Merge #7756: Add cursor to iterate over utxo set, use this in `gettxo…
…utsetinfo`

509cb00 txdb: Add Cursor() method to CCoinsView to iterate over UTXO set (Wladimir J. van der Laan)

codablock added a commit to codablock/dash that referenced this pull request Sep 19, 2017

Merge #7756: Add cursor to iterate over utxo set, use this in `gettxo…
…utsetinfo`

509cb00 txdb: Add Cursor() method to CCoinsView to iterate over UTXO set (Wladimir J. van der Laan)

codablock added a commit to codablock/dash that referenced this pull request Sep 27, 2017

Merge #7756: Add cursor to iterate over utxo set, use this in `gettxo…
…utsetinfo`

509cb00 txdb: Add Cursor() method to CCoinsView to iterate over UTXO set (Wladimir J. van der Laan)

codablock added a commit to codablock/dash that referenced this pull request Oct 12, 2017

Merge #7756: Add cursor to iterate over utxo set, use this in `gettxo…
…utsetinfo`

509cb00 txdb: Add Cursor() method to CCoinsView to iterate over UTXO set (Wladimir J. van der Laan)

codablock added a commit to codablock/dash that referenced this pull request Oct 19, 2017

Merge #7756: Add cursor to iterate over utxo set, use this in `gettxo…
…utsetinfo`

509cb00 txdb: Add Cursor() method to CCoinsView to iterate over UTXO set (Wladimir J. van der Laan)

UdjinM6 added a commit to UdjinM6/dash that referenced this pull request Nov 8, 2017

Merge #7756: Add cursor to iterate over utxo set, use this in `gettxo…
…utsetinfo`

509cb00 txdb: Add Cursor() method to CCoinsView to iterate over UTXO set (Wladimir J. van der Laan)

lateminer added a commit to lateminer/bitcoin that referenced this pull request Jan 12, 2018

txdb: Fix assert crash in new UTXO set cursor
Remove the mistaken assumption that GetKey returning false signifies
an internal database issue. It will return false when the key cannot
be deserialized into the (char,uint256) stanza, which indicates
that the cursor has reached a different kind of key.

Fixes bug #7890 introduced in #7756.

str4d added a commit to str4d/zcash that referenced this pull request Jan 27, 2018

Break circular dependency main ↔ txdb
Break the circular dependency between main and txdb by:

- Moving `CBlockFileInfo` from `main.h` to `chain.h`. I think this makes
  sense, as the other block-file stuff is there too.

- Moving `CDiskTxPos` from `main.h` to `txdb.h`. This type seems
  specific to txdb.

- Pass a functor `insertBlockIndex` to `LoadBlockIndexGuts`. This leaves
  it up to the caller how to insert block indices.

Zcash: This does not actually break the circular dependency for us yet, as we
still need to pull in bitcoin/bitcoin#7756 and bitcoin/bitcoin#7904.

str4d added a commit to str4d/zcash that referenced this pull request Mar 9, 2018

Break circular dependency main ↔ txdb
Break the circular dependency between main and txdb by:

- Moving `CBlockFileInfo` from `main.h` to `chain.h`. I think this makes
  sense, as the other block-file stuff is there too.

- Moving `CDiskTxPos` from `main.h` to `txdb.h`. This type seems
  specific to txdb.

- Pass a functor `insertBlockIndex` to `LoadBlockIndexGuts`. This leaves
  it up to the caller how to insert block indices.

Zcash: This does not actually break the circular dependency for us yet, as we
still need to pull in bitcoin/bitcoin#7756 and bitcoin/bitcoin#7904.

str4d added a commit to str4d/zcash that referenced this pull request Mar 12, 2018

Break circular dependency main ↔ txdb
Break the circular dependency between main and txdb by:

- Moving `CBlockFileInfo` from `main.h` to `chain.h`. I think this makes
  sense, as the other block-file stuff is there too.

- Moving `CDiskTxPos` from `main.h` to `txdb.h`. This type seems
  specific to txdb.

- Pass a functor `insertBlockIndex` to `LoadBlockIndexGuts`. This leaves
  it up to the caller how to insert block indices.

Zcash: This does not actually break the circular dependency for us yet, as we
still need to pull in bitcoin/bitcoin#7756 and bitcoin/bitcoin#7904.

str4d added a commit to str4d/zcash that referenced this pull request Mar 15, 2018

Break circular dependency main ↔ txdb
Break the circular dependency between main and txdb by:

- Moving `CBlockFileInfo` from `main.h` to `chain.h`. I think this makes
  sense, as the other block-file stuff is there too.

- Moving `CDiskTxPos` from `main.h` to `txdb.h`. This type seems
  specific to txdb.

- Pass a functor `insertBlockIndex` to `LoadBlockIndexGuts`. This leaves
  it up to the caller how to insert block indices.

Zcash: This does not actually break the circular dependency for us yet, as we
still need to pull in bitcoin/bitcoin#7756 and bitcoin/bitcoin#7904.

str4d added a commit to str4d/zcash that referenced this pull request Mar 30, 2018

Break circular dependency main ↔ txdb
Break the circular dependency between main and txdb by:

- Moving `CBlockFileInfo` from `main.h` to `chain.h`. I think this makes
  sense, as the other block-file stuff is there too.

- Moving `CDiskTxPos` from `main.h` to `txdb.h`. This type seems
  specific to txdb.

- Pass a functor `insertBlockIndex` to `LoadBlockIndexGuts`. This leaves
  it up to the caller how to insert block indices.

Zcash: This does not actually break the circular dependency for us yet, as we
still need to pull in bitcoin/bitcoin#7756 and bitcoin/bitcoin#7904.

str4d added a commit to str4d/zcash that referenced this pull request Apr 4, 2018

Break circular dependency main ↔ txdb
Break the circular dependency between main and txdb by:

- Moving `CBlockFileInfo` from `main.h` to `chain.h`. I think this makes
  sense, as the other block-file stuff is there too.

- Moving `CDiskTxPos` from `main.h` to `txdb.h`. This type seems
  specific to txdb.

- Pass a functor `insertBlockIndex` to `LoadBlockIndexGuts`. This leaves
  it up to the caller how to insert block indices.

Zcash: This does not actually break the circular dependency for us yet, as we
still need to pull in bitcoin/bitcoin#7756 and bitcoin/bitcoin#7904.

str4d added a commit to str4d/zcash that referenced this pull request May 31, 2018

Break circular dependency main ↔ txdb
Break the circular dependency between main and txdb by:

- Moving `CBlockFileInfo` from `main.h` to `chain.h`. I think this makes
  sense, as the other block-file stuff is there too.

- Moving `CDiskTxPos` from `main.h` to `txdb.h`. This type seems
  specific to txdb.

- Pass a functor `insertBlockIndex` to `LoadBlockIndexGuts`. This leaves
  it up to the caller how to insert block indices.

Zcash: This does not actually break the circular dependency for us yet, as we
still need to pull in bitcoin/bitcoin#7756 and bitcoin/bitcoin#7904.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment