Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newsletters: add 64 (2019-09-18) #226

Merged
merged 2 commits into from Sep 18, 2019

Conversation

@harding
Copy link
Contributor

@harding harding commented Sep 14, 2019

Happy issue 2^6!

@jnewbery
Copy link
Contributor

@jnewbery jnewbery commented Sep 14, 2019

Happy issue 2^6!

Yay! A round number!

Copy link
Contributor

@jnewbery jnewbery left a comment

ACK 65cd259

My only comment is that I think we should drop the SPV fraud proofs without a soft fork piece. Everything else is 💯


## News

- **SPV fraud proofs without a soft fork:** Ruben Somsen started a
Copy link
Contributor

@jnewbery jnewbery Sep 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My opinion is that we shouldn't cover this: the proposal is too immature, there are too many unanswered questions, and I doubt it will get implemented. I've only skimmed the proposal after reading your summary, but I had the following questions:

  • how does the algorithm verify coinbase maturity of spends? Does the utreexo TXO model include coinbaseness and height? (I had a look at the utreexo paper and implementation and couldn't find an answer, although I guess it must do, otherwise utreexo wouldn't work
  • how does the fork-finding algorithm work? You write "In the worst case, this increases the cost of the fraud proof by a few megabytes per lying peer.", but it seems to me that a malicious adversary could make an SPV download lots of utreexo commitments to find the fork point, or stall the node.
  • does this require all honest nodes to store utreexo commitments for all blocks in order to serve them to SPVs in the case of a fork?
  • this requires SPV clients to be able to validate all the consensus rules for blocks and transactions in the case of a fork. Can we expect clients to do that?

Copy link
Contributor Author

@harding harding Sep 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I think you answered this yourself.

  2. Worst case would be the SPV client would need to download one commitment for every block in the block chain, e.g. 32 MB for 32-byte commitments on a chain with a million blocks. Practically, I'd expect SPV clients to ship with checkpoints (many do already) that wouldn't believe a node that claimed a fork further back than a few months, or about 1 MB of commitments. (Even more practically, though perhaps slightly risky, an SPV client may refuse to believe in forks over a week long, or about 32 kB of commitments.) Clients can detect stalling and disconnect the offending node; Bitcoin users should be used to waiting an hour or more for ~6 confirmations, so this process doesn't need to be be super fast.

  3. It's worse than storing commitments (those are just a small hash), I think they need to store (1) the UTXOs being spent in a block (i.e. undo data), (2) the utreexo partial merkle branches connecting the UTXOs to the utreexo merkle root, (3) the block itself. My very rough guess is that this would require about double the disk space of a current archival full node. That said, I don't see any complexity to storing and distributing this information (like BIP158 filters, it should be compute-once and store-on-disk) and if there's an expectation that SPV clients won't believe in forks more than n blocks old, then a node serving these fraud proofs only needs to store n blocks worth of this data. All that said, this was my main concern---that too few full node operators will care to generate and serve this data, making it easy to sybil SPV clients needing that data and so making the "one honest peer" assumption even easier to violate than it is now.

  4. This is explicitly mentioned in the newsletter description. But, yeah, I don't expect SPV clients to actually do that (though some developers have been claiming that they'll do stuff like that to make SPV secure in the future).

In general, I was pretty dubious about this proposal myself and my initial response was to draft an email for the mailing list describing its flaws. However, each flaw I thought I found was one I also found a reasonable solution for, even if I think some of them are unlikely to be implemented. So I wrote this description because SPV fraud proofs under reasonable assumptions without a soft fork seems cool. However, if after reading this too-long comment reply, you still think the proposal is too immature, I'm happy to drop it. This week's newsletter is long enough already.

Copy link
Contributor Author

@harding harding Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I just replied to Somsen on list with several of these points (plus others) so that he's aware of the possible criticism.

at the time the block was created. The client can then perform the
other consensus checks on block validity which don't depend on state
(e.g., all the fields in its block header are correct, none of its
outputs spends more than 21 million BTC, etc...). With all that
Copy link
Contributor

@jnewbery jnewbery Sep 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add "all scripts and signatures are valid" as an example of a non-contextual check.

Copy link
Contributor Author

@harding harding Sep 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That requires UTXO entries.

transition from a state they both agreed upon to a state they find
disagreeable and can perform the same procedure. In the worst case,
this increases the cost of the fraud proof by a few megabytes per
lying peer.
Copy link
Contributor

@jnewbery jnewbery Sep 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"this increases the cost of the fraud proof by a few megabytes per lying peer." How did you get to this figure? I don't see it in the mailing list post.

Copy link
Contributor Author

@harding harding Sep 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mailing list post has estimates for the cost of verifying the state transition from the pre-fork block to one of the fork blocks:

1. block N+1 from the most PoW chain (~1-2MB)
2. the utreexo hash commitment inside of block N (e.g. a merkle path
to the coinbase)
3. the utreexo merkle proofs which prove that all inputs of N+1 are
part of the UTXO set (~1MB)

This is the same procedure except that it's performed on an earlier block, therefore I expect it to have the same costs.

Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"this increases"... perhaps insert a word/phrase describing what causes the increase and how?

@jnewbery
Copy link
Contributor

@jnewbery jnewbery commented Sep 16, 2019

I've added a commit that removes the SPV fraud proofs section. We shouldn't squash that, so that we have it in the git history in case we want to use it in a future issue.

Copy link
Collaborator

@jonatack jonatack left a comment

What a newsletter! Feel free to ignore any of the comments below.

fork. Then the client requests each of its peers commit to the UTXO
state from the block before the fork via a utreexo merkle root.
After that commitment, the client downloads the fork block, copies
of the UTXOs it spends, and the partial merkle branches connecting
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"copies" in line 37 could be a either a verb or a noun, which caused this dense reader to initially think that "of" should be removed. Perhaps s/block, copies of/block,/

at the time the block was created. The client can then perform the
other consensus checks on block validity which don't depend on state
(e.g., all the fields in its block header are correct, none of its
outputs spends more than 21 million BTC, etc...). With all that
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps s/With all that/With that/

In case any peers disagree about the value of the utreexo state at
the pre-fork block, the SPV client can work with them to find a
transition from a state they both agreed upon to a state they find
disagreeable and can perform the same procedure. In the worst case,
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"a transition from a state they both agreed upon to a state they find disagreeable"

does this mean the point at which the disagreement arose?

transition from a state they both agreed upon to a state they find
disagreeable and can perform the same procedure. In the worst case,
this increases the cost of the fraud proof by a few megabytes per
lying peer.
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"this increases"... perhaps insert a word/phrase describing what causes the increase and how?

## Bitcoin Edge Dev++

The Scaling Bitcoin conference was preceded by the community-organized
Bitcoin Edge Dev++ training sessions. Recording of the sessions are
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recordings?


- [Bitcoin Core #16787][] extends the `getpeerinfo` and `getnetworkinfo`
RPCs with a new field that decodes the services bitfield which
indicates what services the peer or local node offer. this is in
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/offer. this/offers. This/

- [Bitcoin Core #16787][] extends the `getpeerinfo` and `getnetworkinfo`
RPCs with a new field that decodes the services bitfield which
indicates what services the peer or local node offer. this is in
addition to a previous field that provided the bitfield itself. E.g.
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhalps s/itself. E.g./itself, e.g./

confusing some users and developers.

- [Bitcoin Core #16714][] updates the GUI first-time wizard with an
option to enable block data pruning. The default depends on the
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps "The default now depends"

option to enable block data pruning. The default depends on the
available disk space, with pruning enabled by default if the user
doesn't have enough space to store the estimated block chain size plus
10 GB
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing period at end of sentence

- [Bitcoin Core #15584][] disables support for the [BIP70][] payment
protocol by default. It can be re-enabled by compiling with the
`--enabled-bip70` configure parameter. See our previous description
in [Newsletter #19][pr14451] for the several reasons why developers
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: remove either "the", or "several"

Copy link
Contributor

@bitschmidty bitschmidty left a comment

tACK 6f20e21

@kanzure
Copy link

@kanzure kanzure commented Sep 16, 2019

By the way, my transcripts at Scaling Bitcoin 2019 are missing the first half of the second day of Scaling because I was ill. If you are evaluating interesting talks to mention based on my transcripts alone, then you will miss a few to consider.

Thanks for the thoughtful summaries as always.

@harding
Copy link
Contributor Author

@harding harding commented Sep 16, 2019

@kanzure I hope you are better now. It's currently my policy to only consider talks if there are transcripts (not necessarily by you), as (1) watching videos to determine whether the topic is interesting takes much longer than skimming transcripts and (2) it is also more work as I either need to take notes during the talk or go back and re-watch particular parts for quotes or data.

It's unfortunately that some might be missing here, but there's lots more missing from all the conferences where they didn't try or weren't able to get you to transcribe. Ultimately, it's my opinion that any good idea for improving Bitcoin technology should be posting to the bitcoin-dev mailing list where I'll see it and so will anyone else who cares.

Thank you for letting me know about the discontinuity and, as always, thank you very much for the transcriptions you did type!

@harding
Copy link
Contributor Author

@harding harding commented Sep 16, 2019

Pushed edits for @jonatack feedback (thanks!).

nodes, such as the LNbig node which has about "20-24% of the liquidity
of the LN in their channels".
of the LN in their channels."
Copy link
Collaborator

@jonatack jonatack Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

@kanzure
Copy link

@kanzure kanzure commented Sep 16, 2019

It's unfortunately that some might be missing here, but there's lots more missing from all the conferences where they didn't try or weren't able to get you to transcribe. Ultimately, it's my opinion that any good idea for improving Bitcoin technology should be posting to the bitcoin-dev mailing list where I'll see it and so will anyone else who cares.

ACK. I am of the same opinion.

be made almost constant with the proposed [erlay][] protocol,
allowing you to have many more peers, which reduces the risk of network
partitioning attacks. Moving on to describing various parts of the
taproot proposal, they show how schnorr signatures make it possible
Copy link
Member

@adamjonas adamjonas Sep 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should Schnorr be capitalized in all instances?

Copy link
Contributor

@bitschmidty bitschmidty Sep 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We decided to keep it lowercase in the style guide

and for only the cost of one. Finally, taproot can commit to a set
of conditions without requiring any party reveal those conditions
unless they need them, which they might not if their protocol allows
them to use schnorr multiparty signatures instead. Each of these
Copy link
Member

@adamjonas adamjonas Sep 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

Core [PR#14626][Bitcoin Core #14626], released as part of 0.18.0,
which made orphan transactions less effective for probing (i.e., it
fixes the problem described in section 4.3 of the TxProbe paper).
[PR#14897][Bitcoin Core #14897], described in [Newsletter
Copy link
Member

@adamjonas adamjonas Sep 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nit: Sergi did talk some about 14897 during the Q&A: https://youtu.be/-gdfxNalDIc?t=13405.

@bitschmidty bitschmidty force-pushed the 2019-09-18-newsletter branch from 841627b to ec19f83 Sep 18, 2019
@bitschmidty
Copy link
Contributor

@bitschmidty bitschmidty commented Sep 18, 2019

pushed a small commit based on @adamjonas feedback. squashed and kept the SPV fraud proofs without a soft fork-removal commit in per @jnewbery 's rec.

@bitschmidty bitschmidty merged commit c32a183 into bitcoinops:master Sep 18, 2019
2 checks passed
@bitschmidty
Copy link
Contributor

@bitschmidty bitschmidty commented Sep 18, 2019

Dev++ and scaling, lots of action
TxProbe mapping using orphan transaction

Amiti talking about tx rebroadcast
BIP70 as a default did not last

Blockchain design patterns to fill your brain
Want to change to signet? Just tweak your -chain

Thanks @harding for getting all the scaling and dev++ content for the newsletter in such a timely manner! Thanks to @adamjonas @jonatack @jnewbery for your 👀

@harding
Copy link
Contributor Author

@harding harding commented Sep 18, 2019

@bitschmidty thanks for the last-minute edit and I love the poem this week!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants