Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newsletters: add 64 (2019-09-18) #226

Merged
merged 2 commits into from Sep 18, 2019

Conversation

@harding
Copy link
Contributor

harding commented Sep 14, 2019

Happy issue 2^6!

@jnewbery

This comment has been minimized.

Copy link
Contributor

jnewbery commented Sep 14, 2019

Happy issue 2^6!

Yay! A round number!

Copy link
Contributor

jnewbery left a comment

ACK 65cd259

My only comment is that I think we should drop the SPV fraud proofs without a soft fork piece. Everything else is 💯


## News

- **SPV fraud proofs without a soft fork:** Ruben Somsen started a

This comment has been minimized.

Copy link
@jnewbery

jnewbery Sep 15, 2019

Contributor

My opinion is that we shouldn't cover this: the proposal is too immature, there are too many unanswered questions, and I doubt it will get implemented. I've only skimmed the proposal after reading your summary, but I had the following questions:

  • how does the algorithm verify coinbase maturity of spends? Does the utreexo TXO model include coinbaseness and height? (I had a look at the utreexo paper and implementation and couldn't find an answer, although I guess it must do, otherwise utreexo wouldn't work
  • how does the fork-finding algorithm work? You write "In the worst case, this increases the cost of the fraud proof by a few megabytes per lying peer.", but it seems to me that a malicious adversary could make an SPV download lots of utreexo commitments to find the fork point, or stall the node.
  • does this require all honest nodes to store utreexo commitments for all blocks in order to serve them to SPVs in the case of a fork?
  • this requires SPV clients to be able to validate all the consensus rules for blocks and transactions in the case of a fork. Can we expect clients to do that?

This comment has been minimized.

Copy link
@harding

harding Sep 15, 2019

Author Contributor
  1. I think you answered this yourself.

  2. Worst case would be the SPV client would need to download one commitment for every block in the block chain, e.g. 32 MB for 32-byte commitments on a chain with a million blocks. Practically, I'd expect SPV clients to ship with checkpoints (many do already) that wouldn't believe a node that claimed a fork further back than a few months, or about 1 MB of commitments. (Even more practically, though perhaps slightly risky, an SPV client may refuse to believe in forks over a week long, or about 32 kB of commitments.) Clients can detect stalling and disconnect the offending node; Bitcoin users should be used to waiting an hour or more for ~6 confirmations, so this process doesn't need to be be super fast.

  3. It's worse than storing commitments (those are just a small hash), I think they need to store (1) the UTXOs being spent in a block (i.e. undo data), (2) the utreexo partial merkle branches connecting the UTXOs to the utreexo merkle root, (3) the block itself. My very rough guess is that this would require about double the disk space of a current archival full node. That said, I don't see any complexity to storing and distributing this information (like BIP158 filters, it should be compute-once and store-on-disk) and if there's an expectation that SPV clients won't believe in forks more than n blocks old, then a node serving these fraud proofs only needs to store n blocks worth of this data. All that said, this was my main concern---that too few full node operators will care to generate and serve this data, making it easy to sybil SPV clients needing that data and so making the "one honest peer" assumption even easier to violate than it is now.

  4. This is explicitly mentioned in the newsletter description. But, yeah, I don't expect SPV clients to actually do that (though some developers have been claiming that they'll do stuff like that to make SPV secure in the future).

In general, I was pretty dubious about this proposal myself and my initial response was to draft an email for the mailing list describing its flaws. However, each flaw I thought I found was one I also found a reasonable solution for, even if I think some of them are unlikely to be implemented. So I wrote this description because SPV fraud proofs under reasonable assumptions without a soft fork seems cool. However, if after reading this too-long comment reply, you still think the proposal is too immature, I'm happy to drop it. This week's newsletter is long enough already.

This comment has been minimized.

Copy link
@harding

harding Sep 16, 2019

Author Contributor

Note: I just replied to Somsen on list with several of these points (plus others) so that he's aware of the possible criticism.

at the time the block was created. The client can then perform the
other consensus checks on block validity which don't depend on state
(e.g., all the fields in its block header are correct, none of its
outputs spends more than 21 million BTC, etc...). With all that

This comment has been minimized.

Copy link
@jnewbery

jnewbery Sep 15, 2019

Contributor

add "all scripts and signatures are valid" as an example of a non-contextual check.

This comment has been minimized.

Copy link
@harding

harding Sep 15, 2019

Author Contributor

That requires UTXO entries.

transition from a state they both agreed upon to a state they find
disagreeable and can perform the same procedure. In the worst case,
this increases the cost of the fraud proof by a few megabytes per
lying peer.

This comment has been minimized.

Copy link
@jnewbery

jnewbery Sep 15, 2019

Contributor

"this increases the cost of the fraud proof by a few megabytes per lying peer." How did you get to this figure? I don't see it in the mailing list post.

This comment has been minimized.

Copy link
@harding

harding Sep 15, 2019

Author Contributor

The mailing list post has estimates for the cost of verifying the state transition from the pre-fork block to one of the fork blocks:

1. block N+1 from the most PoW chain (~1-2MB)
2. the utreexo hash commitment inside of block N (e.g. a merkle path
to the coinbase)
3. the utreexo merkle proofs which prove that all inputs of N+1 are
part of the UTXO set (~1MB)

This is the same procedure except that it's performed on an earlier block, therefore I expect it to have the same costs.

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

"this increases"... perhaps insert a word/phrase describing what causes the increase and how?

@jnewbery

This comment has been minimized.

Copy link
Contributor

jnewbery commented Sep 16, 2019

I've added a commit that removes the SPV fraud proofs section. We shouldn't squash that, so that we have it in the git history in case we want to use it in a future issue.

Copy link

jonatack left a comment

What a newsletter! Feel free to ignore any of the comments below.

fork. Then the client requests each of its peers commit to the UTXO
state from the block before the fork via a utreexo merkle root.
After that commitment, the client downloads the fork block, copies
of the UTXOs it spends, and the partial merkle branches connecting

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

"copies" in line 37 could be a either a verb or a noun, which caused this dense reader to initially think that "of" should be removed. Perhaps s/block, copies of/block,/

at the time the block was created. The client can then perform the
other consensus checks on block validity which don't depend on state
(e.g., all the fields in its block header are correct, none of its
outputs spends more than 21 million BTC, etc...). With all that

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

Perhaps s/With all that/With that/

In case any peers disagree about the value of the utreexo state at
the pre-fork block, the SPV client can work with them to find a
transition from a state they both agreed upon to a state they find
disagreeable and can perform the same procedure. In the worst case,

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

"a transition from a state they both agreed upon to a state they find disagreeable"

does this mean the point at which the disagreement arose?

transition from a state they both agreed upon to a state they find
disagreeable and can perform the same procedure. In the worst case,
this increases the cost of the fraud proof by a few megabytes per
lying peer.

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

"this increases"... perhaps insert a word/phrase describing what causes the increase and how?

## Bitcoin Edge Dev++

The Scaling Bitcoin conference was preceded by the community-organized
Bitcoin Edge Dev++ training sessions. Recording of the sessions are

This comment has been minimized.

Copy link
@jonatack

- [Bitcoin Core #16787][] extends the `getpeerinfo` and `getnetworkinfo`
RPCs with a new field that decodes the services bitfield which
indicates what services the peer or local node offer. this is in

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

s/offer. this/offers. This/

- [Bitcoin Core #16787][] extends the `getpeerinfo` and `getnetworkinfo`
RPCs with a new field that decodes the services bitfield which
indicates what services the peer or local node offer. this is in
addition to a previous field that provided the bitfield itself. E.g.

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

perhalps s/itself. E.g./itself, e.g./

confusing some users and developers.

- [Bitcoin Core #16714][] updates the GUI first-time wizard with an
option to enable block data pruning. The default depends on the

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

perhaps "The default now depends"

option to enable block data pruning. The default depends on the
available disk space, with pruning enabled by default if the user
doesn't have enough space to store the estimated block chain size plus
10 GB

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

missing period at end of sentence

- [Bitcoin Core #15584][] disables support for the [BIP70][] payment
protocol by default. It can be re-enabled by compiling with the
`--enabled-bip70` configure parameter. See our previous description
in [Newsletter #19][pr14451] for the several reasons why developers

This comment has been minimized.

Copy link
@jonatack

jonatack Sep 16, 2019

suggestion: remove either "the", or "several"

Copy link
Contributor

bitschmidty left a comment

tACK 6f20e21

@kanzure

This comment has been minimized.

Copy link

kanzure commented Sep 16, 2019

By the way, my transcripts at Scaling Bitcoin 2019 are missing the first half of the second day of Scaling because I was ill. If you are evaluating interesting talks to mention based on my transcripts alone, then you will miss a few to consider.

Thanks for the thoughtful summaries as always.

@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Sep 16, 2019

@kanzure I hope you are better now. It's currently my policy to only consider talks if there are transcripts (not necessarily by you), as (1) watching videos to determine whether the topic is interesting takes much longer than skimming transcripts and (2) it is also more work as I either need to take notes during the talk or go back and re-watch particular parts for quotes or data.

It's unfortunately that some might be missing here, but there's lots more missing from all the conferences where they didn't try or weren't able to get you to transcribe. Ultimately, it's my opinion that any good idea for improving Bitcoin technology should be posting to the bitcoin-dev mailing list where I'll see it and so will anyone else who cares.

Thank you for letting me know about the discontinuity and, as always, thank you very much for the transcriptions you did type!

@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Sep 16, 2019

Pushed edits for @jonatack feedback (thanks!).

nodes, such as the LNbig node which has about "20-24% of the liquidity
of the LN in their channels".
of the LN in their channels."

This comment has been minimized.

Copy link
@jonatack
@kanzure

This comment has been minimized.

Copy link

kanzure commented Sep 16, 2019

It's unfortunately that some might be missing here, but there's lots more missing from all the conferences where they didn't try or weren't able to get you to transcribe. Ultimately, it's my opinion that any good idea for improving Bitcoin technology should be posting to the bitcoin-dev mailing list where I'll see it and so will anyone else who cares.

ACK. I am of the same opinion.

be made almost constant with the proposed [erlay][] protocol,
allowing you to have many more peers, which reduces the risk of network
partitioning attacks. Moving on to describing various parts of the
taproot proposal, they show how schnorr signatures make it possible

This comment has been minimized.

Copy link
@adamjonas

adamjonas Sep 18, 2019

Contributor

Should Schnorr be capitalized in all instances?

This comment has been minimized.

Copy link
@bitschmidty

bitschmidty Sep 18, 2019

Contributor

We decided to keep it lowercase in the style guide

and for only the cost of one. Finally, taproot can commit to a set
of conditions without requiring any party reveal those conditions
unless they need them, which they might not if their protocol allows
them to use schnorr multiparty signatures instead. Each of these

This comment has been minimized.

Copy link
@adamjonas

adamjonas Sep 18, 2019

Contributor

Same as above.

Core [PR#14626][Bitcoin Core #14626], released as part of 0.18.0,
which made orphan transactions less effective for probing (i.e., it
fixes the problem described in section 4.3 of the TxProbe paper).
[PR#14897][Bitcoin Core #14897], described in [Newsletter

This comment has been minimized.

Copy link
@adamjonas

adamjonas Sep 18, 2019

Contributor

Super nit: Sergi did talk some about 14897 during the Q&A: https://youtu.be/-gdfxNalDIc?t=13405.

@bitschmidty bitschmidty force-pushed the harding:2019-09-18-newsletter branch from 841627b to ec19f83 Sep 18, 2019
@bitschmidty

This comment has been minimized.

Copy link
Contributor

bitschmidty commented Sep 18, 2019

pushed a small commit based on @adamjonas feedback. squashed and kept the SPV fraud proofs without a soft fork-removal commit in per @jnewbery 's rec.

@bitschmidty bitschmidty merged commit c32a183 into bitcoinops:master Sep 18, 2019
2 checks passed
2 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
netlify/bitcoinops/deploy-preview Deploy preview ready!
Details
@bitschmidty

This comment has been minimized.

Copy link
Contributor

bitschmidty commented Sep 18, 2019

Dev++ and scaling, lots of action
TxProbe mapping using orphan transaction

Amiti talking about tx rebroadcast
BIP70 as a default did not last

Blockchain design patterns to fill your brain
Want to change to signet? Just tweak your -chain

Thanks @harding for getting all the scaling and dev++ content for the newsletter in such a timely manner! Thanks to @adamjonas @jonatack @jnewbery for your 👀

@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Sep 18, 2019

@bitschmidty thanks for the last-minute edit and I love the poem this week!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.