Add Developer Guide To Bitcoin.org

[harding]

This pull request proposes to add to Bitcoin.org a Developer Guide providing the equivalent of 70 printed pages of introductions to key Bitcoin concepts.

A preview is available at: http://bitcoindev.us.to/en/developer-documentation

The Guide is not complete---there is much more we plan to write over the coming months---but the version in this pull request is internally consistent and reasonably comprehensive. In addition to the Guide, this pull request also includes a Developer Reference with 90 more pages of detailed technical material, such as RPC descriptions, and a portal page linking to the various sections of both the Guide and the Reference.

To keep comments on this pull request focused, we'd appreciate it if suggestions about possible future content or better ways to organize this information were made on our Content Suggestions Wiki Page, restricting comments on this pull request to only things which should be fixed or changed before merging.

If you want to help review but don't have time to read over 150 pages of text, we recommend that you review just one of the sections listed below, preferably one that hasn't been well-reviewed yet. We'll add your GitHub @ name to the Being Reviewed By column for any section where you leave a line comment, and add your name to the ACK column if you ACK any section.

Section Preview (Source Files)	Being Reviewed By	ACKs From
Block Chain (BC)
Transactions (T)	@TierNolan
Contracts (C)	@mikehearn
Wallets (W)
Payment Processing (PP)
Operating Modes (OM)
P2P Network (P2P)
Mining (M)	@luke-jr
Block Chain & Transaction Reference (BC & T)
Bitcoin Core RPCs (A-G, H-N, O-T, U-Z)

If you leave a line comment or general comment for a particular section, we'd appreciate it if you ACK the section once it meets your expectations. We hope this will allow us to build consensus to merge without everyone having to read the entire text.

Many thanks are owed for the two months of work which went into this pull request. An incomplete, alphabetical list is: @cbeams, @harding, @instagibbs, @mikehearn, @saivann, and @tgeller.

TODO:

• Replace P2PH by P2PKH. Fixed in #400 (merged)
• Replace DOS by DoS. Fixed in #394 (merged)
• GBT can use a persistent TCP socket, without a persistent session. Fixed in #394 (merged)
• More emphasis on key reuse security risks? Fixed in #395 (merged)
• Contracts section improvements suggested by @mikehearn. Fixed in #394 (merged)
• Remove assertion that pay-to-pubkey (unhashed) is used in all coinbase txes and is more convenient. Fixed in #394 (merged)

[luke-jr]

This seems to confuse addresses with scriptPubKeys, as well as some fungibility concerns... It also uses the new abbreviation P2PH, where "p2pkh" has been more frequently used. The section on key reuse fails to mention known security risks, only mentioning security in a hypothetical context. DoS is improperly all uppercased. Also, contrary to the section on Stratum, GBT can and often does use a persistent TCP socket - although without a persistent session, which is perhaps what was intended.

[saivann]

@luke-jr: Thanks for your feedback!

scriptPubKeys / fungibility: If you can provide more details, or better, a pull req or suggested improvements, this would be appreciated.

Key reuse & security risks: @mikehearn suggested we don't mention this risk since ECDSA public keys are secure (today). But I knew you and other developers would want it, so we opted for a non-alarmist and short version that wouldn't suggest ECDSA is insecure. But if there is any specific example you'd like mentioned, feel free to mention them here.

P2PKH / DoS: That's fine with me, I can update all occurences.

Stratum / GBT: I'll let others comment about this.

[harding]

@luke-jr thank you! Two notes in addition to @saivann's response:

DOS is all uppercased to be consistent with the style used throughout the document which does not lowercase interior prepositions in captialized phrases. For example, see all the headings with prepositions. We can special case DoS or abbreviations in general, or (more radically) change the entire text and all the images. However, how do you feel about keeping the current consistent usage?

Stratum/GBT: I'll make that update.

[luke-jr]

What repository do I clone and send a merge request to?

ECDSA public keys are not necessarily secure today, given multiple signatures. Just because we've worked around every possible exploit situation doesn't mean we should pass it off as if it has no known risks... Ignoring the risks exposes developers to them, so they need to be explicitly mentioned so developers know to avoid them.

"DOS" is "Disk Operating System"

[saivann]

@luke-jr You can clone and send pull requests to the devel-docs branch on this repository (bitcoin/bitcoin.org). Edit: preferably not too much unrelated change per pull request would work better.

[mikehearn]

I'm all for mentioning the risks of poorly implementing ECDSA! But I think a blanket statement like "address reuse is bad for security" is very strong given the state of play, if I wasn't intimately familiar with the details and read this I'd be struggling to figure out how this is a problem, given the prevalence of multiple signatures and published public keys. The wider security community doesn't make any such security recommendations so it'd be weird if the Bitcoin world was not in sync with that.

But I don't feel very strongly about this issue. If someone wants to write up an explanation of what can go wrong with ECDSA and note that avoiding pubkey reuse can sometimes mitigate those failures, I won't object.

[mikehearn]

Better phrasing:

The complexity of transactions has so far worked against you by making the system harder to understand. But the benefits become clear once we start using contracts.

[mikehearn]

It's not very important, but this example is very USA centric. Cheques are basically dead inside Europe (and elsewhere too?): the only people I know who still use them are my grandparents. I wonder if we can find a more internationalised example.

[instagibbs]

"Alice and Bob might only know each other casually over the Internet; they would never open a checking account together---one of them could
take the money without their partner's permission."

[harding]

Thanks, @instagibbs! This has actually already been fixed in #394 (see the todo list in the main pull request description for an updated list of problems and who's assigned to fixing them.) Thanks again!

[mikehearn]

Technically no placeholder is necessary and you can just send around raw signatures, if all sides know what the tx is meant to look like. But it's probably unimportant to mention that.

[wbnns]

@harding Hey! Just wanted to reach out and say BIG THANKS to you for working on this! :) You are the man! Thank you and @saivann both for putting this initiative together. This is going to be a big deal! Huge thanks also to @mikehearn and @luke-jr for helping polish this up! :)

[harding]

@gwb3 Thanks! All of us who worked on the docs are pleased with the result. There's still lots more work to be done to make this guide reasonably complete, so expect many more pull requests. :-)

[mikehearn]

How about we get this merged? It can always be improved live.

[harding]

@mikehearn I think @saivann was planning on merging it on May 24th after asking for a final round of reviews. As the most recent bit of incorrect information in the guide was only discovered a few days ago (see #408), I think waiting another 5 days is reasonable. (Although I admit to being quite eager to see this on Bitcoin.org myself.)

[saivann]

@mikehearn @harding Agreed, thanks!

In the absence of critical feedback, this pull request will be merged on May 24th (Edit: 15:00:00 +0000). Additional feedback and reviews are welcome and appreciated.

[harding]

In anticipation of the merge tomorrow at 15:00 UTC, I made a local test merge and jekyll compile, and then ran some tests. Except for one tiny mistake (mine) fixed now in 32c8d59, everything looks good to me.

[saivann]

Merging this!

Huge thanks to @cbeams, @harding, @instagibbs, @mikehearn, and @tgeller for your work on making this possible!

Additional issues and improvements can be reported as issues or pull request on GitHub. More general discussions can take place on the mailing list:
https://groups.google.com/forum/#!forum/bitcoin-documentation

[mikehearn]

Amazing work guys. Really, really impressed by what you have accomplished.