/bitcoin.org

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Dev Docs: Document Blocks-First IBD & Orphan Blocks #685

Merged
merged 2 commits into from Feb 1, 2015

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
Dev Docs
Projects
None yet
Milestone
No milestone
2 participants
@harding @saivann
@harding
Contributor

harding commented Dec 18, 2014

Dev Docs: Document Blocks-First IBD & Orphan Blocks

Preview IBD section: http://dg0.dtrt.org/en/developer-guide#initial-block-download
Preview Orphan Blocks section: http://dg0.dtrt.org/en/developer-guide#orphan-blocks

This pull adds a new section about the Initial Block Download (IBD) method used in Bitcoin Core 0.9, which I've retroactively named blocks-first. It also adds a short section explaining the difference between orphan blocks and stale blocks. See the commit message for minor details.

This is a partial resubmission of closed pull #660 focusing only on the current IBD method. This keeps the text shorter and, hopefully, easier to review. I also think this version is much, much better---it includes nearly 100% rewritten text plus five new illustrations that didn't appear in the original pull. After we get this merged, I'll rewrite and resubmit the headers-first part of pull #660.

@harding harding added the Dev Docs label Dec 18, 2014

@harding
Contributor

harding commented Jan 14, 2015

Rebased to fix merge conflicts.

@saivann saivann commented on an outdated diff Jan 31, 2015

View full outdated diff
_includes/guide_p2p_network.md
+instance of the object. For blocks, the unique identifier is a hash of
+the block's header.
+
+The block inventories appear in the `inv` message in the same order they
+appear in the block chain, so this first `inv` message contains
+inventories for blocks 1 through 501. (For example, the hash of block 1
+is 4860...0000 as seen in the illustration above.)
+
+The IBD node uses the received inventories to request 128 blocks from
+the sync node in the `getdata` message illustrated below.
+
+![First GetData Message Sent During IBD](/img/dev/en-ibd-getdata.svg)
+
+It's important to headers-first nodes that the blocks be requested and
+sent in order because each block header references the header hash of
+the preceeding block. That means the IBD node can't fully validate a
@saivann

saivann Jan 31, 2015

Contributor

@harding s/preceeding/preceding/

@saivann saivann commented on an outdated diff Jan 31, 2015

View full outdated diff
_includes/guide_p2p_network.md
+![Second GetBlocks Message Sent During IBD](/img/dev/en-ibd-getblocks2.svg)
+
+Upon receipt of the second `getblocks` message, the sync node takes the
+first listed header hash and searches its local best block chain for a
+block with that header hash. If it finds a block with that hash, it
+replies with 500 block inventories starting with the following block.
+But if it doesn't find a block with that hash, it takes the next header
+hash from the `getblocks` message and searches its block chain for that
+hash. If that hash matches, it will reply with 500 block inventories
+starting with the following hash from that point. But, again, if it
+doesn't find that hash, it will proceed to check the next hash in the
+message (and so on until it runs out of hashes in the message). If the
+last hash in the message (besides the stopping hash) doesn't match, it
+assumes the only block the two nodes have in common is block 0 and so it
+sends an `inv` starting with block 1 (the same `inv` message seen
+several illustrations above).
@saivann

saivann Jan 31, 2015

Contributor

@harding FWIW, I was temporarily confused here, probably due to the repeating and conditional sentences, and felt this could perhaps be explained in a more concise way. What would you think of something similar to the following?

Upon receipt of the second getblocks message, the sync node searches its local best block chain for a block that matches one of the header hashes in the message, trying each hash in the order they were received. As soon as it finds a matching hash, it will reply with 500 block inventories starting with the following hash from that point. However if there is no matching hash (besides the stopping hash), it assumes the only block the two nodes have in common is block 0 and so it sends an inv starting with block 1 (the same inv message seen several illustrations above).
@saivann
Contributor

saivann commented Jan 31, 2015

@harding Thanks!! LGTM

harding added some commits Nov 28, 2014

@harding
Dev Docs: Document Blocks-First IBD & Orphan Blocks 
This commit provides a detailed overview of the current block download
method, which I've retroactively named blocks-first for parallelism with
headers-first.

New And Significantly Revised:

* New Initial Block Download (IBD) section (h3) with Blocks-First
  subsection (h4)

* New Orphan Block subsection (under Blocks Broadcasting) describing
  orphan blocks and how they're handled under blocks-first. Also
  includes a simple illustration of the difference between orphan blocks
  and stale blocks. Thanks to luke-jr for his s/orphan block/stale
  block/ commit a couple months ago---that made this commit much easier.

Edits:

* Cleaned up a couple cases missed by previous s/orphan/stale/ commit
  because they used past tense (orphaned).

* In P2P reference section, mentioned that a `block` message can be sent
  unsolicited by miners.

* Mention that `getheaders` and `headers` were added in protocol
  version 31800.

* Moved a few internal links around and added a few new internal links.

Administrivia:

* Started adding "TODOv0.10" in HTML comments to places that need to be
  updated when 0.10 is released so that I can easily git grep for that
  tag later.
This commit was signed with a verified signature.
@harding harding David A. Harding
GPG key ID: 4B29C30FF29EC4B7 Learn about signing commits
ccda4ca
@harding
Dev Docs: Corrections to IBD Suggested By Saivann (Thanks!)
This commit was signed with a verified signature.
@harding harding David A. Harding
GPG key ID: 4B29C30FF29EC4B7 Learn about signing commits
3109903
@harding
Contributor

harding commented Jan 31, 2015

Rebased to fix merge conflicts and added commit 3109903 making the changes @saivann suggested. (Thanks!) Preview updated.

In the absence of critical feedback, this will be merged around 22:00 UTC Sunday.

@harding harding merged commit 3109903 into bitcoin-dot-org:master Feb 1, 2015

harding added a commit that referenced this pull request Feb 1, 2015

@harding
Dev Docs: Merge Pull #685 and Pull #721 
* 685: Dev Docs: Document Blocks-First IBD & Orphan Blocks
* 721: Dev Docs: Mention Upcoming v3 Blocks/BIP66
This commit was signed with a verified signature.
@harding harding David A. Harding
GPG key ID: 4B29C30FF29EC4B7 Learn about signing commits
e0f1a36

@harding harding deleted the harding:ibd2 branch Feb 25, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment