Full and archive nodes #293

grantnoble · 2020-04-01T10:34:29Z

This combines the changes made in PR #283

priv_getCode API method (#145)

Include a section (in Pruning.md) about the difference between the two nodes and how to run them. Misc changes to the various pruning options and sync mode options. Signed-off-by: grantnoble <grant.noble@consensys.net>

Signed-off-by: grantnoble <grant.noble@consensys.net>

Included a "Fast Synchronization" section in Pruning.md Signed-off-by: grantnoble <grant.noble@consensys.net>

Specifically: - Archive node is full sync and not pruned - Full node is fast sync and pruned - Pruning is default on for fast sync and off for full sync. Can turn pruning on/off explicitly if they so choose. - Full nodes have the current state, so can’t serve the network with any data request (i.e. balance of address X at block Y, where Y is old). You can guarantee the latest state for everything (and some older states, but not all). Signed-off-by: grantnoble <grant.noble@consensys.net>

Signed-off-by: grantnoble <grant.noble@consensys.net>

NicolasMassart

Looks good to me.

NicolasMassart · 2020-04-02T14:33:30Z

docs/HowTo/Send-Transactions/Revert-Reason.md

@@ -49,7 +49,7 @@ the revert reason as an ABI-encoded string.
    The revert reason is not included in the transactions receipts root hash. Not including the
    revert reason in the transactions receipts root hash means the revert reason is only available
    to nodes that execute the transaction when importing the block. That is, the revert reason is
-    not available if using fast sync.
+    not available if using fast synchronization (`--sync-mode=FAST`).


maybe add a link to ../../Reference/CLI/CLI-Syntax.md#sync-mode ?

timbeiko · 2020-04-02T18:18:13Z

docs/Concepts/Pruning.md

+
+Besu supports two node types, commonly referred to as _full nodes_ and _archive nodes_.
+
+Full nodes have just the current state of the blockchain so cannot serve the network with all data


I would remove the just to reflect that they can have more than the current state.

timbeiko · 2020-04-02T18:26:40Z

docs/Concepts/Pruning.md

+
+Archive nodes require significantly more disk space (approximately 3TB) than full nodes
+(approximately 750GB).
+


Maybe one thing we should add is that it is possible to re-generate all the data for an archive node using a full node (it's long, inconvenient, etc., but the data is not lost).

What is the process for this? If we say you can do it, we should document how.

It's not trivial, not something that most people will want to do (easier to just sync an archive node), but it just speaks to the security properties of a full node. Thinking about it again, I'm not sure how relevant it is here 😅

OK. We'll leave it as is. Thanks!

timbeiko · 2020-04-02T18:27:12Z

docs/Concepts/Pruning.md

+contract for every block since the genesis block. With an archive node you can do everything you
+do with a full node, as well as access historical state data.
+
+Archive nodes require significantly more disk space (approximately 3TB) than full nodes


Maybe worth adding these are Besu numbers, as they differ from other client implementations, especially for full nodes. We should also precise this is for mainnet.

timbeiko · 2020-04-02T18:29:28Z

docs/Concepts/Pruning.md

+
+Instead of a full synchronization ([`--sync-mode=FULL`](../Reference/CLI/CLI-Syntax.md#sync-mode)),
+which starts from the genesis block and reprocesses all transactions, fast synchronization
+([`--sync-mode=FAST`](../Reference/CLI/CLI-Syntax.md#sync-mode)) downloads the transaction receipts


Doesn't it also verify some blocks? I thought it did verify 1/n blocks, see: ethereum/go-ethereum#1889. I'm not sure if our implementation is the same, though. cc: @RatanRSur

Also, it's worth highlighting that past the pivot block, fast sync switches to full sync.

It verifies the chain of headers from the genesis until a block when it switches to processing blocks in their entirety (full sync as you mentioned).

I would mention that it downloads the headers but I wouldn't mention that it switches to full sync at because it's an implementation detail and isn't really something they can rely on. The guarantee we give them is that they'll have the full current state but we don't want them assuming they'll have full state from any previous block because the pivot block switches many times while downloading the world state.

timbeiko · 2020-04-02T18:30:35Z

docs/Concepts/Pruning.md


-Use pruning to reduce storage required for the world state. Pruning removes state trie nodes that
-are not required.
+Pruning reduces the storage required for the world state, removing state trie nodes that are not


Doesn't it reduce the storage required by removing state trie nodes from previous states? cc: @RatanRSur

Pruning reduces the storage required for by Besu by removing state trie nodes that are unreachable from recent blocks

You can then link to the option --pruning-blocks-retained after the recent blocks statement to make it explicit how "recent" is defined.

Signed-off-by: grantnoble <grant.noble@consensys.net>

…thub.com/grantnoble/besu-docs into grantnoble-grantnoble-full-archive-nodes

Signed-off-by: grantnoble <grant.noble@consensys.net>

Grant Noble added 5 commits April 1, 2020 16:47

Merge pull request #1 from hyperledger/master

57fa3cd

priv_getCode API method (#145)

Difference between full node and archive node

6a47c1c

Include a section (in Pruning.md) about the difference between the two nodes and how to run them. Misc changes to the various pruning options and sync mode options. Signed-off-by: grantnoble <grant.noble@consensys.net>

Fix two self-induced markdown lint errors.

8e263f8

Signed-off-by: grantnoble <grant.noble@consensys.net>

Fix incorrect link to CLI-Syntax.md

92ff8e5

Signed-off-by: grantnoble <grant.noble@consensys.net>

Updates based on @MadelineMurray feedback

5286a4c

Included a "Fast Synchronization" section in Pruning.md Signed-off-by: grantnoble <grant.noble@consensys.net>

grantnoble requested review from NicolasMassart, MadelineMurray and bgravenorst April 1, 2020 10:34

grantnoble self-assigned this Apr 1, 2020

grantnoble mentioned this pull request Apr 1, 2020

Difference between full node and archive node #283

Closed

grantnoble added 2 commits April 2, 2020 11:18

Fixed broken links

d46c869

Signed-off-by: grantnoble <grant.noble@consensys.net>

grantnoble requested a review from timbeiko April 2, 2020 01:25

NicolasMassart approved these changes Apr 2, 2020

View reviewed changes

timbeiko reviewed Apr 2, 2020

View reviewed changes

Grant Noble added 4 commits April 3, 2020 10:59

Merge branch 'master' into grantnoble-grantnoble-full-archive-nodes

b7a89a0

Updates based on @NicolasMassart, @timbeiko, and @RatanRSur feedback

c818342

Signed-off-by: grantnoble <grant.noble@consensys.net>

Merge branch 'grantnoble-grantnoble-full-archive-nodes' of https://gi…

71bf88f

…thub.com/grantnoble/besu-docs into grantnoble-grantnoble-full-archive-nodes

Some updated inadvertently missed in the last update.

262c408

Signed-off-by: grantnoble <grant.noble@consensys.net>

timbeiko approved these changes Apr 3, 2020

View reviewed changes

Merge branch 'master' into grantnoble-grantnoble-full-archive-nodes

d5ce7f7

grantnoble merged commit c67cdae into hyperledger:master Apr 4, 2020

grantnoble deleted the grantnoble-grantnoble-full-archive-nodes branch April 4, 2020 00:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Full and archive nodes #293

Full and archive nodes #293

grantnoble commented Apr 1, 2020

NicolasMassart left a comment

NicolasMassart Apr 2, 2020

grantnoble Apr 3, 2020

timbeiko Apr 2, 2020

grantnoble Apr 3, 2020

timbeiko Apr 2, 2020

grantnoble Apr 3, 2020

timbeiko Apr 3, 2020

grantnoble Apr 4, 2020

timbeiko Apr 2, 2020

grantnoble Apr 3, 2020

timbeiko Apr 2, 2020

RatanRSur Apr 2, 2020

grantnoble Apr 3, 2020

timbeiko Apr 2, 2020

RatanRSur Apr 2, 2020

grantnoble Apr 3, 2020


		Besu supports two node types, commonly referred to as _full nodes_ and _archive nodes_.

		Full nodes have just the current state of the blockchain so cannot serve the network with all data


		Archive nodes require significantly more disk space (approximately 3TB) than full nodes
		(approximately 750GB).

Full and archive nodes #293

Full and archive nodes #293

Conversation

grantnoble commented Apr 1, 2020

NicolasMassart left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment