Skip to content
Permalink
Browse files

Dynamize trusted validator list and quorum (RIPD-1220):

Instead of specifying a static list of trusted validators in the config
or validators file, the configuration can now include trusted validator
list publisher keys and URIs where validator lists are served.
Publisher public keys should use the account public Base58 encoding to
distinguish them from node public keys. The sites are regularly queried
for the latest recommended list of validators from the trusted
publishers.

The trusted validator list and quorum are now reset each consensus
round using the fetched validator lists and the list of recent
validations seen. The minimum validation quorum is now only
configurable via the command line.

Validator lists are expected to be served as JSON and include the
following fields:

* "manifest": Base64-encoded serialization of a manifest containing the
  validator publisher's master and signing public keys. This has the
  same format as the [validation_manifest] config section.

* "blob": Base64-encoded JSON string containing a "sequence" and
  "validators" field. "validators" contains an array of objects with
  "validation_public_key" and "manifest" fields.
  "validation_public_key" must be an Ed25519 master public key.

* "signature": Hex-encoded signature of the blob using the publisher's
  signing key.

* "version": 1

* "refreshInterval" (optional)
  • Loading branch information...
wilsonianb
wilsonianb committed Aug 30, 2016
1 parent ffbd447 commit 08906f96c910ef3c76f0b0f5622a4521475edf35
Showing with 3,573 additions and 1,221 deletions.
  1. +18 −10 Builds/VisualStudio2015/RippleD.vcxproj
  2. +24 −9 Builds/VisualStudio2015/RippleD.vcxproj.filters
  3. +0 −3 bin/python/ripple/util/ValidatorManifestTest.py
  4. +0 −5 bin/python/ripple/util/test_ConfigFile.py
  5. +17 −16 doc/rippled-example.cfg
  6. +23 −17 doc/validators-example.txt
  7. +0 −6 src/ripple/app/ledger/LedgerMaster.h
  8. +2 −1 src/ripple/app/ledger/impl/LedgerConsensusImp.cpp
  9. +6 −55 src/ripple/app/ledger/impl/LedgerMaster.cpp
  10. +30 −13 src/ripple/app/main/Application.cpp
  11. +2 −0 src/ripple/app/main/Application.h
  12. +2 −9 src/ripple/app/main/Main.cpp
  13. +156 −79 src/ripple/{overlay/impl → app/misc}/Manifest.h
  14. +8 −4 src/ripple/app/misc/NetworkOPs.cpp
  15. +80 −18 src/ripple/app/misc/Validations.cpp
  16. +3 −1 src/ripple/app/misc/Validations.h
  17. +407 −55 src/ripple/app/misc/ValidatorList.h
  18. +47 −0 src/ripple/app/misc/detail/Work.h
  19. +222 −0 src/ripple/app/misc/detail/WorkBase.h
  20. +76 −0 src/ripple/app/misc/detail/WorkPlain.h
  21. +137 −0 src/ripple/app/misc/detail/WorkSSL.h
  22. +87 −197 src/ripple/{overlay → app/misc}/impl/Manifest.cpp
  23. +609 −96 src/ripple/app/misc/impl/ValidatorList.cpp
  24. +1 −2 src/ripple/core/Config.h
  25. +3 −2 src/ripple/core/ConfigSections.h
  26. +0 −1 src/ripple/core/Job.h
  27. +0 −1 src/ripple/core/JobTypes.h
  28. +99 −83 src/ripple/core/impl/Config.cpp
  29. +1 −1 src/ripple/net/InfoSub.h
  30. +0 −12 src/ripple/overlay/Overlay.h
  31. +7 −57 src/ripple/overlay/impl/OverlayImpl.cpp
  32. +0 −17 src/ripple/overlay/impl/OverlayImpl.h
  33. +1 −1 src/ripple/overlay/impl/PeerImp.cpp
  34. +2 −3 src/ripple/protocol/JsonFields.h
  35. +3 −6 src/ripple/rpc/handlers/UnlList.cpp
  36. +1 −0 src/ripple/unity/app_misc.cpp
  37. +0 −1 src/ripple/unity/overlay.cpp
  38. +1 −1 src/test/app/AmendmentTable_test.cpp
  39. +185 −165 src/test/{overlay/manifest_test.cpp → app/Manifest_test.cpp}
  40. +1,182 −173 src/test/app/ValidatorList_test.cpp
  41. +126 −99 src/test/core/Config_test.cpp
  42. +4 −1 src/test/rpc/ServerInfo_test.cpp
  43. +1 −0 src/unity/app_test_unity.cpp
  44. +0 −1 src/unity/overlay_test_unity.cpp
@@ -997,6 +997,14 @@
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\misc\CanonicalTXSet.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\Work.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\WorkBase.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\WorkPlain.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\WorkSSL.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\FeeVote.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\misc\FeeVoteImpl.cpp">
@@ -1023,6 +1031,10 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\misc\impl\Manifest.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\misc\impl\Transaction.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -1037,6 +1049,8 @@
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\misc\LoadFeeTrack.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\Manifest.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\misc\NetworkOPs.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -2242,12 +2256,6 @@
</ClCompile>
<ClInclude Include="..\..\src\ripple\overlay\impl\ConnectAttempt.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\overlay\impl\Manifest.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\overlay\impl\Manifest.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\overlay\impl\Message.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -4340,6 +4348,10 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\app\Manifest_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\app\MultiSign_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -4612,10 +4624,6 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\overlay\manifest_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\overlay\short_read_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -109,6 +109,9 @@
<Filter Include="ripple\app\misc">
<UniqueIdentifier>{5A1509B2-871B-A7AC-1E60-544D3F398741}</UniqueIdentifier>
</Filter>
<Filter Include="ripple\app\misc\detail">
<UniqueIdentifier>{2919FCCC-A707-22B8-FFB4-89494A8AC070}</UniqueIdentifier>
</Filter>
<Filter Include="ripple\app\misc\impl">
<UniqueIdentifier>{C4BDB9F8-7DB7-E304-D286-098085D5D16E}</UniqueIdentifier>
</Filter>
@@ -1524,6 +1527,18 @@
<ClInclude Include="..\..\src\ripple\app\misc\CanonicalTXSet.h">
<Filter>ripple\app\misc</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\Work.h">
<Filter>ripple\app\misc\detail</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\WorkBase.h">
<Filter>ripple\app\misc\detail</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\WorkPlain.h">
<Filter>ripple\app\misc\detail</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\detail\WorkSSL.h">
<Filter>ripple\app\misc\detail</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\FeeVote.h">
<Filter>ripple\app\misc</Filter>
</ClInclude>
@@ -1548,6 +1563,9 @@
<ClCompile Include="..\..\src\ripple\app\misc\impl\LoadFeeTrack.cpp">
<Filter>ripple\app\misc\impl</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\misc\impl\Manifest.cpp">
<Filter>ripple\app\misc\impl</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\misc\impl\Transaction.cpp">
<Filter>ripple\app\misc\impl</Filter>
</ClCompile>
@@ -1560,6 +1578,9 @@
<ClInclude Include="..\..\src\ripple\app\misc\LoadFeeTrack.h">
<Filter>ripple\app\misc</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\misc\Manifest.h">
<Filter>ripple\app\misc</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\misc\NetworkOPs.cpp">
<Filter>ripple\app\misc</Filter>
</ClCompile>
@@ -2841,12 +2862,6 @@
<ClInclude Include="..\..\src\ripple\overlay\impl\ConnectAttempt.h">
<Filter>ripple\overlay\impl</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\overlay\impl\Manifest.cpp">
<Filter>ripple\overlay\impl</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\overlay\impl\Manifest.h">
<Filter>ripple\overlay\impl</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\overlay\impl\Message.cpp">
<Filter>ripple\overlay\impl</Filter>
</ClCompile>
@@ -5157,6 +5172,9 @@
<ClCompile Include="..\..\src\test\app\LoadFeeTrack_test.cpp">
<Filter>test\app</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\app\Manifest_test.cpp">
<Filter>test\app</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\app\MultiSign_test.cpp">
<Filter>test\app</Filter>
</ClCompile>
@@ -5352,9 +5370,6 @@
<ClCompile Include="..\..\src\test\overlay\cluster_test.cpp">
<Filter>test\overlay</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\overlay\manifest_test.cpp">
<Filter>test\overlay</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\overlay\short_read_test.cpp">
<Filter>test\overlay</Filter>
</ClCompile>
@@ -104,9 +104,6 @@
n9KiYM9CgngLvtRCQHZwgC2gjpdaZcCcbt3VboxiNFcKuwFVujzS RL4
n9LdgEtkmGB9E2h3K4Vp7iGUaKuq23Zr32ehxiU8FWY7xoxbWTSA RL5
[validation_quorum]
3
[validation_seed]
{validation_seed}
#vaidation_public_key: {validation_public_key}
@@ -45,7 +45,6 @@ def test_full(self):
'websocket_public_port': '5206',
'peer_ip': '0.0.0.0',
'rpc_port': '5205',
'validation_quorum': '3',
'websocket_ip': '127.0.0.1'}

FULL = """
@@ -131,10 +130,6 @@ def test_full(self):
n9KiYM9CgngLvtRCQHZwgC2gjpdaZcCcbt3VboxiNFcKuwFVujzS RL4
n9LdgEtkmGB9E2h3K4Vp7iGUaKuq23Zr32ehxiU8FWY7xoxbWTSA RL5
# Ditto.
[validation_quorum]
3
[validation_seed]
sh1T8T9yGuV7Jb6DPhqSzdU2s5LcV
@@ -499,8 +499,7 @@
#
# These settings affect the behavior of the server instance with respect
# to Ripple payment protocol level activities such as validating and
# closing ledgers, establishing a quorum, or adjusting fees in response
# to server overloads.
# closing ledgers or adjusting fees in response to server overloads.
#
#
#
@@ -560,11 +559,16 @@
# to always accept as validators as well as the minimum number of validators
# needed to accept consensus.
#
# The contents of the file should include a [validators] and a
# [validation_quorum] entry. [validators] should be followed by
# a list of validation public keys of nodes, one per line, optionally
# followed by a comment separated by whitespace.
# [validation_quorum] should be followed by a number.
# The contents of the file should include a [validators] and/or
# [validator_list_sites] and [validator_list_keys] entries.
# [validators] should be followed by a list of validation public keys of
# nodes, one per line.
# [validator_list_sites] should be followed by a list of URIs each serving a
# list of recommended validators.
# [validator_list_keys] should be followed by a list of keys belonging to
# trusted validator list publishers. Validator lists fetched from configured
# sites will only be considered if the list is accompanied by a valid
# signature from a trusted publisher key.
#
# Specify the file by its name or path.
# Unless an absolute path is specified, it will be considered relative to
@@ -576,14 +580,11 @@
#
# Example content:
# [validators]
# n949f75evCHwgyP4fPVgaHqNHxUVN15PsJEZ3B3HnXPcPjcZAoy7 RL1
# n9MD5h24qrQqiyBC8aeqqCWvpiBiYQ3jxSr91uiDvmrkyHRdYLUj RL2
# n9L81uNCaPgtUJfaHh89gmdvXKAmSt5Gdsw2g1iPWaPkAHW5Nm4C RL3
# n9KiYM9CgngLvtRCQHZwgC2gjpdaZcCcbt3VboxiNFcKuwFVujzS RL4
# n9LdgEtkmGB9E2h3K4Vp7iGUaKuq23Zr32ehxiU8FWY7xoxbWTSA RL5
#
# [validation_quorum]
# 3
# n949f75evCHwgyP4fPVgaHqNHxUVN15PsJEZ3B3HnXPcPjcZAoy7
# n9MD5h24qrQqiyBC8aeqqCWvpiBiYQ3jxSr91uiDvmrkyHRdYLUj
# n9L81uNCaPgtUJfaHh89gmdvXKAmSt5Gdsw2g1iPWaPkAHW5Nm4C
# n9KiYM9CgngLvtRCQHZwgC2gjpdaZcCcbt3VboxiNFcKuwFVujzS
# n9LdgEtkmGB9E2h3K4Vp7iGUaKuq23Zr32ehxiU8FWY7xoxbWTSA
#
#
# [path_search]
@@ -974,7 +975,7 @@ pool.ntp.org
[ips]
r.ripple.com 51235

# File containing validation quorum and trusted validator keys.
# File containing trusted validator keys or validator list publishers.
# Unless an absolute path is specified, it will be considered relative to the
# folder in which the rippled.cfg file is located.
[validators_file]
@@ -13,8 +13,6 @@
# [validators]
#
# List of the validation public keys of nodes to always accept as validators.
# A comment may, optionally, be associated with each entry, separated by
# whitespace from the validation public key.
#
# The latest list of recommended validators can be obtained from
# https://ripple.com/ripple.txt
@@ -23,26 +21,34 @@
#
# Examples:
# n9KorY8QtTdRx7TVDpwnG9NvyxsDwHUKUEeDLY3AkiGncVaSXZi5
# n9MqiExBcoG19UXwoLjBJnhsxEhAZMuWwJDRdkyDz1EkEkwzQTNt John Doe
# n9MqiExBcoG19UXwoLjBJnhsxEhAZMuWwJDRdkyDz1EkEkwzQTNt
#
# [validator_list_sites]
#
# List of URIs serving lists of recommended validators.
#
# [validation_quorum]
# Examples:
# https://ripple.com/validators
# http://127.0.0.1:8000
#
# [validator_list_keys]
#
# Sets the minimum number of trusted validations a ledger must have before
# the server considers it fully validated. Note that if you are validating,
# your validation counts.
# List of keys belonging to trusted validator list publishers.
# Validator lists fetched from configured sites will only be considered
# if the list is accompanied by a valid signature from a trusted
# publisher key.
# Validator list keys use the account public key encoding.
#
# Examples:
# aBRPVJYX5kBbxtBZY8jnJcwg9FNdtzkZ29votFZRfNbDKULKQEeB
# aBR4qTgmetCjWCftWKpHzX5Dnw5oERnv6SHuzqU67TaECX9zrJCP
# aBR7wQ6VRpd2jgXvMMxUgWHMjBis8yPje9KedYRJ9E7ht3kWQTtJ
#

# Public keys of the validators that this rippled instance trusts.
[validators]
n949f75evCHwgyP4fPVgaHqNHxUVN15PsJEZ3B3HnXPcPjcZAoy7 RL1
n9MD5h24qrQqiyBC8aeqqCWvpiBiYQ3jxSr91uiDvmrkyHRdYLUj RL2
n9L81uNCaPgtUJfaHh89gmdvXKAmSt5Gdsw2g1iPWaPkAHW5Nm4C RL3
n9KiYM9CgngLvtRCQHZwgC2gjpdaZcCcbt3VboxiNFcKuwFVujzS RL4
n9LdgEtkmGB9E2h3K4Vp7iGUaKuq23Zr32ehxiU8FWY7xoxbWTSA RL5

# The number of validators rippled needs to accept a consensus.
# Don't change this unless you know what you're doing.
[validation_quorum]
3
n949f75evCHwgyP4fPVgaHqNHxUVN15PsJEZ3B3HnXPcPjcZAoy7
n9MD5h24qrQqiyBC8aeqqCWvpiBiYQ3jxSr91uiDvmrkyHRdYLUj
n9L81uNCaPgtUJfaHh89gmdvXKAmSt5Gdsw2g1iPWaPkAHW5Nm4C
n9KiYM9CgngLvtRCQHZwgC2gjpdaZcCcbt3VboxiNFcKuwFVujzS
n9LdgEtkmGB9E2h3K4Vp7iGUaKuq23Zr32ehxiU8FWY7xoxbWTSA
@@ -111,10 +111,6 @@ class LedgerMaster
std::chrono::seconds getValidatedLedgerAge ();
bool isCaughtUp(std::string& reason);

int getMinValidations ();

void setMinValidations (int v, bool strict);

std::uint32_t getEarliestFetch ();

bool storeLedger (std::shared_ptr<Ledger const> ledger);
@@ -312,8 +308,6 @@ class LedgerMaster

std::unique_ptr <detail::LedgerCleaner> mLedgerCleaner;

int mMinValidations; // The minimum validations to publish a ledger.
bool mStrictValCount; // Don't raise the minimum
uint256 mLastValidateHash;
std::uint32_t mLastValidateSeq;

@@ -35,6 +35,7 @@
#include <ripple/app/misc/NetworkOPs.h>
#include <ripple/app/misc/TxQ.h>
#include <ripple/app/misc/Validations.h>
#include <ripple/app/misc/ValidatorList.h>
#include <ripple/app/tx/apply.h>
#include <ripple/basics/contract.h>
#include <ripple/basics/CountedObject.h>
@@ -1275,7 +1276,7 @@ LedgerConsensusImp<Traits>::makeInitialPosition () ->
return v.second->isTrusted();
});

if (count >= ledgerMaster_.getMinValidations())
if (count >= app_.validators ().quorum ())
{
feeVote_.doVoting (
previousLedger_,

0 comments on commit 08906f9

Please sign in to comment.
You can’t perform that action at this time.