Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

authorithy-discovery: Make changing of peer-id while active a bit more robust #3786

Open
wants to merge 57 commits into
base: master
Choose a base branch
from
Open

authorithy-discovery: Make changing of peer-id while active a bit more robust #3786

wants to merge 57 commits into from

Commits on Mar 21, 2024

  1. authorithy-discovery: Make changing of peer-id while active a bit mor… 

    …e robust

In the case when nodes don't persist their node-key or they want to
generate a new one while being in the active set, things go wrong
because both the old addresses and the new ones will still be present in
DHT, so because of the distributed nature of the DHT both will survive
in the network untill the old ones expires which is 36 hours.
Nodes in the network will randomly resolve the authorithy-id to the old
address or the new one.

More details in: #3673

This PR proposes we mitigate this problem, by:

1. Let the query for a DHT key retrieve all the results, that is usually
   bounded by the replication factor which is 20, currently we interrupt
   the querry on the first result.
2. Modify the authority-discovery service to keep all the discovered
   addresses around for 24h since they last seen an address.
3. Plumb through other subsystems where the assumption was that an
   authorithy-id will resolve only to one PeerId. Currently, the
   authorithy-discovery keeps just the last record it received from DHT
   and queries the DHT every 10 minutes. But they could always receive
   only the old address, only the new address or a flip-flop between
   them depending on what node wins the race to provide the record
4. Update gossip-support to try resolve authorities more often than
   every session.

This would gives us a lot more chances for the nodes in the networks to
also discover not only the old address of the node but also the new one
and should improve the time it takes for a node to be properly connected
in the network. The behaviour won't be deterministic because there is no
guarantee the all nodes will see the new record at least once, since
they could query only nodes that have the old one.

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    0103c7f View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. Make clippy happy 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    4172677 View commit details
    Browse the repository at this point in the history

  2. Fix warnings 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    64f38d2 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. Refactor gossip support 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    ce87688 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into fix_change_node_id_… 

    …at_restart
    @alexggh
    alexggh committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    bd69a55 View commit details
    Browse the repository at this point in the history

  3. Make clippy happy 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    a69ba99 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart
    @alexggh
    alexggh committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    311aade View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. Some other hacks 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    1305763 View commit details
    Browse the repository at this point in the history

  2. Add more changes 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    7e16e58 View commit details
    Browse the repository at this point in the history

  3. More refactoring 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    a82ebed View commit details
    Browse the repository at this point in the history

  4. A bit more refactoring 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    d569ab3 View commit details
    Browse the repository at this point in the history

  5. A few more improvements 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    1c9a40d View commit details
    Browse the repository at this point in the history

  6. Fixup even more 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    91e647c View commit details
    Browse the repository at this point in the history

  7. Another something 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    0a53ec2 View commit details
    Browse the repository at this point in the history

  8. Fixup everything 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    ef6ddb6 View commit details
    Browse the repository at this point in the history

  9. Post refactoring 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    b3eb615 View commit details
    Browse the repository at this point in the history

  10. Fixup Cargo's 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    f6d4b29 View commit details
    Browse the repository at this point in the history

  11. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    b41cdf7 View commit details
    Browse the repository at this point in the history

  12. Make clippy happy 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    1ae1759 View commit details
    Browse the repository at this point in the history

  13. Some minor tweaks 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    7b56bc8 View commit details
    Browse the repository at this point in the history

  14. Minor updates 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    d610df6 View commit details
    Browse the repository at this point in the history

  15. Minor review feedback 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    4263dc4 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Add authorithy-discovery-unittests 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    4d7b164 View commit details
    Browse the repository at this point in the history

  2. Add unittest for gossip-support changes 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    7cc3333 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    fabdb4b View commit details
    Browse the repository at this point in the history

  2. Simplify indentation levels 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    42afffc View commit details
    Browse the repository at this point in the history

  3. Minor cleanups 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    28ac0a2 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart
    @alexggh
    alexggh committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    95950a7 View commit details
    Browse the repository at this point in the history

  2. Update litep2p 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    bd21fda View commit details
    Browse the repository at this point in the history

  3. Use put_valut_to from litep2p 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    57f1b38 View commit details
    Browse the repository at this point in the history

  4. Minor feedback 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    9f56b02 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Minor changes 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    6d196c4 View commit details
    Browse the repository at this point in the history

  2. Minor updates 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    9979f8f View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Integrate with paritytech/litep2p#96 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    fd5dc46 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. Fix warning on quorum failed 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed May 1, 2024
    Configuration menu
    Copy the full SHA
    20e351e View commit details
    Browse the repository at this point in the history

  2. Reconnect only if new peer ids pop-up 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed May 1, 2024
    Configuration menu
    Copy the full SHA
    98398e3 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. Revert kademlia removal 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed May 2, 2024
    Configuration menu
    Copy the full SHA
    8066044 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart
    @alexggh
    alexggh committed May 2, 2024
    Configuration menu
    Copy the full SHA
    8eef5c3 View commit details
    Browse the repository at this point in the history

  3. Update cargo.lock 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed May 2, 2024
    Configuration menu
    Copy the full SHA
    b4fe357 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into alexaggh/fix_change_node_id_at_restart

    @alexggh
    alexggh committed May 2, 2024
    Configuration menu
    Copy the full SHA
    97f09a3 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart
    @alexggh
    alexggh committed May 9, 2024
    Configuration menu
    Copy the full SHA
    586a0f1 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2024

  1. Update substrate/client/network/src/litep2p/mod.rs 

    Co-authored-by: Dmitry Markin <dmitry@markin.tech>
    @alexggh @dmitry-markin
    alexggh and dmitry-markin committed May 12, 2024
    Configuration menu
    Copy the full SHA
    09b6306 View commit details
    Browse the repository at this point in the history

  2. Update polkadot/node/network/gossip-support/src/tests.rs 

    Co-authored-by: Dmitry Markin <dmitry@markin.tech>
    @alexggh @dmitry-markin
    alexggh and dmitry-markin committed May 12, 2024
    Configuration menu
    Copy the full SHA
    fb534b5 View commit details
    Browse the repository at this point in the history

  3. Update substrate/client/authority-discovery/src/worker/tests.rs 

    Co-authored-by: Dmitry Markin <dmitry@markin.tech>
    @alexggh @dmitry-markin
    alexggh and dmitry-markin committed May 12, 2024
    Configuration menu
    Copy the full SHA
    7399dc5 View commit details
    Browse the repository at this point in the history

  4. Update assert messages 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed May 12, 2024
    Configuration menu
    Copy the full SHA
    17df838 View commit details
    Browse the repository at this point in the history

  5. Address review feedback 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed May 12, 2024
    Configuration menu
    Copy the full SHA
    3313fd7 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Use a single signature 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed May 30, 2024
    Configuration menu
    Copy the full SHA
    999a710 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart_v2
    @alexggh
    alexggh committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    3b4650a View commit details
    Browse the repository at this point in the history

  2. Remove unused 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    7e81ca2 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. Update substrate/client/network/src/litep2p/mod.rs 

    Co-authored-by: Dmitry Markin <dmitry@markin.tech>
    @alexggh @dmitry-markin
    alexggh and dmitry-markin committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    f083318 View commit details
    Browse the repository at this point in the history

  2. Update substrate/client/authority-discovery/src/worker.rs 

    Co-authored-by: Dmitry Markin <dmitry@markin.tech>
    @alexggh @dmitry-markin
    alexggh and dmitry-markin committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    a5fcc6b View commit details
    Browse the repository at this point in the history

  3. Update substrate/client/authority-discovery/src/worker.rs 

    Co-authored-by: Dmitry Markin <dmitry@markin.tech>
    @alexggh @dmitry-markin
    alexggh and dmitry-markin committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    e54c96d View commit details
    Browse the repository at this point in the history

  4. Update documentation 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    66959dd View commit details
    Browse the repository at this point in the history

  5. Update substrate/client/authority-discovery/src/worker.rs 

    Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
    @alexggh @lexnv
    alexggh and lexnv committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    06285ef View commit details
    Browse the repository at this point in the history

  6. Address review feedback 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    3a23fbd View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Merge remote-tracking branch 'origin/master' into alexaggh/fix_change… 

    …_node_id_at_restart_v3
    @alexggh
    alexggh committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    77e4061 View commit details
    Browse the repository at this point in the history

  2. Add PrDoc 

    Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
    @alexggh
    alexggh committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    0256b3c View commit details
    Browse the repository at this point in the history