Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proposed DHTv2 Changes #473

Merged
merged 61 commits into from
Apr 6, 2020
Merged

Proposed DHTv2 Changes #473

merged 61 commits into from
Apr 6, 2020

Conversation

aschmahmann
Copy link
Contributor

This branch contains the various changes to the DHT that will result in us bumping the DHT protocol version.

The changes include:

@aschmahmann aschmahmann mentioned this pull request Mar 3, 2020
Closed
@Stebalien Stebalien force-pushed the cypress branch 4 times, most recently from 657cbbd to e505723 Compare March 6, 2020 04:02
aarshkshah1992
aarshkshah1992 reviewed Mar 6, 2020
View reviewed changes
routing.go Outdated Show resolved Hide resolved
query.go Outdated Show resolved Hide resolved
aarshkshah1992
aarshkshah1992 reviewed Mar 6, 2020
View reviewed changes
query.go Outdated Show resolved Hide resolved
aarshkshah1992
aarshkshah1992 reviewed Mar 6, 2020
View reviewed changes
query.go Outdated Show resolved Hide resolved
@Stebalien Stebalien added this to the Working Kademlia milestone Mar 10, 2020
@aschmahmann aschmahmann force-pushed the cypress branch 2 times, most recently from 82d847c to 314213d Compare March 26, 2020 23:25
aschmahmann and others added 17 commits April 3, 2020 12:50
@aschmahmann @Stebalien
upgrade deprecated query events
57a12ce
@aschmahmann @Stebalien
first pass at proper kad
6793dc1
@aschmahmann @Stebalien
fix Search/GetValue to be Kad compliant. Default quorum is now 0 whic…
564764f 
…h means do not abort the query early
@aschmahmann @Stebalien
fix refreshing buckets during query. getting topK peers is more effic…
616b0e1 
…ient.
@aschmahmann @Stebalien
Search/GetValues stops if using quorum
4aee8bb
@aschmahmann @Stebalien
error on queries to empty routing table
6dac71c
@aschmahmann @Stebalien
query switches from alpha mode to k mode if no peers closer than one …
85f88e1 
…we have heard about (in a given path) has been found
@aschmahmann @Stebalien
SearchValue will only update k closest peers to have latest record
c7a8d0d
@aschmahmann @Stebalien
added todo comment about FindPeer and peer connectedness
cfb4b40
@aschmahmann @Stebalien
fix: use dht.bucketSize instead of KValue for queries
19e4d2e
@aschmahmann @Stebalien
test: switch tests to use dht.bucketSize instead of KValue for queries
11776e0
@aschmahmann @Stebalien
options: KValue and AlphaValue global variables no longer used intern…
e967aca 
…ally. Concurrency option now sets alpha. DisjointPaths option now sets d. Default number of disjoint paths is now bucketSize/2.
@aschmahmann @Stebalien
try another peer sorting metric. fix bug in when findproviders returns
dd0bbb2
@aschmahmann @Stebalien
check for stopping the query in between dialing and querying
4c2cf46
@aschmahmann @Stebalien
cleanup unused code
41f47f7
@aschmahmann @raulk
@aarshkshah1992 @Stebalien
Utilize identify events to add peers to the routing table (#472)
509c0bc 
* feat: consume identify events to evaluate routing table addition
* fix: routing table no longer gets an update just because new messages have arrived or been sent
* fix: add already connected peers into the routing table before listening to events

Co-authored-by: Raúl Kripalani <raul.kripalani@gmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
@Stebalien
fix: improve context deadline handling
1de9b90 
1. Continue to best-effort provide, but still return an error when we fail to
send provider records to the _best_ peers.
2. Continue returning the best peer's we've found in GetClosestPeers, but also
return an error to indicate that we didn't find the closest ones.

And fix the hang test.
Stebalien and others added 13 commits April 3, 2020 12:50
@Stebalien
chore: run with race detector enabled and disabled
fa52ddd
@aschmahmann @willscott @Stebalien
Configurable Peer Filtering (#471)
0543571 
Allows specifying peer filter functions on query and on adding peers to the routing table. This patch also includes some reasonable default functions for a public-only and private-only DHT.

Co-authored-by: Will Scott <will@cypherpunk.email>
Co-authored-by: Steven Allen <steven@stebalien.com>
@aarshkshah1992 @Stebalien
bump the peerstore version
29bc71a
@Stebalien
feat: reduce request read timeout (#527)
fad6c8a 
This should stop us from waiting on unresponsive peers. This only kicks in when
we've already _sent_ a request so this:

* Shouldn't be blocked on other requests we're planning on making.
* Shouldn't be blocked on dialing.
@petar @aschmahmann @Stebalien
Emit events during async query. (#504)
5a41195 
* Add Kademlia async lookup implementation (replaces prior implementation)
* Add lookup events that allow full lookup reconstruction from logs
* Introduce lookup stopping condition, based on parameter "beta"

Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
@aschmahmann @Stebalien
fix: cleanup context cancellations (#530)
39990c0
@Stebalien
feat: return target peer if we know about them (#511)
d0770d2 
Previously, we'd only return the target peer if we were connected to them.
However, this makes it difficult to impossible to find peers that are mostly
disconnected from the network.

This change also changes `p` to `from` in several places as `p` is _very_
confusing. We should probably switch away from using `p` everywhere.
@aarshkshah1992 @Stebalien
New RT management policy (#520)
8fa0438 
* new RT management policy
@aschmahmann @Stebalien
spawn the correct number of simultaneous queries
d715495
@aarshkshah1992 @Stebalien
Periodic Pinging and active fixLowPeers (#533)
2539114 
* fix Low Peers and Cleanup
@aschmahmann @Stebalien
Record valuable peers will only take into account latency of seed pee…
6a36698 
…rs (#536)

* record valuable peers algorithm only takes into account latency of seed peers
@Stebalien
chore: update deps
485c60a
@aschmahmann @Stebalien
refreshing the routing table after the dht context is cancelled retur…
a4eaa70 
…ns an error instead of hanging
Stebalien and others added 7 commits April 3, 2020 15:19
@Stebalien
chore: fix lints
5aad954 
Except deprecated warn calls
@Stebalien
feat: better logging
445472b 
* Remove deprecated events.
* Fix warning -> warn.
* Use structured logging (some places).
@Stebalien
ci: use IPFS circle configs
dfea791
@Stebalien
fix(test): don't panic on read/write error
d520a1e 
The querying node _is_ allowed to reset and go away.

fixes #431
@Stebalien
chore(ci): remove travis
deee8b7
@aschmahmann @Stebalien
remove deprecated global variables
5e318d3
@aschmahmann @Stebalien
use dht context during refreshes so that they can be logged
6f1df26
@Stebalien Stebalien mentioned this pull request Apr 5, 2020
Closed
Stebalien and others added 4 commits April 5, 2020 19:28
@Stebalien
feat: add a v1 compatibility mode
9bab969 
When this option is passed, the DHT node will listen on and query _only_ using
the old DHT protocol. Importantly, the node won't even pretend to be a new DHT
node because it's routing table includes V1 peers.
@Stebalien
chore: remove unused testProtocols
8af12dd 
We no longer need this now that we explicitly support v1 compatibility mode.
@Stebalien
chore: update kbucket
7035c59
@aschmahmann @Stebalien
test: fix TestFindPeerQuery and TestFindPeerQueryMinimal to not try a…
a7fd35f 
…nd refresh routing tables for orphaned nodes
@Stebalien Stebalien marked this pull request as ready for review April 6, 2020 04:27
@aschmahmann aschmahmann merged commit 1ba0310 into master Apr 6, 2020
@Stebalien
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aarshkshah1992 aarshkshah1992 aarshkshah1992 left review comments

@Stebalien Stebalien Stebalien approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Working Kademlia
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@aschmahmann @Stebalien @aarshkshah1992 @petar