Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DHT: improve iterative find #3562

Merged
merged 6 commits into from Feb 16, 2022
Merged

Conversation

shyba
Copy link
Member

@shyba shyba commented Feb 8, 2022

  • uses a dict instead
  • bottoming out on peer search gives a warning and returns empty (returning earlier could give bad results for announcements, so imo it is preferable to fail entirely)
  • probes a lot less (stops probing if it goes outside of the top max_result closest nodes) = less packets, same precision, more concurrency expected

depends: #3559

@coveralls
Copy link

coveralls commented Feb 8, 2022

Coverage Status

Coverage increased (+0.02%) to 69.702% when pulling 4c7e345 on dht_iterative_find_stop_condition into 87ec7ac on dht_bug.

@shyba shyba added area: DHT type: improvement Existing (or partially existing) functionality needs to be changed labels Feb 9, 2022
@shyba shyba marked this pull request as ready for review February 14, 2022 14:56
@lbry-bot lbry-bot assigned jackrobison and unassigned shyba Feb 14, 2022
@shyba shyba merged commit 097f59d into dht_bug Feb 16, 2022
@shyba shyba deleted the dht_iterative_find_stop_condition branch February 16, 2022 16:01
self.exclude = exclude or []

self.active: typing.Set['KademliaPeer'] = set()
self.active: typing.Dict['KademliaPeer', int] = {} # peer: distance, sorted
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be an ordered dict to be explicit

@lbry-bot lbry-bot assigned shyba and unassigned jackrobison Feb 16, 2022
self.delayed_call.cancel() # ensure anything scheduled gets cancelled
self._search_round()
#if self.running:
# self.delayed_call = self.loop.call_later(delay, self._search)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can delayed_call be deleted?

self.tasks.append(self.loop.create_task(self._search_task(0.0)))
for peer in [peer for peer, task in self.running_probes.items() if task.done() or task == t]:
del self.running_probes[peer]
self._search_task(0.0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the delay isn't needed as an arg anymore


def _search(self):
self.tasks.append(self.loop.create_task(self._search_task()))
self._search_task()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete this method?

@@ -4885,20 +4885,16 @@ async def jsonrpc_blob_delete(self, blob_hash):
"""

@requires(DHT_COMPONENT)
async def jsonrpc_peer_list(self, blob_hash, search_bottom_out_limit=None, page=None, page_size=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be documented as a breaking change

@jackrobison jackrobison removed type: improvement Existing (or partially existing) functionality needs to be changed area: DHT labels Mar 4, 2022
@shyba shyba added type: improvement Existing (or partially existing) functionality needs to be changed area: DHT and removed type: improvement Existing (or partially existing) functionality needs to be changed area: DHT labels Mar 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants