Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix: Network Explorer: Add freegeoip API key and split out tasks for country distributions #806

Merged
merged 13 commits into from
Oct 19, 2021

Conversation

mmsinclair
Copy link
Contributor

This PR fixes up a bug in the country distribution tasks and hopefully improves the code structure by splitting out the tasks more.

  • freegeoip requires an API key for 15,000 free requests per hour (and the API hostname has changed)
  • country distribution is in a separate task that runs every 15 mins
  • mix node cache refresh occurs every 10 minutes
  • geolocation runs as a permanent task that picks off a mix node that hasn't been located (or where the location cache time has expired - 1 day) and is rate limited to run every 50ms (a variation on the suggestion by @futurechimp )
  • error reporting has been improved by adding more Error types to the location Result to log when the explorer-api is being rate limited by freegoip and also when a mix node's IP address cannot be located (there are a few that can't be found)

Things that could be improved / ideas / concerns:

  • the task timers (e.g. mix nodes refreshed every 10 mins, distribution every 15 mins, location cache TTL of 1 day) could be tweaked / fiddled with because I just picked some numbers out of the air
  • the code structure can be improved, apologies in advance for my bad rusting
  • the geolocation job could create a queue of the mix nodes to locate and pop them off, instead of the for (...) { ... return; } pattern (a throw back to my C++ days)

I am much happier with the location task picking things off every 50ms because it works better with the naive way we spawn tokio tasks. The service will actually stop now when a halt signal is sent.

@mmsinclair mmsinclair force-pushed the bugfix/network-explorer-api-geoip branch from e9d5cf7 to ceeaa46 Compare October 11, 2021 11:33
@mmsinclair mmsinclair merged commit 062a5ed into develop Oct 19, 2021
@mmsinclair mmsinclair deleted the bugfix/network-explorer-api-geoip branch October 19, 2021 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working network-explorer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants