New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch indexer for osu_scores*_high #1

Closed
wants to merge 29 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@notbakaneko

notbakaneko commented May 31, 2018

Indexes the contents of the osu_scores*_high tables into elasticsearch.
It turns out it's significantly faster and less resource intensive to have a separate program in C# perform the index population instead of running it though osu-web.

The main use for this, initially, is for osu-web to be able to run aggregation queries for the user's high score data though elasticsearch. Eventually, we should probably look at moving all the index population over?

There's still a bunch of stuff to do like

  • Watching for new data.
  • Handle elasticsearch response errors (throttle and retry if node is busy).
  • Option to ignore existing index and create a new one.

notbakaneko and others added some commits May 31, 2018

@notbakaneko

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment