Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch indexer for osu_scores*_high #1

Closed
wants to merge 29 commits into from

Conversation

@notbakaneko
Copy link

@notbakaneko notbakaneko commented May 31, 2018

Indexes the contents of the osu_scores*_high tables into elasticsearch.
It turns out it's significantly faster and less resource intensive to have a separate program in C# perform the index population instead of running it though osu-web.

The main use for this, initially, is for osu-web to be able to run aggregation queries for the user's high score data though elasticsearch. Eventually, we should probably look at moving all the index population over?

There's still a bunch of stuff to do like

  • Watching for new data.
  • Handle elasticsearch response errors (throttle and retry if node is busy).
  • Option to ignore existing index and create a new one.
@notbakaneko notbakaneko force-pushed the notbakaneko:feature/elastic-index branch from 9df5b28 to e829e5a Jun 1, 2018
@notbakaneko notbakaneko force-pushed the notbakaneko:feature/elastic-index branch from 0436509 to 312c4d9 Jun 1, 2018
@notbakaneko notbakaneko force-pushed the notbakaneko:feature/elastic-index branch from 186dc17 to e508cb3 Jun 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants