Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up phrase suggestion scoring #5396

Closed
wants to merge 2 commits into from

Commits on Mar 13, 2014

  1. Make indexRandom handle many documents better

    * Index one at a time only rarely if doing more then 300.
    * When launching async actions, take some care to make sure you don't already
    have more then 150 other async actions in flight.
    * When indexing in bulk split into chunks of 1000 documents.
    nik9000 committed Mar 13, 2014
    Copy the full SHA
    f08a645 View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2014

  1. Speed up phrase suggestion scoring

    Two changes:
    1.  In the StupidBackoffScorer only look for the trigram if there is a bigram.
    2.  Cache the frequencies in WordScorer so we don't look them up again and
    again and again.  This is implemented by wrapping the TermsEnum in a special
    purpose wrapper that really only works in context of the WordScorer.
    
    This provides a pretty substantial speedup when there are many candidates.
    
    Closes elastic#5395
    nik9000 committed Mar 17, 2014
    Copy the full SHA
    0c92055 View commit details
    Browse the repository at this point in the history