-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved API for loading Terrier indices into memory #386
Conversation
What's going on here -- can I have a bit of context? |
Woops. The pr omitted my explanations or any meaningful title. See https://gist.github.com/cmacdonald/c12ddacd73ad379b3b0f7a8b7cf1d080 |
""" | ||
|
||
@staticmethod | ||
def _load_into_memory(index, structures=['lexicon', 'direct', 'inverted', 'meta'], load=False): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about document index? It is accessed for getting doc lens, so quite often during ranking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
document lengths are always in memory. the choice of memory or not relates to the other information, specifically the bits used to access the direct index
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No changes, just a note about document index.
(WIP)