Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable Solr caches #758

Closed
WadeBarnes opened this issue Oct 6, 2023 · 3 comments
Closed

Disable Solr caches #758

WadeBarnes opened this issue Oct 6, 2023 · 3 comments
Assignees

Comments

@WadeBarnes
Copy link
Member

As discussed some of the articles listed here caching is only useful when there are repeated queries for the same information, otherwise they can be detrimental to memory use. Queries in OrgBook tend to be random (Humans) or sequential (Spiders and Bots), so there are few, if any, cache hits. This could be, at least part of, the cause for the high memory usage we're seeing in prod.

Try disabling the caches as recommended to see the impact on memory use.

This issue is related to the investigation happening here; bcgov/orgbook-configurations#137

@WadeBarnes WadeBarnes self-assigned this Oct 6, 2023
WadeBarnes added a commit to WadeBarnes/aries-vcr that referenced this issue Oct 6, 2023
- Addresses bcgov#758

Signed-off-by: Wade Barnes <wade@neoterictech.ca>
@WadeBarnes
Copy link
Member Author

WadeBarnes commented Oct 6, 2023

I created a search-engine image off #759 and smoke tested in dev and test before deploying to prod. Performing some load testing before leaving it over the weekend to collect some metrics.

@WadeBarnes
Copy link
Member Author

Disabling the Solr caches had no effect. If anything the OOM restart issues became worse following the change, though this is just speculation and the increased number of issue could be attributed to query patterns and volume. Some metrics collected here; bcgov/orgbook-configurations#137 (comment).

@WadeBarnes
Copy link
Member Author

This actually helped reduce resource usage. It was actually unbound queries (#762) that were the main issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant