Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

increase min memory to 15MB for indexing #2176

Merged
merged 1 commit into from
Sep 13, 2023
Merged

increase min memory to 15MB for indexing #2176

merged 1 commit into from
Sep 13, 2023

Conversation

PSeitz
Copy link
Contributor

@PSeitz PSeitz commented Sep 10, 2023

With tantivy 0.20 the minimum memory consumption per SegmentWriter increased to
12MB. 7MB are for the different fast field collectors types (they could be
lazily created). Increase the minimum memory budget from 3MB to 15MB to avoid single doc segments.

Change memory variable naming from arena to budget.

closes #2156

With tantivy 0.20 the minimum memory consumption per SegmentWriter increased to
12MB. 7MB are for the different fast field collectors types (they could be
lazily created). Increase the minimum memory from 3MB to 15MB.

Change memory variable naming from arena to budget.

closes #2156
@codecov-commenter
Copy link

Codecov Report

Patch coverage: 95.38% and project coverage change: -0.03% ⚠️

Comparison is base (1932513) 94.42% compared to head (9b5e9b1) 94.39%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2176      +/-   ##
==========================================
- Coverage   94.42%   94.39%   -0.03%     
==========================================
  Files         322      322              
  Lines       63104    63122      +18     
==========================================
  Hits        59583    59583              
- Misses       3521     3539      +18     
Files Changed Coverage Δ
src/functional_test.rs 0.00% <0.00%> (ø)
src/query/boolean_query/boolean_query.rs 95.72% <ø> (ø)
src/query/disjunction_max_query.rs 82.14% <ø> (ø)
src/indexer/index_writer.rs 96.84% <94.73%> (+0.02%) ⬆️
src/collector/count_collector.rs 100.00% <100.00%> (ø)
src/collector/facet_collector.rs 98.90% <100.00%> (ø)
src/collector/histogram_collector.rs 98.53% <100.00%> (ø)
src/collector/mod.rs 54.63% <100.00%> (ø)
src/collector/multi_collector.rs 99.41% <100.00%> (ø)
src/core/index.rs 87.43% <100.00%> (ø)
... and 7 more

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@PSeitz PSeitz merged commit 2d73903 into main Sep 13, 2023
5 checks passed
@PSeitz PSeitz deleted the check_budget branch September 13, 2023 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Slow indexing
3 participants