Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Also mmap terms index (.tip) files for hybridfs #43150

Merged
merged 3 commits into from
Jun 12, 2019

Conversation

jimczi
Copy link
Contributor

@jimczi jimczi commented Jun 12, 2019

This change adds the terms index (.tip) to the list of extensions
that are memory-mapped by hybridfs. These files used to be accessed
only once to load the terms index on-heap but since #42838 they can
now be used to read the binary FST directly so it is beneficial to
memory-map them instead of accessing them via NIO.
This commit should fix the slowdowns that appeared in the large terms query (nightly-oss-geonames-add-defaults-large_terms-latency) of the geonames rally track after #42838:
https://elasticsearch-benchmarks.elastic.co/index.html#tracks/geonames/nightly/oss/

I ran the same benchmark locally with and without this patch and it seems that the patch restores the original performance:

|                                                        Metric |        Task |   Baseline |   Contender |     Diff |   Unit |
|--------------------------------------------------------------:|------------:|-----------:|------------:|---------:|-------:|
|                                            Total Young Gen GC |             |      0.385 |        0.33 |   -0.055 |      s |
|                                              Total Old Gen GC |             |          0 |           0 |        0 |      s |
|                                                    Store size |             |    3.17182 |     3.17182 |        0 |     GB |
|                                                 Translog size |             |    2.79254 |     2.79254 |        0 |     GB |
|                                        Heap used for segments |             |    4.62684 |     4.62684 |        0 |     MB |
|                                      Heap used for doc values |             |  0.0322456 |   0.0322456 |        0 |     MB |
|                                           Heap used for terms |             |    3.45925 |     3.45925 |        0 |     MB |
|                                           Heap used for norms |             |  0.0705566 |   0.0705566 |        0 |     MB |
|                                          Heap used for points |             |   0.277195 |    0.277195 |        0 |     MB |
|                                   Heap used for stored fields |             |    0.78759 |     0.78759 |        0 |     MB |
|                                                 Segment count |             |         91 |          91 |        0 |        |
|                                                Min Throughput | large_terms |    1.28375 |     1.50115 |   0.2174 |  ops/s |
|                                             Median Throughput | large_terms |    1.33047 |     1.50278 |  0.17231 |  ops/s |
|                                                Max Throughput | large_terms |     1.3353 |     1.50363 |  0.16833 |  ops/s |
|                                       50th percentile latency | large_terms |    21281.2 |     339.103 | -20942.1 |     ms |
|                                       90th percentile latency | large_terms |    32425.3 |     376.747 | -32048.6 |     ms |
|                                       99th percentile latency | large_terms |    34225.8 |      481.39 | -33744.4 |     ms |
|                                      100th percentile latency | large_terms |    34351.5 |     539.584 | -33811.9 |     ms |
|                                  50th percentile service time | large_terms |    780.101 |     337.226 | -442.875 |     ms |
|                                  90th percentile service time | large_terms |    1030.17 |     375.923 | -654.251 |     ms |
|                                  99th percentile service time | large_terms |    1098.15 |     480.034 | -618.113 |     ms |
|                                 100th percentile service time | large_terms |    1145.94 |     535.561 | -610.381 |     ms |
|                                                    error rate | large_terms |          0 |           0 |        0 |      % |

This change adds the terms index (`.tip`) to the list of extensions
that are memory-mapped by hybridfs. These files used to be accessed
only once to load the terms index on-heap but since elastic#42838 they can
now be used to read the binary FST directly so it is benefical to
memory-map them instead of accessing them via NIO.
@jimczi jimczi added >enhancement :Distributed/Engine Anything around managing Lucene and the Translog in an open shard. v8.0.0 v7.3.0 labels Jun 12, 2019
@jimczi jimczi requested a review from jpountz June 12, 2019 11:53
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@s1monw
Copy link
Contributor

s1monw commented Jun 12, 2019

awesome!

s1monw added a commit to s1monw/elasticsearch that referenced this pull request Jun 12, 2019
…ctionary

Given the significant performance impact that NIOFS has when term dicts are
loaded off-heap this change enforces FstLoadMode#AUTO that loads term dicts
off heap only if the underlying index input indicates a memory map.

Relates to elastic#43150
@jimczi jimczi merged commit ef8f90c into elastic:master Jun 12, 2019
@jimczi jimczi deleted the mmap_terms_index branch June 12, 2019 18:53
jimczi added a commit that referenced this pull request Jun 12, 2019
This change adds the terms index (`.tip`) to the list of extensions
that are memory-mapped by hybridfs. These files used to be accessed
only once to load the terms index on-heap but since #42838 they can
now be used to read the binary FST directly so it is benefical to
memory-map them instead of accessing them via NIO.
s1monw added a commit that referenced this pull request Jun 13, 2019
…ctionary (#43158)

Given the significant performance impact that NIOFS has when term dicts are
loaded off-heap this change enforces FstLoadMode#AUTO that loads term dicts
off heap only if the underlying index input indicates a memory map.

Relates to #43150
s1monw added a commit that referenced this pull request Jun 13, 2019
…ctionary (#43158)

Given the significant performance impact that NIOFS has when term dicts are
loaded off-heap this change enforces FstLoadMode#AUTO that loads term dicts
off heap only if the underlying index input indicates a memory map.

Relates to #43150
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Engine Anything around managing Lucene and the Translog in an open shard. >enhancement v7.3.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants