Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Also mmap terms index (`.tip`) files for hybridfs #43150

Merged
merged 3 commits into from Jun 12, 2019

Conversation

@jimczi
Copy link
Member

commented Jun 12, 2019

This change adds the terms index (.tip) to the list of extensions
that are memory-mapped by hybridfs. These files used to be accessed
only once to load the terms index on-heap but since #42838 they can
now be used to read the binary FST directly so it is beneficial to
memory-map them instead of accessing them via NIO.
This commit should fix the slowdowns that appeared in the large terms query (nightly-oss-geonames-add-defaults-large_terms-latency) of the geonames rally track after #42838:
https://elasticsearch-benchmarks.elastic.co/index.html#tracks/geonames/nightly/oss/

I ran the same benchmark locally with and without this patch and it seems that the patch restores the original performance:

|                                                        Metric |        Task |   Baseline |   Contender |     Diff |   Unit |
|--------------------------------------------------------------:|------------:|-----------:|------------:|---------:|-------:|
|                                            Total Young Gen GC |             |      0.385 |        0.33 |   -0.055 |      s |
|                                              Total Old Gen GC |             |          0 |           0 |        0 |      s |
|                                                    Store size |             |    3.17182 |     3.17182 |        0 |     GB |
|                                                 Translog size |             |    2.79254 |     2.79254 |        0 |     GB |
|                                        Heap used for segments |             |    4.62684 |     4.62684 |        0 |     MB |
|                                      Heap used for doc values |             |  0.0322456 |   0.0322456 |        0 |     MB |
|                                           Heap used for terms |             |    3.45925 |     3.45925 |        0 |     MB |
|                                           Heap used for norms |             |  0.0705566 |   0.0705566 |        0 |     MB |
|                                          Heap used for points |             |   0.277195 |    0.277195 |        0 |     MB |
|                                   Heap used for stored fields |             |    0.78759 |     0.78759 |        0 |     MB |
|                                                 Segment count |             |         91 |          91 |        0 |        |
|                                                Min Throughput | large_terms |    1.28375 |     1.50115 |   0.2174 |  ops/s |
|                                             Median Throughput | large_terms |    1.33047 |     1.50278 |  0.17231 |  ops/s |
|                                                Max Throughput | large_terms |     1.3353 |     1.50363 |  0.16833 |  ops/s |
|                                       50th percentile latency | large_terms |    21281.2 |     339.103 | -20942.1 |     ms |
|                                       90th percentile latency | large_terms |    32425.3 |     376.747 | -32048.6 |     ms |
|                                       99th percentile latency | large_terms |    34225.8 |      481.39 | -33744.4 |     ms |
|                                      100th percentile latency | large_terms |    34351.5 |     539.584 | -33811.9 |     ms |
|                                  50th percentile service time | large_terms |    780.101 |     337.226 | -442.875 |     ms |
|                                  90th percentile service time | large_terms |    1030.17 |     375.923 | -654.251 |     ms |
|                                  99th percentile service time | large_terms |    1098.15 |     480.034 | -618.113 |     ms |
|                                 100th percentile service time | large_terms |    1145.94 |     535.561 | -610.381 |     ms |
|                                                    error rate | large_terms |          0 |           0 |        0 |      % |
This change adds the terms index (`.tip`) to the list of extensions
that are memory-mapped by hybridfs. These files used to be accessed
only once to load the terms index on-heap but since #42838 they can
now be used to read the binary FST directly so it is benefical to
memory-map them instead of accessing them via NIO.
@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Jun 12, 2019

@s1monw

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2019

awesome!

@s1monw
s1monw approved these changes Jun 12, 2019
s1monw added a commit to s1monw/elasticsearch that referenced this pull request Jun 12, 2019
…ctionary

Given the significant performance impact that NIOFS has when term dicts are
loaded off-heap this change enforces FstLoadMode#AUTO that loads term dicts
off heap only if the underlying index input indicates a memory map.

Relates to elastic#43150
@jimczi jimczi merged commit ef8f90c into elastic:master Jun 12, 2019
8 checks passed
8 checks passed
CLA All commits in pull request signed
Details
elasticsearch-ci/1 Build finished.
Details
elasticsearch-ci/2 Build finished.
Details
elasticsearch-ci/bwc Build finished.
Details
elasticsearch-ci/default-distro Build finished.
Details
elasticsearch-ci/docbldesx Build finished.
Details
elasticsearch-ci/oss-distro-docs Build finished.
Details
elasticsearch-ci/packaging-sample Build finished.
Details
@jimczi jimczi deleted the jimczi:mmap_terms_index branch Jun 12, 2019
jimczi added a commit that referenced this pull request Jun 12, 2019
This change adds the terms index (`.tip`) to the list of extensions
that are memory-mapped by hybridfs. These files used to be accessed
only once to load the terms index on-heap but since #42838 they can
now be used to read the binary FST directly so it is benefical to
memory-map them instead of accessing them via NIO.
s1monw added a commit that referenced this pull request Jun 13, 2019
…ctionary (#43158)

Given the significant performance impact that NIOFS has when term dicts are
loaded off-heap this change enforces FstLoadMode#AUTO that loads term dicts
off heap only if the underlying index input indicates a memory map.

Relates to #43150
s1monw added a commit that referenced this pull request Jun 13, 2019
…ctionary (#43158)

Given the significant performance impact that NIOFS has when term dicts are
loaded off-heap this change enforces FstLoadMode#AUTO that loads term dicts
off heap only if the underlying index input indicates a memory map.

Relates to #43150
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.