Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge histograms without losing precision #93704

Commits on Feb 10, 2023

  1. fix: use max value 3 for numberOfSignificantValueDigits

    For unmapped histogram fields we instantiate by default a DoubleHistogram
    with 3 significant digits precision. The test generates a random value
    for the number of significant digits that is in the range [0, 5].
    As a result, if the test runs with 4 or 5 significant value digits but
    the HdrHistogram sketch only uses 3, checking errors on results will fail.
    
    Here we change the maximum value for the significant value digits to 3
    if the query involves an index with unmapped fields.
    salvatore-campagna committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    98ff032 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fb6ac0d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f49036f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    39b1c42 View commit details
    Browse the repository at this point in the history
  5. fix: update yaml

    salvatore-campagna committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    1f5fc39 View commit details
    Browse the repository at this point in the history
  6. fix: always merge histograms using the higher precision

    Normallt we would create a histogram with the correct number of digits,
    nut for empty histograms we end up serializing and deserializing large
    arrays for empty aggregations. Here we have a kind of workaroud were we
    use just 3 digits for empty histograms and, at reduce time, we always
    merge usign the larger number of digits among all histograms.
    salvatore-campagna committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    00fa30b View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    8721bc9 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    18b79a6 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    da68312 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    7c2f22b View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    4fc598a View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    6683480 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    c1be8c2 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    44e739e View commit details
    Browse the repository at this point in the history
  15. fix: restore test

    salvatore-campagna committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    9f8907f View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2023

  1. Configuration menu
    Copy the full SHA
    7eac3f8 View commit details
    Browse the repository at this point in the history
  2. fix: prevent serialization of empty tdigest objects

    When an empty aggregations uses a TDigest object the udnelrying
    arrays used by the AvlTree TDigest implementation is eagerly
    allocated. If the aggreation produces no result, we serialize and
    deserialize an array which might be large if the comrpession value
    is large (about 5 * compression centroids are tracked). Here we
    use a null value for empty aggreations while building the result
    and later on use a static empty TDigest object at reduce time and
    merge it with the non-empty results.
    salvatore-campagna committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    5b13951 View commit details
    Browse the repository at this point in the history
  3. fix: undo changes to TDigest percentiles

    This change will end up in another PR focused on improving
    memory usage for TDigest percentiles aggregations.
    salvatore-campagna committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    30126e7 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    c57b68a View commit details
    Browse the repository at this point in the history