Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Search Pipelines] Add stats for search pipelines #8053

Merged
merged 4 commits into from
Jun 28, 2023
Merged

[Search Pipelines] Add stats for search pipelines #8053

merged 4 commits into from
Jun 28, 2023

Commits on Jun 22, 2023

  1. [Search Pipelines] Add stats for search pipelines 

    This adds statistics on executions and time spent on search pipeline
operations, similar to the stats that are available for ingest
pipelines.

Signed-off-by: Michael Froh <froh@amazon.com>
    @msfroh
    msfroh committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    23730fc View commit details
    Browse the repository at this point in the history

  2. Compare parsed JSON structure, not exact JSON string 

    As @lukas-vlcek pointed out, asserting equality with an exact JSON
string is sensitive to formatting, which makes the test brittle.

Instead, we can parse the expected JSON and compare as Maps.

Signed-off-by: Michael Froh <froh@amazon.com>
    @msfroh
    msfroh committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    fa7a406 View commit details
    Browse the repository at this point in the history

  3. Refactor to common stats/metrics classes 

    Search pipelines and ingest pipelines had identical functionality for
tracking metrics around operations and converting those to immutable
"stats" objects.

That approach isn't even really specific to pipelines, but can be used
to track metrics on any repeated operation, so I moved that common
logic to the common.metrics package.

Signed-off-by: Michael Froh <froh@amazon.com>
    @msfroh
    msfroh committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    585231f View commit details
    Browse the repository at this point in the history

  4. Split pipeline metrics tracking into its own class 

    Thanks @saratvemulapalli for the suggestion! This lets the Pipeline
class focus on transforming requests / responses, while the subclass
focuses on tracking and managing metrics.

Signed-off-by: Michael Froh <froh@amazon.com>
    @msfroh
    msfroh committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    159e568 View commit details
    Browse the repository at this point in the history