New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DM-21919: Run ap_verify end-to-end in Gen 3 #97
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think everything is minor/typo related.
This brings alert handling in line with database and config handling.
# - location: $AP_VERIFY_DIR/pipelines/MetricsKpm.yaml # Currently empty, therefore invalid | ||
- location: $AP_VERIFY_DIR/pipelines/MetricsRuntime.yaml | ||
- location: $AP_VERIFY_DIR/pipelines/MetricsMisc.yaml | ||
tasks: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In writing pipelines I've been either writing meta pipelines with only inherits
blocks or pipelines with only tasks
blocks. I agree, in this case, that having the two mixed makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So are you saying you would create an ApPipeForApVerify.yaml
that does the config override, then import it in ApVerify.yaml
in place of the original ApPipe.yaml
?
How ap_verify
handles pipelines is likely to get cargo-culted by other AP and/or metrics users, so it's worth thinking about whether it's doing things "right".
These pipelines chain the AP pipeline with packages of metrics. These are equivalent to the metrics defined in the config/ directory.
This driver mostly delegates to the pipetask application, but with a few customizations to use ap_verify infrastructure.
This mode calls the existing wrappers for Gen 3 ingestion and Gen 3 pipeline execution. The documentation has been updated to reflect Gen 3 behavior.
This lets users customize the metrics they are running in Gen 3, much as they can in Gen 2.
685e26e
to
7d89571
Compare
This PR implements the ability to run
ap_verify
entirely in Gen 3. This is a preliminary implementation designed to get something working in CI; see Jira for a list of ticketed improvements.This PR is relative to
tickets/DM-26070
to make it easy to see what is changing on this ticket; it must not be merged tomaster
before #96.