chore(config): build the first components from the typed configuration model#1979
chore(config): build the first components from the typed configuration model#1979webern wants to merge 2 commits into
Conversation
|
Binary Size Analysis (Agent Data Plane)Baseline: 2715682 · Comparison: 0a7a0e6 · diff ✅ Binary size difference within thresholdChanges by Module
Detailed Symbol Changes |
Regression Detector (Agent Data Plane)Run ID: Optimization Goals: ✅ No significant changes detectedFine details of change detection per experiment (3)Experiments configured
Bounds Checks: ✅ Passed (3)
ExplanationA change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression ( |
## AI Summary Build the OTTL filter processor from the resolved traces-domain configuration instead of parsing `GenericConfiguration` in the component. Remove the component-owned source serde configuration and use the shared `OttlErrorMode` model type. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` - `make check-all` - `make test` - `make check-docs` - Targeted OTTL filter processor tests ## References Related to configuration cutover PR #1979.
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
2523534 to
9353c90
Compare
## AI Summary Build the OTTL filter processor from the resolved traces-domain configuration instead of parsing `GenericConfiguration` in the component. Remove the component-owned source serde configuration and use the shared `OttlErrorMode` model type. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` - `make check-all` - `make test` - `make check-docs` - Targeted OTTL filter processor tests ## References Related to configuration cutover PR #1979.
## AI Summary Build the OTTL filter processor from the resolved traces-domain configuration instead of parsing `GenericConfiguration` in the component. Remove the component-owned source serde configuration and use the shared `OttlErrorMode` model type. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` - `make check-all` - `make test` - `make check-docs` - Targeted OTTL filter processor tests ## References Related to configuration cutover PR #1979.
## AI Summary Build the OTTL filter processor from the resolved traces-domain configuration instead of parsing `GenericConfiguration` in the component. Remove the component-owned source serde configuration and use the shared `OttlErrorMode` model type. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` - `make check-all` - `make test` - `make check-docs` - Targeted OTTL filter processor tests ## References Related to configuration cutover PR #1979.
## AI Summary Build the OTTL filter processor from the resolved traces-domain configuration instead of parsing `GenericConfiguration` in the component. Remove the component-owned source serde configuration and use the shared `OttlErrorMode` model type. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` - `make check-all` - `make test` - `make check-docs` - Targeted OTTL filter processor tests ## References Related to configuration cutover PR #1979.
## AI Summary Build the OTTL filter processor from the resolved traces-domain configuration instead of parsing `GenericConfiguration` in the component. Remove the component-owned source serde configuration and use the shared `OttlErrorMode` model type. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` - `make check-all` - `make test` - `make check-docs` - Targeted OTTL filter processor tests ## References Related to configuration cutover PR #1979.
## Human Summary There's some weird stuff going on here with preserving "absence" in an attempt to preserve exact behavioral parity, but I think it's OK. ## AI Summary Cuts the OTLP components over from the raw `GenericConfiguration` map to the typed `SalukiConfiguration` model, following the PR #1979 pattern: `OtlpConfig` (shared receiver config), the OTLP source, decoder, forwarder, and relay. Each component now builds from typed model slices (`domains.otlp`, `domains.traces.otlp`) instead of deserializing the raw config: - **Source / relay / decoder / forwarder**: `from_configuration` takes borrowed model slices; the call site in `run.rs` reads them from the config system. Component structs carry no serde and no defaults. - **Seeded OTLP knobs** (context sizing, HTTP receiver transport, trace interner size, top-level-by-span-kind) are converted to the required-with-default doctrine: non-optional in `SalukiOnly` with the default defined once as a shared `const`/`fn` in `agent-data-plane-config` and referenced by both the model `Default` and the `SalukiOnly` serde default. The two byte-size interner keys are typed `ByteSize` so they accept both a bare integer and a suffixed string. - **`OtlpTracesTranslator`** now takes the two native booleans it actually reads instead of the whole raw `TracesConfig`. `TracesConfig` is kept only for the not-yet-migrated Datadog trace encoder. - The per-component config smoke tests are retired (coverage now lives in the translator and construction tests). ### Behavior notes Because the OTLP receiver keys are witnessed against the vendored Datadog schema, their defaults now come from that schema rather than Saluki's former hardcoded values. When a key is left unset: - gRPC/HTTP receiver endpoints default to `localhost:4317` / `localhost:4318` (previously `0.0.0.0:...`), matching the Datadog Agent. - `otlp_config.logs.enabled` defaults to `false` (previously `true`), matching the Datadog Agent. - `otlp_config.receiver.protocols.grpc.max_recv_msg_size_mib` of `0` now maps to grpc-go's built-in 4 MiB limit, matching the Agent and the schema documentation (previously `0` produced a 0-byte limit). Deployments that set these keys explicitly (including the OTLP correctness cases, which pin the gRPC endpoint to `0.0.0.0:4317`) are unaffected. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` (no generated drift), `make check-all`, `make check-docs`. - Unit tests for `agent-data-plane-config`, `agent-data-plane-config-system`, `saluki-components`, and `agent-data-plane`, including new `SalukiOnly` transport/default tests (both byte-size input forms) and per-component construction tests. - Statically reviewed the OTLP correctness cases: all send over gRPC on the explicitly configured `0.0.0.0:4317`, so the receiver default changes above do not affect them. ## References - Progresses #1788 - Merges into `m/pr5-cutover`
## AI Summary Build the OTTL filter processor from the resolved traces-domain configuration instead of parsing `GenericConfiguration` in the component. Remove the component-owned source serde configuration and use the shared `OttlErrorMode` model type. ## Change Type - [x] Non-functional (chore, refactoring, docs) ## How did you test this PR? - `make build-schema-overlay && make fmt` - `make check-all` - `make test` - `make check-docs` - Targeted OTTL filter processor tests ## References Related to configuration cutover PR #1979.
There's some weird stuff going on here with preserving "absence" in an attempt to preserve exact behavioral parity, but I think it's OK. Cuts the OTLP components over from the raw `GenericConfiguration` map to the typed `SalukiConfiguration` model, following the PR #1979 pattern: `OtlpConfig` (shared receiver config), the OTLP source, decoder, forwarder, and relay. Each component now builds from typed model slices (`domains.otlp`, `domains.traces.otlp`) instead of deserializing the raw config: - **Source / relay / decoder / forwarder**: `from_configuration` takes borrowed model slices; the call site in `run.rs` reads them from the config system. Component structs carry no serde and no defaults. - **Seeded OTLP knobs** (context sizing, HTTP receiver transport, trace interner size, top-level-by-span-kind) are converted to the required-with-default doctrine: non-optional in `SalukiOnly` with the default defined once as a shared `const`/`fn` in `agent-data-plane-config` and referenced by both the model `Default` and the `SalukiOnly` serde default. The two byte-size interner keys are typed `ByteSize` so they accept both a bare integer and a suffixed string. - **`OtlpTracesTranslator`** now takes the two native booleans it actually reads instead of the whole raw `TracesConfig`. `TracesConfig` is kept only for the not-yet-migrated Datadog trace encoder. - The per-component config smoke tests are retired (coverage now lives in the translator and construction tests). Because the OTLP receiver keys are witnessed against the vendored Datadog schema, their defaults now come from that schema rather than Saluki's former hardcoded values. When a key is left unset: - gRPC/HTTP receiver endpoints default to `localhost:4317` / `localhost:4318` (previously `0.0.0.0:...`), matching the Datadog Agent. - `otlp_config.logs.enabled` defaults to `false` (previously `true`), matching the Datadog Agent. - `otlp_config.receiver.protocols.grpc.max_recv_msg_size_mib` of `0` now maps to grpc-go's built-in 4 MiB limit, matching the Agent and the schema documentation (previously `0` produced a 0-byte limit). Deployments that set these keys explicitly (including the OTLP correctness cases, which pin the gRPC endpoint to `0.0.0.0:4317`) are unaffected. - [x] Non-functional (chore, refactoring, docs) - `make build-schema-overlay && make fmt` (no generated drift), `make check-all`, `make check-docs`. - Unit tests for `agent-data-plane-config`, `agent-data-plane-config-system`, `saluki-components`, and `agent-data-plane`, including new `SalukiOnly` transport/default tests (both byte-size input forms) and per-component construction tests. - Statically reviewed the OTLP correctness cases: all send over gRPC on the explicitly configured `0.0.0.0:4317`, so the receiver default changes above do not affect them. - Progresses #1788 - Merges into `m/pr5-cutover`
3a5cfe3 to
24621f3
Compare
Cut the aggregate transform over from raw GenericConfiguration access to the typed SalukiConfiguration model. AggregateConfiguration builds from the dogstatsd aggregation slice and the shared histogram encoding; histogram parsing moves into HistogramConfiguration::from_encoding, and the component config struct no longer carries a Default or deserializes Datadog config. The aggregation slice is made authoritative from the Agent schema, dropping the redundant Saluki-only aggregation keys. run.rs reads the typed config and hands the transform its subtree.
…onfig Cut the debug-log destination over to the typed SalukiConfiguration model. DogStatsDDebugLogConfiguration builds from the dogstatsd debug-log slice and holds a Live view, so the destination reacts to metrics-stats changes through the live view instead of a string-key watch. run.rs hands it the typed subtree and mints the live view from the configuration system.

Human Summary
This PR serves as a foundational model for cutting over components to typed config. It serves two purposes:
An issue with dotted path environment variable support also had to be dealt with in this PR rather than laboriously shuttling it downward in the stack.
Key aliases that seemed to have arisen by mistake, such that we have a "saluki only" key for something that is Datadog schema-provided are treated as bugs as we go and the "saluki-only" aliases will be removed. I believe there are only a couple cases of this, but one of them happens to be in this PR.
AI Summary
Establishes the reference pattern for constructing pipeline components from the typed
SalukiConfigurationmodel instead of the rawGenericConfigurationmap, and cuts two DogStatsDcomponents over to it.
AggregateConfigurationbuilds from the typed DogStatsDaggregation slice and shared histogram-encoding settings. Source serde behavior is removed from
the component. Histogram parsing moves to a fallible conversion, and construction rejects a zero
aggregation window.
DogStatsDDebugLogConfigurationbuilds from the typeddebug-log slice and holds a
Live<DebugLog>. The destination reacts to runtime changes throughthe typed live view instead of watching a raw string key.
Topology assembly now supplies each component its typed configuration subtree and, where needed, a
live view from
ConfigurationSystem.The aggregate cutover also fixes two duplicate configuration paths:
dogstatsd_flush_incomplete_bucketsnow drives the aggregation field the transform reads; theduplicate Saluki-only path is removed.
dogstatsd_expiry_secondsis the sole source for counter expiry; the duplicate Saluki-only key isremoved.
Translation seeds Saluki-only values first and applies the Datadog source afterward, making the
Datadog source authoritative for every field it owns. Obsolete component-level configuration smoke
tests are removed; equivalent coverage lives in translation and component construction tests.
Change Type
How did you test this PR?
cargo testforsaluki-components,agent-data-plane-config, andagent-data-plane-config-systemcargo check -p agent-data-planeIntegration and correctness suites were not run locally.
References