Skip to content

Define dataflows for container-integrations-owned integrations - #3102

Open
philjlee wants to merge 1 commit into
masterfrom
philip.lee/dataflows-extras-batch1-container-integrations
Open

Define dataflows for container-integrations-owned integrations#3102
philjlee wants to merge 1 commit into
masterfrom
philip.lee/dataflows-extras-batch1-container-integrations

Conversation

@philjlee

@philjlee philjlee commented Aug 6, 2026

Copy link
Copy Markdown

Jira: TXP-277

Adds assets/dataflows.yaml to 1 integration owned by @DataDog/container-integrations, declaring 2 dataflows. Mechanical change: no code, no metrics, no manifests touched.

Precedent: #2925 "Define dataflows for saas-integrations", which created 13 of the 14 existing dataflow files in this repo.

Selection criteria

A directory is in this batch only if all of the following hold:

  1. It has no assets/dataflows.yaml today.
  2. It has a parseable manifest.json. The dataflows validator hard-requires one — dataflows_validation_handler.go:56.
  3. The correct data_type is mechanically derivable from a committed artifact, with no judgement call:
    • metadata.csv with at least one data row → metrics
    • a log pipeline under assets/logs/*.yamllogs
    • both → both entries
  4. .github/CODEOWNERS resolves <dir>/assets/dataflows.yaml (last-match-wins) to @DataDog/container-integrations.

aqua/ has @DataDog/container-integrations as its sole Datadog team CODEOWNER, so review from that team covers the whole PR.

Field values

provides:
  - id: <app_id>-<data_type>
    always_on: true
    granular: false
    data_type: <metrics|logs>
    direction: inbound

always_on: true / granular: false / direction: inbound matches 14 of the 17 dataflow entries already in the repo. The only deviation in the repo is vercel, which is always_on: false for a Serverless product-enablement reason that does not apply here.

Dataflow IDs are <app_id>-<data_type>, taking app_id from manifest.json rather than the directory name. Both IDs were checked for collisions against every existing ID in the repo.

Validation

This file was validated by executing the real validator, DataflowsValidationHandler from dd-source/domains/integrationscatalog/libs/catalogassetslib/dataflows_validation_handler.go, at ddoghq/dd-source@main, against every dataflows.yaml file in the repo working tree with this PR applied. Result: 0 failures, covering per-file unmarshalling, proto constraint validation, and the cross-file HandleLibrary ID-uniqueness check.

The harness was negative-tested first and confirmed to reject: a missing always_on; a data_type outside validDataTypes; an id breaking ^[a-z0-9-]+$; an id under 3 characters; a file with neither provides nor uses; a .yml extension; a missing manifest.json; and the same dataflow ID provided by two apps.

This matters because APW does not post validator comments on integrations-extras PRs (enable_validator_comments is set only for pub-platform-staging and publishing-platform). A malformed dataflows.yaml merges cleanly here and only fails afterwards, in the shared asset pipeline.

Contents

Metrics and logs (1)

Integration Dataflow IDs
aqua aqua-metrics, aqua-logs

🤖 Generated with Claude Code

Split of #3093 by CODEOWNERS team.
Jira: TXP-277
@philjlee
philjlee requested a review from a team as a code owner August 6, 2026 19:18
@philjlee philjlee changed the title Define dataflows for container-integrations-owned integrations (batch 1c) Define dataflows for container-integrations-owned integrations Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants