osquery_manager: generate ECS section mappings from config

[marc-gr]

Proposed commit message

osquery_manager: generate ECS section mappings from config

Move the osquery-gen ECS keep list into config.yml and pass it through the generator instead of embedding a separate text file. This lets the generated ecs.yml include object/nested parent mappings such as dll.pe.sections, file.pe.sections, and related threat.*.file.pe.sections fields consistently when artifacts are regenerated.

Refresh generated osquery artifacts for osquery 5.23.0 and bump the package to 1.29.0 so the changelog captures both the ECS generation fix and osquery version update.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• Regenerated osquery_manager artifacts with osquery-gen.
• Confirmed the PR osquery_manager: update ECS mapping #16650 ECS parent *.sections fields are generated in data_stream/result/fields/ecs.yml.

How to test this PR locally

cd packages/osquery_manager/_dev/scripts/osquery-gen
go run . -config ./config.yml -skip-package-check
go test ./...

Also run:

git diff --check

Not run locally: full elastic-package check.

Related issues

• Relates osquery_manager: update ECS mapping #16650

Screenshots

N/A

[github-actions]

Vale Linting Results

Summary: 1 warning found

⚠️ Warnings (1)

File	Line	Rule	Message
packages/osquery_manager/_dev/scripts/osquery-gen/README.md	7	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.

---

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

[infra-vault-gh-plugin-prod]

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: fc6c4a1

Failed CI Steps

• Check integrations osquery_manager

History

[github-actions]

TL;DR

The failure is configuration-related, not a test/code regression: CI tries to boot Elastic Stack 9.4.2, but Docker images for that tag are unavailable (manifest unknown).
The immediate fix is to avoid pinning osquery_manager to an unavailable patch version.

Remediation

• Update packages/osquery_manager/manifest.yml to use a supported Kibana constraint (e.g. revert conditions.kibana.version from ^9.4.2 to ^9.4.0, or another currently published version).
• Re-run the Buildkite job after the manifest constraint change.

Investigation details

Root Cause

packages/osquery_manager/manifest.yml:11 in this PR changed Kibana constraint from ^9.4.0 to ^9.4.2, which drives stack selection during package testing.

/.buildkite/scripts/common.sh:479-497 computes the stack version from package constraints when STACK_VERSION is unset. The test step then calls elastic-package stack up --version <resolved-version>.

Buildkite failed while pulling stack images for 9.4.2, so tests never reached package logic.

Evidence

• Build: https://buildkite.com/elastic/integrations/builds/42784
• Job/step: Check integrations osquery_manager
• PR diff evidence:
  • packages/osquery_manager/manifest.yml
    • conditions.kibana.version: "^9.4.0" → "^9.4.2"
• Key log excerpts:
  • Error response from daemon: manifest for docker.elastic.co/elastic-agent/elastic-agent-wolfi:9.4.2 not found: manifest unknown
  • Error response from daemon: manifest for docker.elastic.co/elasticsearch/elasticsearch:9.4.2 not found: manifest unknown
  • booting up the stack failed ... exit status 18: manifest ... not found

Verification

• Not run locally in this workflow (read-only detective analysis).

Follow-up

If 9.4.2 is truly required for a specific API/behavior, use a stack version/tag that is actually published in CI (or wait for artifact availability) before re-enabling that constraint.

Note

🔒 Integrity filter blocked 2 items

The following items were blocked because they don't meet the GitHub integrity level.

• #18989 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• osquery_manager: generate ECS section mappings from config #18989 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.