Skip to content

[Elastic Workflows] add elastic_workflows Fleet content package#18433

Open
rosomri wants to merge 9 commits intoelastic:mainfrom
rosomri:feat/elastic-workflows-content-package
Open

[Elastic Workflows] add elastic_workflows Fleet content package#18433
rosomri wants to merge 9 commits intoelastic:mainfrom
rosomri:feat/elastic-workflows-content-package

Conversation

@rosomri
Copy link
Copy Markdown

@rosomri rosomri commented Apr 16, 2026

Summary

Adds the elastic_workflows Fleet content package — a dashboard-only integration for monitoring Elastic Workflows executions, performance, and failure rates.

Package contents

  • Dashboard: "[Elastic Workflows] Execution Overview" with 15 Lens panels:
    • KPI strip: Total Executions, Avg Duration, Slowest Workflow, Success Rate, Timed Out (with trendline), Failures (with trendline)
    • Executions Over Time (stacked bar by workflow)
    • Trigger Breakdown (treemap)
    • Failure Rate Trend per workflow
    • Duration Distribution (bucketed bar)
    • Avg Duration Over Time per workflow
    • Status Breakdown (treemap)
    • Slowest Workflows (p95 table)
    • Recent Failures (table with drilldown to workflow executions)
    • Per-Workflow Summary (table with drilldown to workflow page)
  • Pinned controls: isTestRun filter (excludes test runs by default) and spaceId filter
  • All panels use an ad-hoc data view (adhoc_wf) pointing to .workflows-executions

Dev scripts (_dev/scripts/)

Three scripts support dashboard development and maintenance:

  • build_dashboard.py: Programmatically defines the dashboard in a readable Python DSL (panel builders, shared constants for KQL queries, ad-hoc data views). Outputs an intermediate JSON to _dev/shared/create_final_dashboard.json.

  • build_so.py: Takes the intermediate representation and builds the final Kibana saved-object JSON at kibana/dashboard/<id>.json. Handles UUID generation for Lens column IDs, reference collection, panelsJSON/optionsJSON string encoding, and the typeMigrationVersion field. This is the authoritative builder for the dashboard.

  • import_dashboard_ndjson.py: Converts a Kibana saved-object NDJSON export back into the integration format. Useful when the dashboard is edited directly in Kibana and re-exported. It:

    • Strips transient fields (accessControl, created_at, updated_at, created_by, updated_by, version, managed, coreMigrationVersion)
    • Re-IDs the object to match the integration's expected dashboard ID
    • Re-injects adHocDataViews definitions that Kibana strips on export (checks both indexPatternId in layers and the references array)
    • Usage: python3 import_dashboard_ndjson.py <path-to-ndjson>

Metadata

  • Package type: content (no data streams or agent policies)
  • Owner: elastic/workflows-eng
  • Minimum Kibana: ^9.3.0
  • Subscription: enterprise

Screenshots

Screenshot 2026-05-03 at 14 54 51 image Screenshot 2026-05-03 at 14 25 38

rosomri added 2 commits April 16, 2026 17:14
Introduce a content package with a Workflows execution overview dashboard
for existing `.workflows-executions` data (Kibana 9.3+). Includes `_dev`
scripts to regenerate the dashboard JSON.

Closes elastic/security-team#15773
@rosomri rosomri requested a review from a team as a code owner April 16, 2026 14:34
@rosomri rosomri added the enhancement New feature or request label Apr 16, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 16, 2026

✅ Vale Linting Results

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

@andrewkroh andrewkroh added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. New Integration Issue or pull request for creating a new integration package. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. labels Apr 16, 2026
Comment thread packages/elastic_workflows/changelog.yml Outdated
@rosomri rosomri added the Team:One Workflow Workflow Automation team [elastic/workflows-eng] label Apr 21, 2026
Comment thread packages/elastic_workflows/changelog.yml Outdated
@andrewkroh andrewkroh removed dashboard Relates to a Kibana dashboard bug, enhancement, or modification. New Integration Issue or pull request for creating a new integration package. labels Apr 23, 2026
…script

Update the dashboard saved object with the latest Kibana export:
- New KPI panels: Slowest Workflow (max duration), Timed Out (with trendline)
- Failures KPI now includes a trendline sparkline
- Compact KPI layout with hidden titles and valueFontMode: fit
- Treemap visualizations for Trigger Breakdown and Status Breakdown
- Duration formatting using humanizePrecise instead of raw ms
- Badge color mode for Per-Workflow Summary table columns
- Pinned panel controls: isTestRun (exclude test runs) and spaceId filter

Add _dev/scripts/import_dashboard_ndjson.py:
- Converts a Kibana saved-object NDJSON export to the integration format
- Strips transient fields (accessControl, created_at, updated_at, etc.)
- Re-IDs the object to match the integration's expected dashboard ID
- Re-injects adHocDataViews definitions that Kibana strips on export
  (checks both indexPatternId in layers and the references array)

Usage: python3 import_dashboard_ndjson.py <path-to-ndjson>
@rosomri rosomri requested a review from dej611 May 3, 2026 10:58
rosomri added 4 commits May 3, 2026 14:11
Kibana omits fromUnit on export, causing duration columns to default to
seconds instead of milliseconds. Fix the Avg Duration KPI and add
automatic fix_duration_formats to the import script for future exports.
… subscription

Dashboard best-practice alignment:
- Rename panel titles to be self-explanatory (remove redundant suffixes)
- Hide axis titles on all XY charts via axisTitlesVisibilitySettings
- Clear redundant column labels used as implicit axis titles
- Fix Per-Workflow Summary duration columns to use humanizePrecise

Update README to reflect current dashboard panels and data sources.
Change subscription requirement from basic to enterprise.
Remove build_dashboard.py and its intermediate output
create_final_dashboard.json. The dashboard is now maintained by editing
in Kibana and importing via import_dashboard_ndjson.py.
@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Team:One Workflow Workflow Automation team [elastic/workflows-eng]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants