Add extended sysinfo for Edge worker

[jscheffl]

With this PR the Edge Worker is able to supply additional metrics from the remote location to the central site. This allows to collect information from remote in order to see health and more details of the remote worker.

As now more data is collected alongside with this PR the worker UI is cleaned and all (previously listed) metrics and details from the worker have been moved to a tooltip/popup.

In a follow-up PR this function also can be used to set the worker automatically into maintenance (and back alive) if the health is not good to prevent failed jobs.

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[Copilot]

Pull request overview

This PR extends Edge Worker “sysinfo” reporting so the central site can store/display richer worker health data, publish additional metrics, and simplify the worker UI by moving detailed info into a tooltip.

Changes:

• Add extended sysinfo collection on the worker (including status/health fields) and persist sysinfo as JSON in the Edge3 DB schema.
• Emit new/extended observability metrics (including edge_worker.status) and update API schema/type hints for sysinfo values.
• Update the Edge3 UI worker list to show a compact “System Status” badge with hover details; add/adjust unit tests and docs.

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml	Adds the edge_worker.status gauge metric definition.
providers/edge3/src/airflow/providers/edge3/cli/worker.py	Collects/merges extended sysinfo and sends it on heartbeat/register/offline transitions.
providers/edge3/src/airflow/providers/edge3/cli/api_client.py	Updates typing for sysinfo payload sent to the central API.
providers/edge3/src/airflow/providers/edge3/cli/example_extended_sysinfo.py	Adds an example async sysinfo provider implementation.
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py	Switches sysinfo to JSON and expands metrics emission from sysinfo fields.
providers/edge3/src/airflow/providers/edge3/models/db.py	Bumps Edge3 DB revision mapping for 3.5.0.
providers/edge3/src/airflow/providers/edge3/migrations/versions/0005_3_5_0_replace_individual_counters_with_.py	Migrates sysinfo to JSON and drops individual job counters.
providers/edge3/src/airflow/providers/edge3/worker_api/datamodels.py	Expands sysinfo value types to include float/date-time and adds status example.
providers/edge3/src/airflow/providers/edge3/worker_api/routes/worker.py	Stores sysinfo as JSON and propagates it into metrics; updates version-compat stats behavior.
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py	Refactors stats emission paths using version compatibility.
providers/edge3/src/airflow/providers/edge3/worker_api/routes/ui.py	Switches UI route to return JSON sysinfo directly.
providers/edge3/src/airflow/providers/edge3/worker_api/v2-edge-generated.yaml	Updates API examples/schema to include status and broader sysinfo value types.
providers/edge3/src/airflow/providers/edge3/executors/edge_executor.py	Resets worker “presented status” when worker is detected lifeless.
providers/edge3/src/airflow/providers/edge3/plugins/www/src/pages/WorkerPage.tsx	Simplifies worker table columns and uses the new status badge/hover UI.
providers/edge3/src/airflow/providers/edge3/plugins/www/src/components/WorkerSysinfoBadge.tsx	New UI component for health badge + hover details of sysinfo/heartbeats.
providers/edge3/tests/unit/edge3/cli/test_worker.py	Updates sysinfo tests for async sysinfo collection and extended sysinfo.
providers/edge3/tests/unit/edge3/worker_api/routes/test_worker.py	Uses a stable mock sysinfo payload for worker API route tests.
providers/edge3/tests/unit/edge3/worker_api/routes/test_jobs.py	Updates stats import/version-compat test logic.
providers/edge3/tests/unit/edge3/executors/test_edge_executor.py	Extends executor tests to cover sysinfo status/status_text updates.
providers/edge3/tests/unit/edge3/models/test_db.py	Updates migration tests for new Edge3 DB head revision and new schema state.
providers/edge3/docs/deployment.rst	Adds “Worker Monitoring” documentation and screenshot reference.
providers/edge3/docs/migrations-ref.rst	Documents the new 3.5.0 migration revision.
providers/edge3/docs/img/edge_sysinfo.png	Adds UI screenshot for sysinfo tooltip/badge.
airflow-core/tests/unit/always/test_project_structure.py	Adds an Edge3 test file to the “providers must have tests” allowlist.
airflow-core/tests/unit/always/test_example_dags.py	Narrows provider “example_*.py” glob to avoid non-DAG example files.

[dheerajturaga]

Awesome!

[AutomationDev85]

Looks like a really nice improvement!