Skip to content

[metric] Add key-value metric group for model and action dimensions#760

Merged
yunfengzhou-hub merged 3 commits into
apache:mainfrom
yunfengzhou-hub:metric-group-kv
Jun 5, 2026
Merged

[metric] Add key-value metric group for model and action dimensions#760
yunfengzhou-hub merged 3 commits into
apache:mainfrom
yunfengzhou-hub:metric-group-kv

Conversation

@yunfengzhou-hub
Copy link
Copy Markdown
Contributor

Linked issue: #744

Purpose of change

Use key-value metric groups for action and model dimensions instead of plain named sub-groups. This makes these dimensions available as labels in reporters that support them (e.g., Prometheus), and produces more structured metric identifiers:

  • Action metrics: <action_name>.numOfActionsExecutedaction.<action_name>.numOfActionsExecuted
  • Token metrics: <action_name>.<model_name>.promptTokensaction.<action_name>.model.<model_name>.promptTokens

Tests

  • Unit tests (value-based and mock-based) verify the key-value group hierarchy and counter accumulation.
  • An end-to-end integration test using InMemoryReporter + MiniClusterExtension validates the complete metric identifier through the real Flink metric system.

API

  • Breaking: The metric identifiers for promptTokens, completionTokens, and per-action execution counters now include the key prefix (action., model.). Metric reporter queries or dashboards referencing the old names need updating.
  • A new getSubGroup(String key, String value) method is added to the FlinkAgentsMetricGroup interface (Java) and get_sub_group(name, value) (Python).
  • getSubGroup(String name) now rejects names containing = to prevent cache-key collisions with the key-value variant.

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

@github-actions github-actions Bot added doc-included Your PR already contains the necessary documentation updates. fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue. labels Jun 4, 2026
@joeyutong
Copy link
Copy Markdown
Collaborator

LGTM once style check is fixed.

@yunfengzhou-hub yunfengzhou-hub merged commit e94da40 into apache:main Jun 5, 2026
25 checks passed
yunfengzhou-hub added a commit to yunfengzhou-hub/flink-agents that referenced this pull request Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-included Your PR already contains the necessary documentation updates. fixVersion/0.3.0 The feature or bug should be implemented/fixed in the 0.3.0 version. priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants