Skip to content

feat(telemetry): add specific PR, issue, and custom tracking IDs for GitHub Actions#21129

Open
cocosheng-g wants to merge 6 commits intogoogle-gemini:mainfrom
cocosheng-g:feat/add-github-action-telemetry-metrics
Open

feat(telemetry): add specific PR, issue, and custom tracking IDs for GitHub Actions#21129
cocosheng-g wants to merge 6 commits intogoogle-gemini:mainfrom
cocosheng-g:feat/add-github-action-telemetry-metrics

Conversation

@cocosheng-g
Copy link
Contributor

@cocosheng-g cocosheng-g commented Mar 4, 2026

Summary

This PR adds new telemetry attributes to the Gemini CLI to support granular usage metrics for GitHub Actions. These metrics will allow us to track unique Issues and PRs touched by the CLI, and distinguish between different workflow triggers (e.g., Automated vs. Scheduled triage).

Will fix #21130

Details

  • Added GEMINI_CLI_GH_EVENT_NAME to EventMetadataKey.
  • Added specific tracking IDs: GEMINI_CLI_GH_PR_NUMBER, GEMINI_CLI_GH_ISSUE_NUMBER, and GEMINI_CLI_GH_CUSTOM_TRACKING_ID replacing generic GH_EVENT_NUMBER.
  • Updated ClearcutLogger to capture these from environment variables (GITHUB_EVENT_NAME, GH_PR_NUMBER, GH_ISSUE_NUMBER, and GH_CUSTOM_TRACKING_ID).
  • The PR/Issue number is logged as a raw string ID (e.g., 123) to avoid fingerprinting PII, relying on the repository context to provide global uniqueness for backend queries.
  • Updated telemetry tests in clearcut-logger.test.ts to cover the new environment variables and metadata keys.
  • Updated telemetry documentation in docs/cli/telemetry.md.

Related Issues

Related to internal PM request for "Gemini CLI GitHub Action Usage Metric".

How to Validate

  1. Run the new tests: npm test -w @google/gemini-cli-core -- src/telemetry/clearcut-logger/clearcut-logger.test.ts
  2. Verify that when GITHUB_EVENT_NAME and specific IDs (GH_PR_NUMBER, etc.) are set in the environment, they appear in the Clearcut log metadata as expected.

Example events

Sample 1: api_request (Automated Issue Triage)
Triggered when the CLI sends a prompt to the model during an "Issue Opened" event.

{
  "event_name": "api_request",
  "console_type": "GEMINI_CLI",
  "application": 102,
  "event_metadata": [
    [
      { "gemini_cli_key": 20,  "value": "gemini-1.5-pro" },         // API_REQUEST_MODEL
      { "gemini_cli_key": 39,  "value": "GitHub" },                 // SURFACE
      { "gemini_cli_key": 130, "value": "gemini-triage" },          // GH_WORKFLOW_NAME
      { "gemini_cli_key": 132, "value": "a1b2c3d4..." },            // GH_REPO_HASH
      { "gemini_cli_key": 176, "value": "issues" },                 // GH_EVENT_NAME
      { "gemini_cli_key": 178, "value": "42" }                      // GH_ISSUE_NUMBER (Issue #42)
    ]
  ]
}

Sample 2: api_response (PR Review)
Triggered when the model returns a response for a Pull Request review.

{
  "event_name": "api_response",
  "console_type": "GEMINI_CLI",
  "application": 102,
  "event_metadata": [
    [
      { "gemini_cli_key": 21,  "value": "gemini-1.5-pro" },         // API_RESPONSE_MODEL
      { "gemini_cli_key": 25,  "value": "1240" },                   // INPUT_TOKEN_COUNT
      { "gemini_cli_key": 26,  "value": "450" },                    // OUTPUT_TOKEN_COUNT
      { "gemini_cli_key": 130, "value": "gemini-review" },          // GH_WORKFLOW_NAME
      { "gemini_cli_key": 176, "value": "pull_request" },           // GH_EVENT_NAME
      { "gemini_cli_key": 177, "value": "101" }                     // GH_PR_NUMBER (PR #101)
    ]
  ]
}

Sample 3: start_session (Scheduled Triage)
Triggered when the CLI starts up as part of a cron job (Scheduled).

{
  "event_name": "start_session",
  "console_type": "GEMINI_CLI",
  "application": 102,
  "event_metadata": [
    [
      { "gemini_cli_key": 1,   "value": "gemini-1.5-flash" },       // START_SESSION_MODEL
      { "gemini_cli_key": 130, "value": "gemini-issue-fixer" },     // GH_WORKFLOW_NAME
      { "gemini_cli_key": 176, "value": "schedule" },               // GH_EVENT_NAME
      { "gemini_cli_key": 179, "value": "101,102,103" }             // GH_CUSTOM_TRACKING_ID 
    ]
  ]
}

Pre-Merge Checklist

  • Updated relevant documentation and README (if needed)
  • Added/updated tests (if needed)
  • Noted breaking changes (if any)
  • Validated on required platforms/methods:
    • MacOS
    • Windows
    • Linux

@cocosheng-g cocosheng-g requested review from a team as code owners March 4, 2026 18:02
@gemini-code-assist

This comment was marked as outdated.

@cocosheng-g cocosheng-g self-assigned this Mar 4, 2026
gemini-code-assist[bot]

This comment was marked as outdated.

@gemini-cli gemini-cli bot added the area/enterprise Issues related to Telemetry, Policy, Quota / Licensing label Mar 4, 2026
@cocosheng-g cocosheng-g enabled auto-merge March 4, 2026 18:45
@cocosheng-g cocosheng-g requested a review from kschaab March 4, 2026 18:46
@cocosheng-g cocosheng-g changed the title feat(telemetry): add github action event name and hashed event number feat(telemetry): add github action event name and event number Mar 4, 2026
@cocosheng-g cocosheng-g changed the title feat(telemetry): add github action event name and event number feat(telemetry): add specific PR, issue, and custom tracking IDs for GitHub Actions Mar 6, 2026
@cocosheng-g cocosheng-g force-pushed the feat/add-github-action-telemetry-metrics branch from a8541db to 7a621a9 Compare March 6, 2026 17:55
…, issue, and custom tracking ids

- Replaced `GEMINI_CLI_GH_EVENT_NUMBER` with `GEMINI_CLI_GH_PR_NUMBER`, `GEMINI_CLI_GH_ISSUE_NUMBER`, and `GEMINI_CLI_GH_CUSTOM_TRACKING_ID` to allow tracking of specific GitHub Action contexts.
- Updated `ClearcutLogger` to extract `GH_PR_NUMBER`, `GH_ISSUE_NUMBER`, and `GH_CUSTOM_TRACKING_ID` from the environment.
- Updated telemetry unit tests to verify the new environment variables and metadata keys.
- Updated telemetry documentation in `docs/cli/telemetry.md` to reflect the new attributes.
@cocosheng-g cocosheng-g force-pushed the feat/add-github-action-telemetry-metrics branch from 7a621a9 to 8fcf4c2 Compare March 6, 2026 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/enterprise Issues related to Telemetry, Policy, Quota / Licensing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add granular GitHub Action usage metrics to telemetry

1 participant