Skip to content

feat(config): add [environment] and [defaults] config sections for auto metadata#692

Merged
kaihowl merged 8 commits into
masterfrom
claude/open-issues-review-IcIqQ
May 24, 2026
Merged

feat(config): add [environment] and [defaults] config sections for auto metadata#692
kaihowl merged 8 commits into
masterfrom
claude/open-issues-review-IcIqQ

Conversation

@kaihowl
Copy link
Copy Markdown
Owner

@kaihowl kaihowl commented May 24, 2026

Summary

Closes #320

  • Adds [environment] config section mapping metadata keys to environment variable names (single string or first-found array)
  • Adds [defaults] config section for static fallback values when env vars are unset
  • Adds --skip-env flag to measure, add, and import commands to bypass environment lookups

Precedence (highest to lowest): CLI --key-value / --metadata > [environment] > [defaults]

Example config:

[environment]
commit   = "GITHUB_SHA"
branch   = "GITHUB_REF_NAME"
runner   = ["RUNNER_NAME", "CI_RUNNER_ID"]   # first non-empty wins

[defaults]
environment = "local"

Test plan

  • 11 new Rust unit tests in config.rs covering all precedence layers, multi-source fallback, and empty-var guard
  • New bash integration test test/test_env_defaults_config.sh with 13 sections covering add, measure, import, --skip-env for all three commands, and no-config baseline
  • Run cargo nextest run -- --skip slow (all pass except pre-existing test_remove libfaketime failure)
  • Manpages regenerated for git-perf-add, git-perf-measure, git-perf-import
  • Documented in CLAUDE.md, README.md, docs/example_config.toml, docs/importing-measurements.md with security note

https://claude.ai/code/session_01EU3bMeB229LZVQpFFUBXNw

…to metadata

Allows users to define metadata sources in .gitperfconfig:

  [environment]
  commit = "GITHUB_SHA"                   # single env var
  runner_id = ["RUNNER_NAME", "CI_RUNNER_ID"]  # first-found wins

  [defaults]
  environment = "local"                   # static fallback

Precedence (highest to lowest): CLI --key-value > [environment] > [defaults].

Adds --skip-env flag to measure, add, and import commands to bypass
[environment] lookups. Sensitive variable names containing TOKEN or SECRET
are blocked and logged as warnings.

Closes #320

https://claude.ai/code/session_01A9QXKgBLkLZaeRxJt57bus
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 24, 2026

Performance Report

Performance Results

Audit Results

Auditing measurement "add-benchmark" (os=ubuntu-22.04/rust=beta):
  ✅ 'add-benchmark'
  Aggregation: min
  z-score (mad): ↓ 1.11
  Head: μ: 16.3ms σ: N/A MAD: 0ns n: 1
  Tail: μ: 18.9ms σ: 3.4ms MAD: 2.3ms n: 11
   [-26.66% – +19.38%] ▇▁▁▁█▇▅▁▅▅▅▂

Auditing measurement "add-benchmark" (os=ubuntu-22.04/rust=stable):
  ⏭️ 'add-benchmark'
  Only 9 historical measurements found. Less than requested min_measurements of 10. Skipping test.
  Aggregation: min
  z-score (mad): ↑ 1.08
  Head: μ: 23.4ms σ: N/A MAD: 0ns n: 1
  Tail: μ: 19.7ms σ: 4.1ms MAD: 3.4ms n: 9
   [-30.81% – +17.66%] ▁█▇█▁▅▅▅▁▇

Auditing measurement "release-binary-size" (os=ubuntu-22.04/rust=stable):
  ✅ 'release-binary-size'
  Aggregation: min
  z-score (stddev): ↑ 2.30
  Head: μ: 8.2MB σ: N/A MAD: 0B n: 1
  Tail: μ: 8.1MB σ: 69.1kB MAD: 37.5kB n: 34
   [-0.70% – +2.24%] ▇▇▇▇▇▇▇▂▃▄▄▄▄▄▃▃▃▃▃▃▃▃▃▃▃▁▁▁▁▁▁▁▁▁█

Auditing measurement "report" (os=ubuntu-22.04/rust=beta):
  ✅ 'report'
  Aggregation: min
  z-score (stddev): ↓ 1.14
  Head: μ: 19.2ms σ: N/A MAD: 0ns n: 1
  Tail: μ: 22.9ms σ: 3.2ms MAD: 495.7µs n: 34
   [-26.10% – +21.29%] ▇▁▁▁█▇▅▁▅▅▅▅▅▅▅▅▆▁▁▅▅▅▅▁▁▅▅▅▅▅▁▁▆▅▂

Auditing measurement "report" (os=ubuntu-22.04/rust=stable):
  ✅ 'report'
  Aggregation: min
  z-score (stddev): ↑ 0.81
  Head: μ: 26.4ms σ: N/A MAD: 0ns n: 1
  Tail: μ: 23.9ms σ: 3.1ms MAD: 591.6µs n: 32
   [-29.22% – +22.79%] ▂█▇█▁▆▅▆▁▅▅▂▅▅▂▆▅▁▅▅▅▅▅▅▅▅▅▅▅▅▅▅▆

Auditing measurement "report-benchmark" (os=ubuntu-22.04/rust=beta):
  ✅ 'report-benchmark'
  Aggregation: min
  z-score (stddev): ↓ 1.04
  Head: μ: 19ms σ: N/A MAD: 0ns n: 1
  Tail: μ: 23.3ms σ: 4.1ms MAD: 3ms n: 11
   [-25.39% – +20.27%] ▇▁▁▁█▇▅▁▅▅▅▁

Auditing measurement "report-benchmark" (os=ubuntu-22.04/rust=stable):
  ⏭️ 'report-benchmark'
  Only 9 historical measurements found. Less than requested min_measurements of 10. Skipping test.
  Aggregation: min
  z-score (stddev): ↑ 0.53
  Head: μ: 26.7ms σ: N/A MAD: 0ns n: 1
  Tail: μ: 24.1ms σ: 4.8ms MAD: 4.2ms n: 9
   [-29.71% – +17.69%] ▁█▇█▁▅▅▅▁▆

Auditing measurement "report-size" (os=ubuntu-22.04/rust=beta):
  ✅ 'report-size'
  Aggregation: min
  z-score (stddev): ↑ 0.89
  Head: μ: 20.3kB σ: N/A MAD: 0B n: 1
  Tail: μ: 17.7kB σ: 3kB MAD: 0B n: 34
   [-38.14% – +0.00%] ██████████████████▄▄▄▄▄▄▄▄▄▄▄▄▁▁▁▁█

Auditing measurement "report-size" (os=ubuntu-22.04/rust=stable):
  ✅ 'report-size'
  Aggregation: min
  z-score (stddev): ↑ 0.94
  Head: μ: 20.3kB σ: N/A MAD: 0B n: 1
  Tail: μ: 17.6kB σ: 3kB MAD: 2.4kB n: 32
   [-29.77% – +13.53%] ████████████████▄▄▄▄▄▄▄▄▄▄▄▄▁▁▁▁█

Auditing measurement "report-size-benchmark" (os=ubuntu-22.04/rust=beta):
  ✅ 'report-size-benchmark'
  Aggregation: median
  z-score (stddev): →
  Head: μ: 20.3kB σ: N/A MAD: 0B n: 1
  Tail: μ: 20.3kB σ: 0B MAD: 0B n: 18
   [+0.00% – +0.00%] ▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅

Auditing measurement "report-size-benchmark" (os=ubuntu-22.04/rust=stable):
  ✅ 'report-size-benchmark'
  Aggregation: median
  z-score (stddev): →
  Head: μ: 20.3kB σ: N/A MAD: 0B n: 1
  Tail: μ: 20.3kB σ: 0B MAD: 0B n: 16
   [+0.00% – +0.00%] ▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅

Overall: PASSED (11/11 groups passed)

Measurement Storage Size

Live Measurement Size Report
============================

⚠️  Shallow clone detected - measurement counts may be incomplete (see FAQ)

Number of commits with measurements: 322
Total measurement data size (on-disk (compressed)): 447.3kB

Repository Statistics (for context):
-------------------------------------
  Loose objects: 0 (0B)
  Packed objects: 9803 (2.6MB)
  Total repository size: 2.6MB

Created by git-perf

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 24, 2026

🧬 PR Mutation Testing Results

Status: ✅ All mutants caught - excellent test coverage!

Mutation Testing Metrics (Changed Code Only)

Metric Count
Total Mutants 38
✅ Caught 38
❌ Missed 0
⏱️ Timeout 0
🚫 Unviable 0

Mutation Score: 100.0% (38/38 viable mutants caught)

✅ Great Job!

All mutants in your changes were caught by the test suite. Your code is well-tested!


Incremental mutation testing only tests code modified in this PR.
Full mutation testing runs weekly to catch issues in unchanged code.

📦 Artifacts

Download full mutation report

claude added 2 commits May 24, 2026 18:33
Escape [environment] in doc comments using backticks to prevent rustdoc
from interpreting them as broken intra-doc links.

Refactor resolve_key_values() to read and parse .gitperfconfig exactly
once instead of twice (previously called read_raw_gitperfconfig() for
both [defaults] and [environment], each spawning a git subprocess).
Extract parse_environment_from_doc() and parse_defaults_from_doc()
helpers that operate on an already-parsed DocumentMut.

https://claude.ai/code/session_01A9QXKgBLkLZaeRxJt57bus
Update generated manpages and markdown docs to reflect the corrected
`[environment]` formatting in the --skip-env argument description.

https://claude.ai/code/session_01A9QXKgBLkLZaeRxJt57bus
Comment thread git_perf/src/config.rs Outdated
claude added 2 commits May 24, 2026 18:59
Single is just a special case of a one-valued list. Remove the enum and
use Vec<String> directly as the value type for read_environment_config()
and parse_environment_from_doc(). The apply_env_source helper now takes
Vec<String> and handles both cases uniformly.

https://claude.ai/code/session_01A9QXKgBLkLZaeRxJt57bus
Remove the incomplete TOKEN/SECRET blocklist from apply_env_source.
The filter gave false confidence while missing most sensitive variable
patterns (PASSWORD, KEY, CREDENTIAL, etc.). A security note is added
to all relevant documentation instead, making the responsibility clear
to users.

Also add missing --skip-env bash integration tests for measure and
import commands, and document the [environment] and [defaults] config
sections in CLAUDE.md, README.md, docs/example_config.toml, and
docs/importing-measurements.md.

https://claude.ai/code/session_01EU3bMeB229LZVQpFFUBXNw
@kaihowl kaihowl marked this pull request as ready for review May 24, 2026 19:32
@kaihowl kaihowl enabled auto-merge (squash) May 24, 2026 20:08
@kaihowl kaihowl merged commit 512a762 into master May 24, 2026
24 checks passed
@kaihowl kaihowl deleted the claude/open-issues-review-IcIqQ branch May 24, 2026 20:10
@release-plz-token release-plz-token Bot mentioned this pull request May 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(config): add default/environment sources key-value pairs

2 participants