Debugging Tool Xemu Cause Counters

xemu cause counters

← Debugging Tools and Commands

Cause counters are narrowly scoped xemu diagnostics for answering why a hot shared helper, return path, cache lookup, or scan repeats.

Property	Details
Supported OS	Counters run wherever xemu runs; the Python analyzer runs on Windows, Linux, or macOS.
Install	Use the matching xemu-perf-tests revision and Python 3.
Verify	python3 --version
Primary output	See the workflow below.

When to use it

Broad traces often identify a hot framework function but not why it was called.

This is where narrowly scoped diagnostic counters are useful.

Examples from the TCG investigation counted lookup/return causes rather than logging every event.

Analyze XEMU_CAUSE records

The repository helper is:

xemu-perf-tests/utils/analyze_cause_counters.py

Command:

python3 utils/analyze_cause_counters.py \
  cause-records.jsonl \
  --start-utc-us 1788970000000000 \
  --end-utc-us   1788970020000000 \
  --output cause-summary.json

The timestamps above are examples; use the actual measurement anchors from the run.

What the analyzer validates

It rejects:

• missing thread/timestamp identity;
• fewer than two samples per thread;
• counter-schema changes;
• counter resets/decreases;
• clock reversal;
• incomplete lookup partitions.

The output intentionally contains:

"purpose": "causal counts, not benchmark acceptance",
"performance_acceptance": false,
"lookup_partition_check": "PASS"

Read the useful fields

For each thread:

• delta — count growth across the bounded interval;
• rates_per_second — event frequency;
• *_cache_hit_percent — lookup hit ratio;
• helper_collision_share_of_misses_percent — one identified miss class;
• zero_arm_scan_percent — share of dirty scans that armed nothing;
• entries_per_scan — average entries inspected;
• new_arms_per_scan — average newly armed entries.

A high event rate tells you what repeats. It does not tell you how many milliseconds are removable.

PR #54 is the canonical example: more than 99% of sampled NULL-return leaves were attributed to the STI/shadow pair, but that result was intentionally not reported as a speedup.

---

Instrumentation must disappear when disabled

Merged PRs #133, #136, and #137 expose a recurring profiling hazard: a disabled diagnostic can still be hot-path work.

Compile-time disabled debug groups

If DEBUG_VK is zero, text-debug macros should compile to no-ops. Do not leave indentation updates, formatting preparation, counters, or other state whose only purpose is disabled text output. Correctness mechanisms such as VK_CHECK remain separate.

Opt-in event logs

A one-shot event logger should avoid string decoding and mutex acquisition after initialization when no output file is configured or the event has already been written.

if (!logger_enabled)
    return;
bit = 1u << typed_event;
if (atomic_written & bit)
    return;
lock();
if (!(atomic_written & bit)) {
    atomic_written |= bit;
    write_event(event);
}
unlock();

Use typed event IDs, one-time initialization, an atomic once-bit, and take the lock only on the rare actual write.

Schema feature identity

PR #133 advertises report_lifecycle as a feature. A schema number alone is not enough when optional groups share a version. Declare enabled feature groups and define whether absent fields mean zero, disabled, or unavailable.

Every instrumentation facility should have a disabled-path test: no file I/O, no lifecycle accounting, no mutex, no string decode/format, no global state mutation, and no per-call clock read unless the metric is enabled.

Acceptance timing still comes from an uninstrumented Release executable.

Boundaries and common mistakes

Prefer aggregate typed counters over per-event text logging. Initialize and publish their schema explicitly, detect resets, and leave the disabled path without locks, I/O, formatting, or per-call clock reads.

Counter runs establish frequency and cause partitions. They are diagnostic evidence, not acceptance timing; validate the final change in an uninstrumented build.

Evidence to retain

• Tool name and exact version.
• Source commit/tree and executable SHA-256.
• The complete command or saved profile used.
• Input artifact hashes and the explicitly selected time or address range.
• Raw output plus any derived report; do not retain only a screenshot or conclusion.
• Admission failures, warnings, dropped events, incomplete rows, or unsupported operations.

Official references

• xemu performance-test helpers

Related xemu guides

• Debugging Tools and Commands
• Debugging Lag and Hot Paths
• Performance Methodology