TrackIQ Monorepo

(Formerly Autoperfpy)
TrackIQ is a multi-tool performance validation repository.
It contains one shared library (trackiq_core) and three tool applications (autoperfpy, minicluster, trackiq_compare) built on top of that library.

Naming and Scope

Use these names consistently:

• trackiq:
  • The repository/monorepo name.
  • Contains all packages and tools.
• trackiq_core:
  • The shared Python library.
  • Defines canonical result schema (TrackiqResult), serialization/validation, hardware and power integrations, UI base components, baseline/report utilities.
• autoperfpy:
  • Edge and inference benchmarking tool.
  • Produces TrackiqResult JSON outputs.
• minicluster:
  • Local distributed training validation tool.
  • Produces TrackiqResult JSON outputs and health checkpoint data.
• trackiq-compare (trackiq_compare package):
  • Result-to-result comparison tool.
  • Compares any two TrackiqResult files regardless of source tool.

Repository Layout

trackiq/
├── trackiq_core/      # shared library
├── autoperfpy/        # inference / edge benchmarking tool
├── minicluster/       # distributed training validation tool
├── trackiq_compare/   # comparison tool
├── dashboard.py       # unified dashboard launcher
└── tests/             # repo-level tests

Installation

# from repo root
py -3.12 -m pip install -e .

Install optional stacks as needed:

# dashboards (Streamlit)
py -3.12 -m pip install -e ".[ui]"

# distributed training / torch features
C

# GPU/NVML integrations
py -3.12 -m pip install -e ".[gpu]"

# full contributor environment
py -3.12 -m pip install -e ".[dev,test,security]"

# contributor environment with all optional runtime stacks
py -3.12 -m pip install -e ".[dev,test,security,ml,ui,gpu,reports]"

If your shell entry points are stale on Windows, use module execution directly:

py -3.12 -m autoperfpy.cli --help
py -3.12 -m minicluster.cli --help
py -3.12 -m trackiq_compare.cli --help

Core Workflow

1. Run a producer tool (autoperfpy or minicluster) to generate canonical TrackiqResult JSON.
2. Compare results with trackiq-compare.
3. View results in dashboards (tool-specific Streamlit apps or unified launcher).

Tool Workflows

AutoPerfPy

Main use: edge/inference benchmark runs, profile-driven validation, and report generation.

autoperfpy devices --list
autoperfpy run --auto --duration 30 --export output/autoperf_result.json
autoperfpy run --profile automotive_safety --batch-size 4 --precision bf16 --export output/autoperf_profile_result.json
autoperfpy run --manual --device cpu_0 --precision int4 --duration 10 --export output/autoperf_manual_result.json
autoperfpy analyze latency --csv output/results.csv
autoperfpy report html --json output/autoperf_result.json --output output/autoperf_report.html
autoperfpy report pdf --json output/autoperf_result.json --output output/autoperf_report.pdf --pdf-backend auto

Precision modes include fp32, fp16, bf16, int8, int4, and mixed with capability-aware fallback per device.

Dashboard options:

# tool-owned app
streamlit run autoperfpy/ui/streamlit_app.py

# unified launcher (note the `--` before dashboard args)
python -m streamlit run dashboard.py -- --tool autoperfpy --result output/autoperf_result.json

# unified launcher browser mode (pick file in UI)
python -m streamlit run dashboard.py -- --tool autoperfpy

# convenience wrapper (no Streamlit `--` separator needed)
python launch_dashboard.py --tool autoperfpy --result output/autoperf_result.json

# all-tools unified mode (default)
python launch_dashboard.py

Unified mode now includes:

• AutoPerfPy run configuration (manual/auto), result browser, and graph-rich dashboard
• MiniCluster run configuration (workers/steps/batch/lr/seed/tdp) and quick smoke run
• Compare configuration (browse/manual inputs, labels, regression threshold) with comparison graphs

MiniCluster

Main use: distributed training correctness/performance/fault validation.

minicluster run --workers 2 --steps 50 --output minicluster_results/run_metrics.json
minicluster run --workers 2 --steps 50 --health-checkpoint-path ./minicluster_results/health.json
minicluster monitor status --checkpoint ./minicluster_results/health.json
minicluster report pdf --result minicluster_results/run_metrics.json --output minicluster_results/report.pdf --pdf-backend auto

Dashboard options:

# tool-owned app
streamlit run minicluster/ui/streamlit_app.py

# unified launcher (note the `--` before dashboard args)
python -m streamlit run dashboard.py -- --tool minicluster --result minicluster_results/run_metrics.json

# convenience wrapper
python launch_dashboard.py --tool minicluster --result minicluster_results/run_metrics.json

TrackIQ Compare

Main use: compare two canonical results and generate terminal/HTML reports.

trackiq-compare run output/autoperf_result.json minicluster_results/run_metrics.json
trackiq-compare run output/autoperf_result.json minicluster_results/run_metrics.json --html output/comparison.html
trackiq-compare run output/autoperf_result.json minicluster_results/run_metrics.json --label-a "AMD MI300X" --label-b "NVIDIA A100"
trackiq-compare report pdf output/autoperf_result.json minicluster_results/run_metrics.json --output output/comparison.pdf --pdf-backend auto

Baseline flows:

trackiq-compare baseline output/autoperf_result.json --name edge_baseline
trackiq-compare vs-baseline output/autoperf_result.json edge_baseline

Dashboard options:

# tool-owned app
streamlit run trackiq_compare/ui/streamlit_app.py

# unified launcher (note the `--` before dashboard args)
python -m streamlit run dashboard.py -- --tool compare --result-a output/autoperf_result.json --result-b minicluster_results/run_metrics.json --label-a "AMD MI300X" --label-b "NVIDIA A100"

# convenience wrapper
python launch_dashboard.py --tool compare --result-a output/autoperf_result.json --result-b minicluster_results/run_metrics.json --label-a "AMD MI300X" --label-b "NVIDIA A100"

Canonical Result Contract

All tool outputs should conform to:

from trackiq_core.schema import TrackiqResult

This is the single schema contract across the ecosystem. trackiq_compare and dashboards assume TrackiqResult-compatible JSON.

Hardware and Power Notes

• trackiq_core includes multi-platform detection and metrics paths for NVIDIA, AMD, Intel, Apple Silicon, and CPU (best effort depending on available system tools).
• Power profiling readers support ROCm SMI, tegrastats, and simulation fallback.

Testing

py -3.12 -m pytest -q

Where to Go Next

• trackiq_core/ui/USAGE.md: library-first dashboard API usage.
• minicluster/monitor/README.md: live health monitoring pipeline and anomaly model.
• trackiq_compare/README.md: comparison semantics, CLI options, and report behavior.

License

MIT. See LICENSE.