Reference implementation of the production evaluation, calibration, and drift-monitoring shell that EVANPARRA.AI LLC builds around object-detection models. It is demonstrated end to end on a public X-ray contraband benchmark (PIDray).
Read this first: this repository demonstrates the engineering pipeline, not detection performance. The detection scores in results/pick1-results.json are produced in demonstration mode and are deliberately not presented as a benchmark result. See Caveats.
The value is the shell around the model, the part that survives contact with production:
| Capability | Evidence in this repo |
|---|---|
| Data card and provenance | Dataset version hash, class inventory, documented split policy, pinned framework versions, run identifier in results/pick1-results.json |
| Evaluation harness | Per-class precision, recall, F1, and average precision across 12 classes; confusion matrices; operating-point curves |
| Confidence calibration | Temperature-scaling step; expected calibration error reduced from 2.6% to 1.8%, Brier from 0.0167 to 0.0156; reliability diagrams |
| Drift monitoring | Synthetic corruption sweep (6 corruption types, 3 severities) that detects degradation; quality holds under noise and contrast, drops sharply under occlusion |
| Latency profiling | Per-batch p50/p95/p99 latency and throughput on a known GPU (9.6 ms, 104 images/sec at batch 1 on an A100) |
| Schema-versioned output | Detection records emitted in a versioned schema, sample detections exported as JSON lines |
Post-hoc temperature scaling (T=0.78) pulls predicted confidence toward observed accuracy, cutting expected calibration error from 2.6% to 1.8%:
notebooks/pick1-pidray-contraband.ipynb # the pipeline, runnable on Colab
results/pick1-results.json # machine-readable run artifact
results/reliability-diagram.png # calibration plot (raw vs temperature-scaled)
These are not fine print. They are the point of being honest about a demonstration.
- Scores are leakage-inflated and not comparable to published baselines. The run used the public test-only HuggingFace mirror of PIDray. The train, validation, and test splits were carved from a single test pool, so the model trains and evaluates on the same distribution. The headline numbers are optimistic by construction. This does not affect the engineering artifacts (calibration behavior, drift detection, latency, provenance), which is what this repo is for.
- Dataset license is academic use only. PIDray is licensed for academic purposes, not commercial use. This repository is a non-commercial engineering demonstration. The dataset is not redistributed here, only code and derived run metadata.
- The detector is a stand-in. A general-purpose off-the-shelf object detector stands in for what would, in a real engagement, be a domain-specialist model. The shell is detector-agnostic.
Open notebooks/pick1-pidray-contraband.ipynb in Google Colab with a GPU runtime. Framework versions are pinned in results/pick1-results.json under provenance.framework_versions (PyTorch 2.4.1, Ultralytics 8.3.40, scikit-learn 1.5.2, netcal 1.3.5, Python 3.12.13).
Code in this repository is released under the MIT License. The PIDray dataset retains its own academic-use-only license and is not included here.