09. Model Interface

09. Model Interface — Bring Your Own Model

DIRD+ ships reference models (DIRDv1r1, DIRDv2r0) but is model-agnostic: any organization can train and plug in its own ONNX model — without modifying the application.

Authoritative spec: docs/model-interface.md. A complete, validated example card is docs/example-card.json. This page is a summary.

What a custom model needs

1. An ONNX model runnable by ONNX Runtime.
   • Input: tensor [1, 3, 640, 640], layout NCHW, dtype float32. Preprocessing: resize to 640×640, letterbox (color 114,114,114), normalize by ÷255, color order RGB.
   • Output: detection boxes. The reference format is yolo_end2end_v2 ([x, y, w, h, confidence, class]), with a configurable max_detections.
2. A model card — a JSON file with the same base filename as the model (my-model.onnx → my-model.json).

Model card (required fields)

{
  "schema_version": "2.0",
  "name": "MyModel",
  "version": "1.0.0",
  "license": "AGPL-3.0",
  "input":  { "tensor_name": "images", "shape": [1,3,640,640], "layout": "NCHW",
              "dtype": "float32", "preprocessing": { "resize_to": [640,640],
              "letterbox": true, "normalize": "divide_by_255", "color_order": "RGB" } },
  "output": { "tensor_name": "output", "format": "yolo_end2end_v2", "max_detections": 300 },
  "classes": [
    { "id": 0, "name": "optic_disc", "severity": "landmark" },
    { "id": 1, "name": "fovea",      "severity": "landmark" }
    /* … your classes … */
  ]
}

Optional but recommended: description, authors, doi, trained_on, validated_on, per_class_thresholds, default_confidence_threshold, minimum_dird_version, tags.

Classes

The reference models use class IDs 0–10:

ID	Class	ID	Class
0	optic_disc	6	edema
1	fovea	7	microaneurysm
2	hard_exudate	8	neovascularization
3	hemorrhage	9	venous_beading
4	cotton_wool_spot	10	IRMA
5	microhemorrhages

Custom models may declare their own classes in the model card. DIRD+ uses that mapping to label detections in the UI and to feed the clinical-guideline engine — the guideline's class_mapping (see 10. Clinical Guidelines) then maps model classes to clinical categories.

Loading & validation

From Settings → AI Models → Add Model (src/components/settings/CustomModelsSection.tsx), DIRD+:

1. Validates the model-card JSON schema (src/lib/ai/model-card-validator.ts).
2. Checks input/output tensor shapes (src/lib/ai/model-registry.ts, sanityCheckModel).
3. Runs an inference sanity check on a test tensor.
4. Registers the model under <user_data>/models/ (src-tauri/src/models.rs) and lets you set it active.

A standalone CLI validator is also provided:

python3 scripts/validate_model_card.py my-card.json --onnx my-model.onnx