v0.22.2

Added

• Run-free ("model v2") predictions. Predictions can now be uploaded and read directly against a Model, with no ModelRun or Dataset involved — the concept is (model, dataset_item) -> prediction. New methods on Model:

  • Model.upload_predictions(predictions, update=False, batch_size=5000, ...) — upserts predictions onto the model (box / polygon / cuboid only), targeting model/{id}/predictions, and reusing the existing PredictionUploader batching machinery. Synchronous only for now: asynchronous=True raises NotImplementedError.
  • Model.predictions_loc(dataset_item_id), Model.predictions_refloc(reference_id), Model.predictions_iloc(i) — model-scoped reads returning the same shape as their Dataset equivalents.
  • Model.copy_predictions_from_run(model_run_id) — synchronously backfills the run-free store from an existing model run, returning a dict {model_id, model_run_ids, predictions_copied, predictions_skipped_unsupported}.

• Model-anchored benchmark evaluations. NucleusClient.create_benchmark_evaluation_v2() accepts a model_id (a prj_* id or a Model) as an alternative to model_run_id; the model-anchored flow evaluates the model's run-free predictions and ignores model runs. Provide exactly one of the two. EvaluationV2 now exposes an optional model_id field alongside model_run_id.

• list_evaluations_v2 accepts a model. NucleusClient.list_evaluations_v2() takes exactly one of model_run_id (run_*) or model_id (prj_* or a Model). The model-anchored path hits GET model/{id}/evaluationsV2 and returns that model's run-free evaluations.

• Model.model_runs(). Lists the ids of every model run for a model — the model-scoped counterpart to Dataset.model_runs(), which only lists a single dataset's runs. Pass include_versions=True to union runs across the model's version lineage (its version root and all descendants). Results are scoped server-side to runs on datasets you can read.

  run_ids = model.model_runs()

Server dependency: requires the GET /nucleus/model/:modelId/modelRun route in scaleapi. Unit tests pass regardless; live calls 404 until that deploys.

Changed

• The existing run-based prediction paths (Dataset.upload_predictions, ModelRun.add_predictions, create_benchmark_evaluation_v2(model_run_id=...)) are unchanged and continue to work; the model-centric methods are purely additive.

Deprecated

• Model-run-anchored Evaluation V2 is deprecated in favor of the run-free (model_id) path. Benchmark.create_evaluation_v2() gains a model_id argument (run-free anchor) to match create_benchmark_evaluation_v2(). Passing model_run_id to create_benchmark_evaluation_v2(), Benchmark.create_evaluation_v2(), or list_evaluations_v2() now emits a DeprecationWarning; all keep working. EvaluationV2.model_run_id is documented as deprecated (it is None on run-free evaluations). On the leaderboard, LeaderboardRankingEntry / LeaderboardF1CurveEntry model_run_id and model_run_name are deprecated and now Optional (they are None for run-free evaluations — previously model_run_id was a required field and would fail to parse), and collapse="allRuns" on leaderboard_ranking() is discouraged. Prefer anchoring on and identifying evaluations by model_id.
• allowed_label_matches on Evaluation V2. create_evaluation_v2_preset(), update_evaluation_v2_preset(), create_benchmark_evaluation_v2(), and Benchmark.create_evaluation_v2() still accept allowed_label_matches / allowed_label_matches_id for backwards compatibility, but they now emit a DeprecationWarning. Use rollup_groups instead. AllowedLabelMatch and the corresponding fields on EvaluationV2 / EvaluationV2Preset are likewise marked deprecated.

Removed

• allowed_label_matches removed from the EvaluationV2 surface (breaking). The run-free (model-source) eval path — the one this SDK now steers toward — rejects allowedLabelMatches server-side (400, "use rollupGroups"); it only survives as a legacy fallback on the deprecated model-run path, where rollupGroups wins anyway. Removed the AllowedLabelMatch class (and its top-level export), the allowed_label_matches / allowed_label_matches_id arguments from create_benchmark_evaluation_v2(), Benchmark.create_evaluation_v2(), create_evaluation_v2_preset(), and update_evaluation_v2_preset(), and the allowed_label_matches* fields from EvaluationV2 and EvaluationV2Preset. Use rollup_groups (:class:RollupGroup) exclusively.
• dataset_id dropped from the EvaluationV2 surface (breaking). An evaluation is no longer anchored on a single dataset — a model run now carries a set of datasets and a benchmark's items may span several — so the backend no longer returns a denormalized dataset on evaluations or leaderboards. Removed EvaluationV2.dataset_id, and dataset_id / dataset_name from LeaderboardRankingEntry and LeaderboardF1CurveEntry, matching the current backend responses. Without this, EvaluationV2.from_json raised KeyError: 'dataset_id' on every model-anchored (run-free) benchmark evaluation, since those payloads never carry a dataset_id.

Fixed

• Model.predictions_loc / predictions_refloc / predictions_iloc now actually parse their responses. The run-free read endpoints return a flat {"predictions": [...]} list (each element carrying its own "type"), but format_prediction_response only understood the legacy type-keyed {"annotations": {"box": [...]}} shape, so these methods returned the raw payload unparsed instead of the documented {"box": [...], "polygon": [...], "cuboid": [...]} dict.