feat(datapoints): support multiple images per media_context#580
feat(datapoints): support multiple images per media_context#580
Conversation
Allow media_contexts to accept list[list[str]] in addition to list[str], so each datapoint can display multiple reference images in the rapid. The type is widened throughout the full call stack: - Datapoint.media_context: str | list[str] | None - DatapointsValidator accepts list[str] | list[list[str]] | None - DatapointUploader, ValidationRapidUploader, AudienceExampleHandler each grow _upload_and_map_media_context() that uploads all URLs and wraps them in IAssetInputMultiAssetInput when more than one is given - AssetUploadOrchestrator extracts all URLs from list media_context - RapidataDataset._create_dataset_groups handles list media_context - Rapid model, RapidsManager rapid builders, all order/job/validation manager methods, and RapidataAudience examples updated accordingly - Fully backward-compatible: existing list[str] callers are unaffected Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-Authored-By: elias <elias@rapidata.ai>
Code Review — PR #580: Support multiple images per
|
| Area | Status |
|---|---|
| Correctness | ✅ Logic is sound and backward-compatible |
| Type annotations | ✅ Consistent with project from __future__ import annotations convention |
| Code duplication | _upload_and_map_media_context duplicated 3–4× |
| Dead code | continue in validator |
| Style consistency | Optional[...] outlier |
| Tests | |
| Docs | ✅ Updated and clear |
The duplication item is the most important to address before merge — everything else is low risk.
Per follow-up feedback: simplify to a single canonical shape and warn on the legacy form instead of supporting both. - Public API ``media_contexts`` is now ``list[list[str]] | None``. A flat ``list[str]`` is still accepted for backward compatibility but emits a deprecation warning and is wrapped automatically (each string -> [str]). - Singular ``media_context`` (``Datapoint``, ``Rapid``, audience example helpers, ``RapidsManager`` rapid builders) is now ``list[str] | None``; passing a ``str`` emits the same deprecation warning and is wrapped. - All uploaders / handlers (``DatapointUploader``, ``ValidationRapidUploader``, ``AudienceExampleHandler``, dataset group creation) drop their isinstance branching — they now always receive a ``list[str]`` and unwrap single-element lists into ``ExistingAssetInput`` for backwards-compatible rendering, only using ``MultiAssetInput`` when there is more than one asset. - ``DatapointsValidator`` exposes a shared ``_normalize_media_contexts`` helper used by ``map_datapoints`` and the two ranking-style entry points (``RapidataOrderManager.create_ranking_order`` and the job-manager equivalent) and by ``ValidationSetManager``. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-Authored-By: elias <elias@rapidata.ai>
Code Review — PR #580: Support multiple images per media_contextOverviewClean, well-scoped feature. The approach — widening Issues1.
|
|
|
||
| > A flat `list[str]` is still accepted for backward compatibility — it triggers a deprecation warning and each string is automatically wrapped in a single-element list. | ||
|
|
| Media URLs shown as reference context alongside each datapoint. Useful when you need to show one or more reference images / videos alongside the item being evaluated. | ||
|
|
||
| **Constraints:** If provided, must have the same length as `datapoints`. | ||
| **Constraints:** If provided, must have the same length as `datapoints`. Each entry is itself a list of media URLs / paths. Use a single-element inner list for one media asset per datapoint, or multiple entries to display several images / videos. |
There was a problem hiding this comment.
do not mention video as an input here. even tho technically supported. it's not the idea to be used for videos. so only mention images
| data_type (str, optional): The type of the datapoint. Defaults to "media" (any form of image, video or audio). | ||
| context (str, optional): The context is text that will be shown in addition to the instruction. Defaults to None. | ||
| media_context (str, optional): The media context is a link to an image / video that will be shown in addition to the instruction (can be combined with context). Defaults to None. | ||
| media_context (list[str], optional): A list of links to images / videos that will be shown in addition to the instruction (can be combined with context). Pass a single-element list for one media asset, or multiple to display multiple images / videos. Defaults to None. |
| data_type (Literal["media", "text"], optional): The data type of the datapoints. Defaults to "media". | ||
| context (str, optional): Additional text context to display with the example. Defaults to None. | ||
| media_context (str, optional): Additional media (URL or path) to display with the example. Defaults to None. | ||
| media_context (list[str], optional): Additional media (URLs or paths) to display with the example. Pass a single-element list for one media asset, or multiple to display multiple images / videos. Defaults to None. |
There was a problem hiding this comment.
will stop commenting now, but please remove the video mention everyhwere
| def _upload_and_map_media_context( | ||
| self, media_context: list[str] | ||
| ) -> IAssetInput: | ||
| """Upload media context asset(s) and map to IAssetInput. | ||
|
|
||
| ``media_context`` is always a list. A single-element list is sent as a | ||
| plain ``ExistingAssetInput`` (preserving the legacy single-image | ||
| rendering); two or more entries are bundled into a ``MultiAssetInput``. | ||
| """ | ||
| uploaded_names = [ | ||
| self.asset_uploader.upload_asset(mc) for mc in media_context | ||
| ] | ||
| if len(uploaded_names) == 1: | ||
| return self.asset_mapper.create_existing_asset_input(uploaded_names[0]) | ||
| return self.asset_mapper.create_existing_asset_input(uploaded_names) | ||
|
|
There was a problem hiding this comment.
why is this code duplicated?
| def _normalize_media_contexts( | ||
| media_contexts: list[list[str]] | list[str] | None, | ||
| ) -> list[list[str]] | None: | ||
| """Normalize media_contexts to ``list[list[str]] | None``. | ||
|
|
||
| Accepts the legacy ``list[str]`` shape (one media context per datapoint) | ||
| by wrapping each string in a single-element list, after emitting a | ||
| deprecation warning. This way the rest of the pipeline only ever has | ||
| to deal with ``list[list[str]]``. | ||
| """ | ||
| if media_contexts is None: | ||
| return None | ||
|
|
||
| if not isinstance(media_contexts, list): | ||
| raise ValueError( | ||
| "media_contexts must be a list of lists of strings or None, " | ||
| f"got {type(media_contexts).__name__}." | ||
| ) | ||
|
|
||
| has_str = any(isinstance(mc, str) for mc in media_contexts) | ||
| has_list = any(isinstance(mc, list) for mc in media_contexts) | ||
| if has_str and has_list: | ||
| raise ValueError( | ||
| "media_contexts must be a list of lists of strings, not a mix of strings and lists." | ||
| ) | ||
|
|
||
| if has_str: | ||
| logger.warning( | ||
| "Passing a flat list of strings for media_contexts is deprecated; " | ||
| "pass a list of lists of strings instead. Each string has been " | ||
| "wrapped in a single-element list." | ||
| ) | ||
| normalized: list[list[str]] = [] | ||
| for mc in media_contexts: | ||
| if not isinstance(mc, str): | ||
| raise ValueError( | ||
| "media_contexts must be a list of lists of strings, " | ||
| f"got element of type {type(mc).__name__}." | ||
| ) | ||
| if mc == "": | ||
| raise ValueError( | ||
| "media_contexts entries cannot be empty strings." | ||
| ) | ||
| normalized.append([mc]) | ||
| return normalized | ||
|
|
||
| # list[list[str]] case | ||
| for mc in media_contexts: | ||
| if not isinstance(mc, list): | ||
| raise ValueError( | ||
| "media_contexts must be a list of lists of strings, " | ||
| f"got element of type {type(mc).__name__}." | ||
| ) | ||
| if len(mc) == 0: | ||
| raise ValueError( | ||
| "Each inner media_contexts list must contain at least one string. " | ||
| "Use None for the whole field if not needed." | ||
| ) | ||
| if any(not isinstance(item, str) or item == "" for item in mc): | ||
| raise ValueError( | ||
| "Every entry in a media_contexts inner list must be a non-empty string." | ||
| ) | ||
| return cast("list[list[str]]", media_contexts) |
There was a problem hiding this comment.
this also seems like duplicate code from that already exists in the datapoint? either do it in the validator or in the datapoint itself. not both
- Drop "video" wording from docstrings and docs — media_context is for images only, even though the plumbing technically also handles other asset kinds. - Remove the deprecation-note note from docs/job_definition_parameters.md. - Stop duplicating asset upload/map. ``_upload_and_map_media_context`` helpers were near-clones of ``_upload_and_map_asset``; both ``DatapointUploader`` and ``ValidationRapidUploader`` now reuse their existing ``_upload_and_map_asset`` for media_context. The ``AudienceExampleHandler`` gains one ``_upload_and_map_asset`` shared by all three callers (main asset, classification, compare) and the inline asset-upload blocks were folded into it. - Stop duplicating coercion logic. The plural ``_normalize_media_contexts`` helper is gone; ``map_datapoints`` delegates per-item coercion to ``coerce_media_context``, which now lives once in ``_datapoint.py`` and is imported by ``Datapoint``/``Rapid`` field validators and by ``RapidataAudience``'s public methods. - ``_create_dataset_groups`` now calls the same shared ``_upload_and_map_asset`` helper instead of branching on str vs list. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-Authored-By: elias <elias@rapidata.ai>
Code Review — PR #580: Support multiple images per
|
The same `_upload_and_map_asset` body was copy-pasted into DatapointUploader, AudienceExampleHandler, and ValidationRapidUploader. Move it onto AssetUploader as `upload_and_map_asset` (returns IAssetInput) plus a `_with_mapping` variant that also returns the original-path -> uploaded-name dict needed for compare-rapid truth translation. All four call sites now share one implementation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Co-Authored-By: lino <lino@rapidata.ai>
Code Review — PR #580: Support multiple images per
|
| Category | Severity |
|---|---|
Single-image MultiAssetInput behavior change |
Medium — needs confirmation with backend team |
| Deprecation warning flood | Medium — will impact existing users |
| "Rejects mixed lists" claim not implemented | Low — misleading PR description |
| Comments violating CLAUDE.md conventions | Low — style issue |
| Missing migration docs | Low — UX improvement |
The core design is solid. The main asks before merging: (1) confirm MultiAssetInput vs ExistingAssetInput is safe for single-image case, and (2) fix the warning-per-item behavior for the backward-compat path.
The two-method API existed only because compare-rapid truth translation needed a path -> uploaded-name dict. But `upload_asset` is always cached (see its docstring: "Caching is always enabled... cannot be disabled"), so the validation uploader can just call it again on `winnerId` / `correctCombinations` entries — those calls are guaranteed cache hits, not re-uploads. That collapses AssetUploader back to a single `upload_and_map_asset` returning `IAssetInput` directly, kills the `upload_and_map_asset_with_mapping(asset)[0]` smell, and isolates the compare-truth concern inside `_translate_compare_truth` where it belongs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Co-Authored-By: lino <lino@rapidata.ai>
Code Review: feat(datapoints) — support multiple images per media_contextOverviewThis PR widens The overall approach is sound — centralising coercion in one helper, moving Issues1. Breaking API change, not fully backward-compatible (type-level)The PR summary says "fully backward-compatible: all existing The runtime is fine —
The inconsistency is visible right now: 2. Mixed-list claim vs. actual behaviourThe PR summary says "rejects mixed lists", but no explicit validation rejects 3. Stale/misleading comment in
|
| Severity | Item |
|---|---|
| High | Public API signatures claim list[list[str]] only — breaks static type checking for existing callers |
| Medium | Mixed-list behaviour differs from documented "rejects mixed lists" claim |
| Low | Stale comment about "bare string slips through" in _rapidata_dataset.py |
| Low | Documentation missing deprecation/compat note for flat list[str] |
| Low | _translate_compare_truth reliance on cache hit for correctness is undocumented |
The core logic is well-structured. The main thing to resolve before merge is whether this is officially a breaking change (and documented as such) or whether the public type signatures should preserve the list[str] union for true backward compatibility.
Code Review: feat(datapoints): support multiple images per media_contextOverviewThis PR widens Issues1. Incomplete backward compatibility — breaking change at public API levelThe PR claims full backward compat, but the public manager APIs now type # rapidata_order_manager.py, rapidata_job_manager.py, etc.
media_contexts: list[list[str]] | None = None # was: list[str] | NoneThe internal Suggestion: Either keep 2. "Slips through" comment undermines the invariant guaranteeIn # The shared upload+map helper bundles it into a
# MultiAssetInput (or ExistingAssetInput if a bare string slips through).If 3. Missing explicit mixed-list rejectionThe PR summary says it "rejects mixed lists" but no such check exists in the diff. 4. Audience API coercion gap
5. Docs show only the new form — no migration noteThe old canonical example: media_contexts=["original1.jpg", "original2.jpg"]is replaced without any migration note. Users upgrading will hit a type error and have no guidance. A short callout like "Previously accepted a flat list of strings — wrap each entry in its own list: Minor
VerdictThe core design — |
Summary
media_contextsfromlist[str]tolist[str] | list[list[str]]across all order/job/validation/audience APIsMultiAssetInput, so annotators see multiple reference images for that datapointlist[str]callers continue to work without changesWhat changed
_datapoint.pymedia_context: str | list[str] | None_datapoints_validator.py_datapoint_uploader.py_upload_and_map_media_context()handles str or list_asset_upload_orchestrator.py_rapidata_dataset.py_create_dataset_groupshandles list media_contextrapids.pyRapid.media_context: str | list[str] | None_validation_rapid_uploader.py_upload_and_map_media_context()rapids_manager.pymedia_contextparams updatedvalidation_set_manager.pycreate_*_setsignatures updatedrapidata_order_manager.pycreate_*_ordersignatures updatedrapidata_job_manager.pycreate_*_job_definitionsignatures updatedrapidata_job_definition.pyupdate_datasetsignature updatedaudience_example_handler.py_upload_and_map_media_context(); both example methods updatedrapidata_audience.pyadd_classification_example/add_compare_examplesignatures updateddocs/job_definition_parameters.mdmedia_contextsreference with multi-image exampleTest plan
0 errors, 0 warnings, 0 informations✅ (verified in this session)media_contexts=[["img1.jpg", "img2.jpg"], ["img3.jpg"]]to a classification order and confirm both images appear as context in the rapidmedia_contexts=["img1.jpg", "img2.jpg"]and confirm no regression🔗 Session: https://session-c1c6a759.poseidon.rapidata.internal/