MVR-Data

███╗   ███╗██╗   ██╗██████╗       ██████╗  █████╗ ████████╗ █████╗
████╗ ████║██║   ██║██╔══██╗      ██╔══██╗██╔══██╗╚══██╔══╝██╔══██╗
██╔████╔██║██║   ██║██████╔╝█████╗██║  ██║███████║   ██║   ███████║
██║╚██╔╝██║╚██╗ ██╔╝██╔══██╗╚════╝██║  ██║██╔══██║   ██║   ██╔══██║
██║ ╚═╝ ██║ ╚████╔╝ ██║  ██║      ██████╔╝██║  ██║   ██║   ██║  ██║
╚═╝     ╚═╝  ╚═══╝  ╚═╝  ╚═╝      ╚═════╝ ╚═╝  ╚═╝   ╚═╝   ╚═╝  ╚═╝

Important

Rewrite in progress. The previous Python reference implementation and its src/ directory have been removed. This revision intentionally provides no installable library, CLI, C++ API, or Python API. The format specification, schemas, examples, and behavioral tests remain as inputs to the new design.

MVR-Data defines how multi-vector retrieval data is organized at runtime and packaged for interchange using Apache Arrow. It covers Raw multimodal content, Embedded multi-vector representations, and long-form relevance judgments.

Target architecture

The replacement implementation will have one shared native core:

• a header-only C++ API under include/mvr_data/;
• a compiled pybind11 Python extension that exposes the same behavior to Python without invoking a subprocess;
• Apache Arrow C++ for schemas, arrays, record batches, IPC, and memory sharing;
• a CMake INTERFACE target for C++ consumers;
• Doxygen comments in the public C++ headers and generated C++ API reference;
• Python modules that remain thin adapters rather than a second implementation.

MVR-Data will remain a reference implementation rather than a distributed data engine. Exact object-ID and ground-truth indexes may be held in memory, and the documentation will state those memory assumptions explicitly.

Although MVR-Data itself will be header-only for C++ consumers, Apache Arrow is a compiled dependency, and the Python extension must still be built as a native module.

Stable format concepts

An object is one retrievable content item, such as an illustrated article, an audio recording, a video, or another multimodal item.

• Raw data stores base and query objects as one or more ordered content components whose payloads live inside the package.
• Embedded data stores base and query object IDs with one or more ordered vectors. Dimension and numeric dtype are fixed package-wide.
• Ground-truth data stores judged query-object pairs with their relevance, split, judgment source, and annotation pool.

If Raw and Embedded packages describe the same collection, the same base or query object must use the same object_id in both packages. Row positions do not need to match; the ID is the link between content and vectors.

Raw base and query tables

Field	Arrow type	Meaning
object_id	string	Unique object identifier within the table.
components	list<struct>	One or more ordered content components of an object.
components[].component_id	string	Component identifier, unique within the object.
components[].modality	string	Extensible modality such as text, image, audio, or video.
components[].media_type	string	MIME type such as text/plain or image/png.
components[].payload_uri	string	Package-local content-addressed payload URI.

Embedded base and query tables

Field	Arrow type	Meaning
object_id	string	Unique object identifier within the table.
vectors	large_list<fixed_size_list<T, dimension>>	One or more ordered vectors using the package-wide dtype and dimension.

Ground-truth table

Field	Arrow type	Meaning
query_id	string	ID of an object in the query table.
object_id	string	ID of an object in the base table.
relevance	int16	Non-negative relevance level.
split_type	string	Data split such as test.
judgment_source	string	Label source such as human or adjudicated.
pool_id	string	Candidate annotation-pool identifier.

Every package also contains manifest.yaml, which identifies the format and data collection, indexes the three Arrow tables, and records package-wide vector settings for Embedded data.

Repository state

Path	Purpose during the rewrite
docs/	Normative and explanatory format documentation.
schemas/	Manifest JSON Schema.
examples/	Example Raw and Embedded Manifests.
tests/	Behavioral requirements retained from the previous implementation.
include/	Planned location of the header-only C++ API; not created yet.
python/	Planned location of pybind11 bindings; not created yet.

The retained tests are not expected to run until the replacement Python module is introduced.

Rewrite sequence

1. Define the public C++ types, status/error model, and CMake INTERFACE target.
2. Implement schemas, Manifest handling, and Arrow IPC reading in headers.
3. Add Raw and Embedded writers plus exact in-memory validation.
4. Bind the C++ API through pybind11 while preserving a concise Python surface.
5. Re-enable the behavioral tests and publish installation and API documentation.

Until those steps land, the repository should be treated as a format and design workspace rather than a usable software release.