Add a Cube converter

[0sparsh2]

Hi all 👋

I'd like to gauge interest in adding a converter between Ossie and Cube (cube.dev) under converters/.

Cube is a widely-used open-source semantic layer. Its data model is defined in YAML as cubes with sql_table/sql (source), dimensions, measures, and joins — conceptually very close to Ossie's dataset / field / metric / relationship:

Cube concept	Ossie concept
cube.name	dataset.name
cube.sql_table / cube.sql	dataset.source
cube.dimensions[]	dataset.fields[]
dimension.sql	field expression
dimension.type: time	field.dimension.is_time
cube.measures[]	metric (dataset-scoped)
measure.sql + measure.type (sum/count/avg/...)	metric.expression + aggregation
cube.joins[] (relationship: many_to_one etc., sql join condition)	relationship (from/to/from_columns/to_columns)
dimension.meta / measure.meta	candidate home for round-trip metadata via custom_extensions

I'd propose following the existing converter pattern (e.g. converters/omni, converters/honeydew):

1. A stable vendor_name: CUBE for custom_extensions.
2. Export (Ossie → Cube): one Cube YAML file (or one-file-per-cube layout) per Ossie dataset, with metrics attached as measures on their owning dataset and relationships translated to Cube joins.
3. Import (Cube → Ossie): parse a Cube schema directory into a single Ossie YAML, preserving anything Cube-specific (e.g. pre_aggregations, data_source, public/shown flags, format) in CUBE custom extensions so nothing is silently dropped.
4. Tests built off the TPC-DS example (examples/tpcds_semantic_model.yaml), including a round-trip test, following the converters/omni test layout.
5. A README documenting the mapping table and any unsupported constructs (Cube's segments, pre_aggregations, and Jinja-templated SQL are likely out of scope for v1 and would be called out as limitations, similar to how converters/honeydew documents its limitations).

Before I go further: would a Cube converter be welcome in this repo? Anything you'd want me to keep in mind on scope or approach — e.g. preferred package name (apache-ossie-cube?) or how you'd like multi-cube-per-file vs. one-file-per-cube handled on export?

Thanks!

[utkarshparekh]

Hi @0sparsh2, thanks for opening this. I’m also interested in helping with Cube integration and would like to collaborate.
I’d be happy to take a clearly bounded part, such as the static YAML importer and its fixtures and tests, and coordinate the shared mapping first.

I reviewed the Ossie converter guide and current implementations, including Omni, Honeydew, Databricks, Wisdom, and OrionBelt. Omni and Honeydew are the closest structural precedents for a directory-based, bidirectional Cube converter, while the other converters provide useful precedents for relationship orientation, ambiguous metric placement, validation, and loss reporting.

The proposed mapping is a good starting point. Before implementation, I think we should agree on the following v1 contracts:

Input and output scope

• Import static Cube .yml and .yaml files only.
• Reject JavaScript models and unrendered Jinja templates with actionable errors. Materialized static YAML can be accepted as ordinary input.
• Accept nested model directories and one or multiple cubes per YAML file. Cube recommends one cube per file, but its compiler does not require that layout.
• Export one file per cube under model/cubes/.yml, following Cube’s recommended layout.

Measure semantics and ownership

• Cube measures are cube-scoped, while Ossie metrics are model-scoped.
• Define explicit mappings for the basic Cube measure types, including count, sum, avg, min, max, and count_distinct.
• Preserve the original Cube measure type, SQL, filters, and other Cube-specific properties in a versioned CUBE extension so export does not depend on reconstructing them from an Ossie SQL expression.
• On import, keep globally unique measure names and qualify collisions as __, while stashing the original name and owning cube. This follows the Omni collision-handling precedent.
• On export, prefer the owning cube recorded in the CUBE extension. Without that metadata, infer an owner only when qualified references identify exactly one cube.
• If an Ossie metric references zero or multiple datasets, require an explicit owner through the CUBE extension or an agreed CLI option. The converter should not attach it to the first dataset arbitrarily.

Relationships

• Map conjunctions of simple equi-join predicates to Ossie column pairs, including composite keys.
• Map many_to_one with the Cube source as the Ossie from side.
• Reverse one_to_many when constructing Ossie because Ossie from represents the many side.
• Handle one_to_one explicitly. Its column equality can be represented, but Ossie does not natively preserve the one-to-one cardinality.
• Preserve the defining cube, original Cube relationship type, direction, and join SQL in a versioned CUBE extension for reliable restoration.
• Warn and preserve non-equi or otherwise non-representable joins in a model-level extension rather than silently dropping them.

Cube-specific constructs and fidelity

• Classify each Cube construct as natively mapped, preserved in an extension, or explicitly rejected with a warning or error.
• This includes views, view groups, pre-aggregations, segments, hierarchies, access policies, rolling windows, masks, formats, and other Cube-specific properties.
• For views and view groups in particular, we should decide whether v1 preserves them opaquely at the model level or rejects them as unsupported.
• Round-trip guarantees should be semantic. YAML comments, formatting, and exact file layout should not be included in the guarantee.

Package and API

• Proposed distribution: apache-ossie-cube
• Python module: ossie_cube
• CLI: ossie-cube import and ossie-cube export
• Stable extension identifier: CUBE
• No core schema change should be required because vendor_name is free-form.
• PyYAML should be sufficient for the static runtime parser. If Cube’s compiler is added for validation, it can remain development or CI tooling, subject to maintainer and ASF dependency guidance.

Validation

• Cube fixture to Ossie to Cube semantic round-trip
• Ossie TPC-DS to Cube to Ossie normalized round-trip
• Imported Ossie documents validated against core-spec/osi-schema.json
• Unit tests for measure types, calculated measures, duplicate names, ownership restoration, all three relationship types, composite joins, unsupported templates, and non-representable joins
• If maintainers accept the Node tooling, generated Cube models validated with Cube’s own compiler using cubejs validate
• A Cube-specific CI workflow following the current converter workflow pattern

Would this scope align?

Maintainers/ @jbonofre , would this converter and proposed v1 scope be welcome? For ambiguous metric placement, my preference is to require explicit owning-cube metadata or an agreed CLI fallback rather than choosing a cube arbitrarily. For naming, I propose apache-ossie-cube, ossie_cube, and ossie-cube, consistent with the newer Apache Ossie converters.

Once the converter is confirmed as welcome and we agree on the scope and division of work, I’d be glad to coordinate the implementation and request review from the relevant converter contributors.