Versioning semantic models

[supertramp17]

Do we plan to support adding version number for the semantic model in the spec ? There would be benefits in doing that especially when it comes to detecting drift and ensuring the modes are always current.

Do we see benefits in having this field ? I'm assuming this might have already been discussed within the core working group.

I couldn't locate this in the roadmap. it could look something like this. Keen to know your thoughts on this.

semantic_model:
     version_number: 0.1
  - name: sales_analytics
    description: Sales and customer analytics model
    ai_context:
      instructions: "Use this model for sales analysis and customer insights"
    datasets: []
    relationships: []
    metrics: []
    custom_extensions:
      - vendor_name: DBT
        data: '{"project_name": "tpcds_analytics", "models_path": "models/semantic"}'

[QMalcolm]

@supertramp17, I think the answer is "maybe"? 😅

Currently in the roadmap we do briefly touch on versioning. Specifically under Catalog Integration & Semantic Services section we mention deliverables of:

• Standalone semantic service / registry
• Discovery, versioning, and access control for OSI models

Now whether versioning takes form as a "version number", I'm unsure. If there is a registry the "version" could be a SHA hash of the semantic model's content. Then knowing if something changed would just be a matter of "check if the hash changed". Without a registry, I believe the assumption is that the OSI documents are stored in a git repository. A commit SHA of the repository would tell one if they are on the latest or not, though a commit SHA changing doesn't necessarily mean anything changed in a specific OSI file, but in the repository overall.

That is to say, there is definitely value in tracking history of a semantic model. The most top of mind cases being "has it changed" and "how is it different from what came before". Whether a version number encoded onto a semantic model is the way to go, I am unsure of (neither positive or negative). My biggest question in my head is "should this be encoded in the spec or should this be an artifact managed by the storage / registry system?"

[supertramp17]

Thanks for the response @QMalcolm . The motivation behind this suggestion is mainly audibility and traceability like you have alluded to as well

The most top of mind cases being "has it changed" and "how is it different from what came before".

Has it changed and what has changed can be found out if we do a diff of two versions but we would need to maintain the version numbers even for that. Why has it changed imo should not be part of the spec. Whatever systems or workflows organisations have can address that. Quite likely they might not even be using git to keep the osi files. and if they are using git, commit messages can address that. Besides, the data stewards/owners might want to have their own review/approve workflows before an updated version of the semantic model is used.

I haven't looked at the roadmap, will have a read and come back.