Discussion: How do we expect OSI to be used

[willpugh]

I'm wondering how we expect OSI to be use. As we move forward, I think that it affects how we reason about and rank solutions.

I see two different approaches that are implicit in some of the discussions I've seen and proposals:

1. OSI is an interchange format. Mainly used to help with conversions, and provide one place to consolidate models from different tools, but does not specify its own semantics or execution model. Vendors can have radically different expression languages, so extension is easy, but ensuring everything acts the same is hard.
2. OSI is a standard like SQL. In this approach, there are semantics, expression language, etc. Vendors are expected to implement the standard, and for the most part, one expression should work across compliant vendors. Vendor extensions can come in new functions and metadata, but semantics MUST comply with the standard.

Clearly approach 1 is easier as a standards body to implement, however, from the meeting a week ago, I got the impression that we think approach 2 is more valuable.

Do folks generally agree with that? or is this contentious?

I bring this up, because there are already things in the proposal that seem to tie us to 1. For example the dialects part of an expression:

expression:
    dialects:
      - dialect: ANSI_SQL
        expression: ss_sold_date_sk

The assertion behind adding this property is:

• It is common enough for expressions to differ by provider, that we should make the concept a first class citizen at the lowest level.
• It is common enough for expressions to have multiple implementations across providers that we should support this directly.

If we were to solve handling vendor extensions with the assumption that we are doing 2, then I would expect an approach that has vendor at the file level and allows for composition so the standards compliant behaviour can be shared.

Does anyone have thoughts on this?
My take is that 2 is a better approach, even though it has a higher amount of work to get right.

[dmariani]

My take is that we should strive for 2, because delivering a language standard would kill two birds with one stone: it can serve as an interchange format and a standard language. Why not strive for both?

[srcolinas]

Are there any progress on this topic? I was looking for a true standard to define the spec of a semantic model for a tool I am working on.

[QMalcolm]

Hi @srcolinas there has been a fair bit of discussion on this. We actually just published the project roadmap yesterday 🎉 Under the Metric Semantics & Core Semantic Model you'll find that one of the deliverables is Standard metrics specification language. There is currently a proposal document being worked on can be found here: https://docs.google.com/document/d/1Jn98kHWsnbvo1MycQBAnWirGTwRmw-DWaen7coNCYWA/edit?usp=sharing

It's less apparent than we'd like because we've been figuring out the proper process and tooling so people can find 😞 We are working to fix that though.

[magerdaniel]

I clearly support option 1. Option 2 is an utopie in my eyes