Struct openness in definition

[cueckoo]

Originally opened by @eonpatapon in cuelang/cue#527

Given:

#B: a: number

#A: {
	c: [string]: #B
	c: {
		foo: _
		bar: _
	}
}

m: #A

m: c: {
	foo: a: 1
	bar: a: 2
	foobar: a: 10
}

I expect #A: c to be closed and so error on field m: c: foobar.

Eval (0.2.2 or 0.3.0):

#B: {
    a: number
}
#A: {
    c: {
        foo: {
            a: number
        }
        bar: {
            a: number
        }
    }
}
m: {
    c: {
        foo: {
            a: 1
        }
        bar: {
            a: 2
        }
        foobar: {
            a: 10
        }
    }
}

In my mind m: c: foobar should not be allowed.

I noticed that the introduction of the template #A: c: [string]: #B makes #A: c open, is it the correct behaviour ?

How to make the #A: c struct closed in this case ?

if I define:

#A: {
	c: [string]: #B
	c: close({
		foo: _
		bar: _
	})
}

It works properly with cue 0.3 (but not cue 0.2.2). Is this the way to go ?

m.c: field `foobar` not allowed:
    ./openness.cue:4:5
    ./openness.cue:5:11
    ./openness.cue:11:4
    ./openness.cue:16:2

In my real use case the template is quite useful.

[cueckoo]

Original reply by @myitcv in cuelang/cue#527 (comment)

Can you clarify what you are trying to achieve with your #A definition? Because currently (emphasis important, see below) it is equivalent to:

#A: {
        c: {
                {
                        [string]: #B
                }
                foo: _
                bar: _
        }
}

which basically says that all the fields of c (including foo and bar) must be of type #B. This constraint does not preclude additional fields beyond foo and bar, just so long as those additional fields are of type #B.

In the near future (not yet implemented), the semantics are going to change to those detailed in the spec: https://tip.cuelang.org/docs/references/spec/#structs. (This was also detailed in the v0.2.0 release under the "Added limitations on bulk optional fields" heading).

Under this new implementation:

#A: {
        c: {
                {
                        [string]: #B
                }
                foo: _
                bar: _
        }
}

would constrain all fields of c to be of type #B but not allow additional fields (because of the embedding of the constraint). i.e. it would simply constrain foo and bar to be of type #B.

Whereas:

#A: {
        c: {
                [string]: #B
                foo: _
                bar: _
        }
}

would allow additional fields in c (beyond foo and bar) that would need to be of type #B. foo and bar could be of any type. This is a pattern constraint. Note the pattern constraint does not apply to foo and bar.

The one additional case which is covered in the spec is that of default constraints.

[cueckoo]

Original reply by @myitcv in cuelang/cue#527 (comment)

One other quick clarifying question, are the types of the various fields of the configurations structs going to vary? Or will they all be precisely #Configuration? e.g for #EnvA.configurations, the fields a and b are currently #Configuration. Is that as you intended or is #Configuration essentially the "base" for more specific configuration types? e.g.

#aFieldType: {
    #Configuration 
    other: string
}

Also noting in the context of this discussion: cuelang/cue#531

[cueckoo]

Original reply by @myitcv in cuelang/cue#527 (comment)

Just further noting that I've retitled #531. The spec is actually incorrect in its suggestion that close() works recursively.

[cueckoo]

Original reply by @eonpatapon in cuelang/cue#527 (comment)

One other quick clarifying question, are the types of the various fields of the configurations structs going to vary? Or will they all be precisely #Configuration? e.g for #EnvA.configurations, the fields a and b are currently #Configuration. Is that as you intended or is #Configuration essentially the "base" for more specific configuration types? e.g.

For now all configurations uses the #Configuration def, it actually represent some terraform manifests in some other repo that I need to pull and apply. And also the environment required to run the configuration (provider access an so on). With the name and a version, I can deduce what to pull. As for the configuration inputs it is completely open (defined as {...}) as I don't want to scare my coworkers.

But yes ideally each configuration (a, b, c) cloud embed #Configuration in their own definition to define precisely the inputs needed. And this discussion wouldn't exists :)

[cueckoo]

Original reply by @eonpatapon in cuelang/cue#527 (comment)

Following on that given:

#Config: {
  commonField: string
}

#ConfigA: {
  #Config 
  aField: int
}

#ConfigB: {
  #Config 
  bField: string
}

#EnvA: {
   configurations: {
      a: #ConfigA
      b: #ConfigB
   }
}

Will cue provide a way to express this kind of constrain: #EnvA: configurations: [string]: [Must contain #Config] ?

But also I'm feeling that what I'm doing here is some kind of inheritence that cue wants to avoid.

[cueckoo]

Original reply by @myitcv in cuelang/cue#527 (comment)

Will cue provide a way to express this kind of constrain: #EnvA: configurations: [string]: [Must contain #Config] ?

😄 - that was precisely the reason for my question because I suspected this was where you were heading.

There's an answer for "currently" and a slightly improved answer for "when the JSON Schema semantics change lands" (what I referenced earlier). The comment below explains the difference:

package x

#Env: {
	configurations: [string]: #AnyConfig
}

#AnyConfig: {
	// When the JSON Schema semantics land, replace this with
	// #Config (at which point it can be inlined above). For now
	// it must be kept in sync with #Config's definition
	commonField: string
	...
}

#Config: {
	commonField: string
}

#EnvA: #Env & {
	configurations: #EnvAConfigurations
}

#EnvAConfigurations: {
	a: #ConfigA
	b: #ConfigB
}

#ConfigA: {
	#Config
	aField: int
}

#ConfigB: {
	#Config
	bField: string
}

myEnv: #EnvA & {
	configurations: {
		a: {
			commonField: "hi"
			aField:      4
		}
		b: {
			commonField: "hi"
			bField:      "ok"
		}
	}
}

[cueckoo]

Original reply by @mpvl in cuelang/cue#527 (comment)

Another way to see this: spec says that definitions are only closed upon reference. So multiple fields with the same value get unified before they get closed. In the past not doing so has led to confusion as

#A: foo: a: int
#A: foo: b: int

was expected to mean the same as

#A: foo: {
  a: int
  b: int
}

From a composability perspective, as well as from the substitution principle, this makes a lot of sense.

close indeed overrides this. Closedness in v0.2.2 was heavily broken (both for the old and new spec). v0.3.0-alpha3 has a pretty solid closedness implementation that adheres closely to the spec.

#A: {
	c: [string]: #B
	c: close({
		foo: _
		bar: _
	})
}

is indeed a valid way to achieve what you want.

Another way would be:

#C: {
  foo: _
  bar: _
}

#B: a: number

#A: #C & {
	c: [string]: #B
}

In this case, #A the referencing of #C by #A closes #C, which carries forward as a closed struct in #A (like the use close).

Note that closedness is not enforced for embedding (analogous to Go), so

#A: {
  #C
  c: [string]: #B
}

would not have the desired effect.