cmd/cue: tags across packages

[cueckoo]

Originally opened by @maurerbot in cuelang/cue#1070

cue seems to be unable to pick up tags across packages.

What version of CUE are you using (cue version)?

cue version v0.4.0 linux/amd64

Does this issue reproduce with the latest release?

Yes

What did you do?

cue/cue.mod/module.cue
module: my.tld

cue/schema/base/base.cue

package base

#metadata: {
	name:         string @tag(name)
	description?: string
}

let Spec = {}

#base: {
	apiVersion: string
	kind:       string
	metadata:   #metadata
	spec?:      Spec
}

cue/schema/v1alpha1/test.cue

package v1alpha1

import "my.tld/schema/base"

let Spec = {
	keys?: [...{
		name: string
		pk:   =~"([a-zA-Z0-9_-]){8,256}"
	}]
} 

test: base.#base & {
	kind:       "test"
	apiVersion: "my.tld/v1alpha1"
	spec: Spec
}

cue eval ./schema/v1alpha1/... -t name="foo" -p v1alpha1 -e test --out=yaml

What did you expect to see?

apiVersion: "my.tld/v1alpha1"
kind:       test
metadata: 
    name: foo

spec: {...}

What did you see instead?

no tag for "name"

Alternatively

cue eval ./schema/... -t name="foo"
#metadata: {
    name: "foo"
}
#base: {
    apiVersion: string
    kind:       string
    metadata: {
        name: "foo"
    }
}
// ---
test: {
    apiVersion: "my.tld/v1alpha1"
    kind:       "test"
    metadata: {
        name: string // this isn't picked up
    }
    spec: {}
}

Other Questions

• Not sure if leveraging the lattice values as a means to define Spec is the right approach

[cueckoo]

Original reply by @maurerbot in cuelang/cue#1070 (comment)

Adding test:metadata:name: string @tag(name) to test.cue seems to work but feels off to me.

[cueckoo]

Original reply by @verdverm in cuelang/cue#1070 (comment)

Related is #1005

[myitcv]

@maurerbot - thanks for raising this. As @verdverm linked above, this is very much related to #1005 insofar as the current documentation is lacking.

FYI - I have just tweaked #1005 to make it clearer to the reader what is going on, and added detail on the ... case.

In case it's useful, here is my encoding of your example using the techniques for creating reproducers described in the wiki:

# No error when at least one package that matches ./schema/...
# does define a tag named "name"
exec cue eval -t name=foo ./schema/...
cmp stdout all.golden

# An error when a named package does not define a tag named "name"
! exec cue eval -t name=foo ./schema/v1alpha1

-- cue.mod/module.cue --
module: "my.tld"
-- schema/base/base.cue --
package base

#metadata: {
	name:         string @tag(name)
	description?: string
}

let Spec = {}

#base: {
	apiVersion: string
	kind:       string
	metadata:   #metadata
	spec?:      Spec
}
-- schema/v1alpha1/test.cue --
package v1alpha1

import "my.tld/schema/base"

let Spec = {
	keys?: [...{
		name: string
		pk:   =~"([a-zA-Z0-9_-]){8,256}"
	}]
}

test: base.#base & {
	kind:       "test"
	apiVersion: "my.tld/v1alpha1"
	spec:       Spec
}
-- all.golden --
#metadata: {
    name: "foo"
}
#base: {
    apiVersion: string
    kind:       string
    metadata: {
        name: "foo"
    }
}
// ---
test: {
    apiVersion: "my.tld/v1alpha1"
    kind:       "test"
    metadata: {
        name: string
    }
    spec: {}
}

This passes. i.e. it's expected that running cue eval -t name=foo with the ... pattern does not fail if any of the packages that match the ... pattern define a tag named name. That command would fail if none of the packages matching ... defined such a tag, just as cue eval -t name=foo $pkg fails if the specific package does not define a tag named name. Notice that tags are not "imported" across package boundaries, not does a -t injection value influence anything other than a package specified for evaluation (to do so would lead to very confusing behaviour, because attributes are entirely arbitrary - the meaning in one package is in no way guaranteed to match the meaning in another).

So I think this is working as expected, modulo us fixing up the documentation to make it clearer (thank you for the ... example, I've just updated #1070 to cover that too).

Stepping back for one second though, can you describe in more detail what you are trying to do here?

[jgillich]

Here is one use case for cross package tags: I'd like to build a system similar to Helm charts for my operator; I put in a CRD and have Cue generate a list of K8s resources. The CRD is always the same type but the "chart" varies depending on a field in the CRD (selection of the chart package happens in the operator).

I inject my CRD using JSON in a tag:

import (
  "encoding/json"
  "github.com/foo/bar/api/v1alpha1"
)

appJson:      string @tag(app)

app: v1alpha1.#App
app: json.Unmarshal(appJson)

Now I a) don't want to include this in my output and b) I don't want to repeat this code in every single chart and c) I want to be able to render just one chart and not all at once.

I think this requires cross package tags. How about a syntax that contains the package name:

cue eval . --inject foo.com/bar/baz.mytag=qux

?

[mpvl]

Being explicit about in which package to inject something seems reasonable.

[mpvl]

I like the package suggestion.

What I also realized, is that for some configurations, maybe yours, there may be full control over the tag namespace. That is, it may be okay to apply tags to all packages in the same module. That will certainly not be okay for all setups, but we could make it an option. For instance, in the above the tag from the package could potentially be defined as

#metadata: {
	name:         string @tag(name, scope=mod)
	description?: string
}

Here, the name tag would be applied to this package when the name tag is applied to any package in the package.

The default scope would be pkg. If the scope is pkg, one could still allow a fully qualified tag like -t path/to/pkg:foo.name=myname.

In a similar vain, I could imagine that a company with multiple CUE modules could want to scope on a specific module prefix, for instance.

Feedback welcome.

[eonpatapon]

Here, the name tag would be applied to this package when the name tag is applied to any package in the packagemodule.

I like the idea.

What would happen if a package x declares:

x: string @tag(name)

And in package y in the same module declares:

y: string @tag(name, scope=mod)

1. Tag is applied from x package
2. Tag is applied from y package
3. Tag is applied at the module root

For me:

1. tag applied in x and y
2. tag applied in y
3. tag applied in y

Maybe another option would be to have a different attribute for module scoped tags, but naming is hard :)

[mpvl]

@eonpatapon @jgillich
In your example, it would depend on what package is loaded as root. If package x is loaded which imports y, then a name tag is applied to both. Whereas if package y is loaded and it imports x, it would only be applied to y.

In other words, for the default scope (package) a tag would only apply if that package is the one that is loaded (the target of the the command line args). For tags listed as module scope, a tag specified on the command line would also apply if the package is merely imported.

So yes, I think we are in agreement.

For people just jumping in: note that tags are a simple pre-evaluation syntactic rewrite. So the loader just scans the syntax tree and rewrites all tags that match. Currently it only does so for the main package. But with this proposal it would need to also scan all imported packages in the same module.

Naming is indeed hard. :) But I would not add another tag-like attribute.

[yoktobit]

Hi, any news about that?
I want to add another use case, because system tags are also affected:

#GroupWithDateInPast: {
    date: time.Format(time.RFC3339Date) & <=_currentDate
    _currentDate: time.Time @tag(now,var=now)
}

If you put that type in some other package and execute cue eval . -T -c on your derived data file, it will fail because the system tag now is not used.