Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Unify Grafana Kind and Kubernetes Kind Formats #32

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

[WIP] Unify Grafana Kind and Kubernetes Kind Formats #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 26 additions & 30 deletions kindcat_custom.cue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ import (
// _kubeObjectMetadata is metadata found in a kubernetes object's metadata field.
// It is not exhaustive and only includes fields which may be relevant to a kind's implementation,
// As it is also intended to be generic enough to function with any API Server.
_kubeObjectMetadata: {
KubeObjectMetadata: {
namespace: string
name: string
Comment on lines +13 to +14
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it feels odd to qualify these as kube metadata... since they are essentially the identifiers. I was expecting kube metadata to be the more essoteric k8s specific things like:

  • finalizers
  • managedFields
  • deletionGracePeriodSeconds
  • etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mainly to make them part of what kubernetes considers "metadata," i.e. what kubernetes puts in its metadata field, so that KubeMetadata is exactly compatible with the metadata field in kubernetes.

We could present a different interface via Resource, but I think we'd want generated code/UnstructuredResource to be formatted the same way the kubernetes object itself is.

uid: string
creationTimestamp: string & time.Time
deletionTimestamp?: string & time.Time
Expand All @@ -18,12 +20,21 @@ _kubeObjectMetadata: {
labels: {
[string]: string
}
annotations: {
[string]: string
}
}

GrafanaMetadata: {
updateTimestamp: string & time.Time
createdBy: string
updatedBy: string
}

// CommonMetadata is a combination of API Server metadata and additional metadata
// intended to exist commonly across all kinds, but may have varying implementations as to its storage mechanism(s).
CommonMetadata: {
_kubeObjectMetadata
KubeObjectMetadata

updateTimestamp: string & time.Time
createdBy: string
Expand All @@ -37,37 +48,20 @@ CommonMetadata: {
}
}

// _crdSchema is the schema format for a CRD.
_crdSchema: {
// metadata contains embedded CommonMetadata and can be extended with custom string fields
// TODO: use CommonMetadata instead of redefining here; currently needs to be defined here
// without external reference as using the CommonMetadata reference breaks thema codegen.
metadata: {
_kubeObjectMetadata

updateTimestamp: string & time.Time
createdBy: string
updatedBy: string

// TODO: additional metadata fields?
// Additional metadata can be added at any future point, as it is allowed to be constant across lineage versions

// extraFields is reserved for any fields that are pulled from the API server metadata but do not have concrete fields in the CUE metadata
extraFields: {
[string]: _
}
} & {
// All extensions to this metadata need to have string values (for APIServer encoding-to-annotations purposes)
// Can't use this as it's not yet enforced CUE:
//...string
// Have to do this gnarly regex instead
[!~"^(uid|creationTimestamp|deletionTimestamp|finalizers|resourceVersion|labels|updateTimestamp|createdBy|updatedBy|extraFields)$"]: string
}
_grdSchema: {
// metadata is the kubernetes metadata
metadata: KubeObjectMetadata
// grafanaMetadata is the grafana metadata
grafanaMetadata: GrafanaMetadata
// kindMetadata is kind-specfic metadata. It may be empty.
kindMetadata?: _
// spec is the resource's body
spec: _

// cuetsy is not happy creating spec with the MinFields constraint directly
_specIsNonEmpty: spec & struct.MinFields(0)

// status is the status subresource
status: {
#OperatorState: {
// lastEvaluation is the ResourceVersion last evaluated
Expand Down Expand Up @@ -120,11 +114,13 @@ Custom: S={
if isCRD {
// If the crd trait is defined, the schemas in the lineage must follow the format:
// {
// "metadata": CommonMetadata & {...string}
// "metadata": KubeMetadata
// "grafanaMetadata": GrafanaMetadata
// "kindMetadata": {...}
// "spec": {...}
// "status": {...}
// }
lineage: joinSchema: _crdSchema
lineage: joinSchema: _grdSchema
}

// crd contains properties specific to converting this kind to a Kubernetes CRD.
Expand Down
158 changes: 118 additions & 40 deletions resource.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
package kindsys

import (
"fmt"
"reflect"
"strings"
"time"
)

Expand Down Expand Up @@ -29,6 +31,16 @@ type UnmarshalConfig struct {

// A Resource is a single instance of a Grafana [Kind], either [Core] or [Custom].
//
// A Resource is broadly composed of metadata, spec, and additional "subresources."
// Metadata is split into three sub-components:
// - KubeMetadata is the metadata provided in the "metadata" component of a kubernetes resource
// - GrafanaMetadata is additional, standard Grafana Resource metadata
// - KindMetadata is metadata specific to the Kind
// Spec is the Resource's main payload, what could classicly be considered the "body" of the Resource.
// Subresources are additional components of the Resource which are not considered part of either the metadata or
// the spec ("body"). For properly-defined Grafana Resources, this will always include the "status" subresource,
// and may include others on a per-Kind basis.
//
// The relationship between Resource and [Kind] is similar to the
// relationship between objects and classes in conventional object oriented
// design:
Expand All @@ -48,13 +60,24 @@ type UnmarshalConfig struct {
// to produce a struct that implements [Resource] for each kind. Such a struct
// can be used as the generic type parameter to create a [TypedCore] or [TypedCustom]
type Resource interface {
// CommonMetadata returns the Resource's CommonMetadata
CommonMetadata() CommonMetadata
// KubeMetadata returns the kubernetes standard resource metadata
KubeMetadata() KubeMetadata

// GrafanaMetadata returns the grafana metadata for the resource
GrafanaMetadata() GrafanaMetadata

// CustomMetadata returns metadata unique to this Resource's kind, as opposed to Common and Static metadata,
// which are the same across all kinds. An object may have no kind-specific CustomMetadata.
// CustomMetadata can only be read from this interface, for use with resource.Client implementations,
// those who wish to set CustomMetadata should use the interface's underlying type.
KindMetadata() CustomMetadata

// SetCommonMetadata overwrites the CommonMetadata of the object.
// Implementations should always overwrite, rather than attempt merges of the metadata.
// Callers wishing to merge should get current metadata with CommonMetadata() and set specific values.
SetCommonMetadata(metadata CommonMetadata)
SetKubeMetadata(metadata KubeMetadata)

SetGrafanaMetadata(metadata GrafanaMetadata)

// StaticMetadata returns the Resource's StaticMetadata
StaticMetadata() StaticMetadata
Expand All @@ -65,17 +88,12 @@ type Resource interface {
// Note that StaticMetadata is only mutable in an object create context.
SetStaticMetadata(metadata StaticMetadata)

// CustomMetadata returns metadata unique to this Resource's kind, as opposed to Common and Static metadata,
// which are the same across all kinds. An object may have no kind-specific CustomMetadata.
// CustomMetadata can only be read from this interface, for use with resource.Client implementations,
// those who wish to set CustomMetadata should use the interface's underlying type.
CustomMetadata() CustomMetadata

// SpecObject returns the actual "schema" object, which holds the main body of data
SpecObject() any

// Subresources returns a map of subresource name(s) to the object value for that subresource.
// Spec is not considered a subresource, and should only be returned by SpecObject
// No guarantees are made that mutations to objects in the map will affect the underlying resource.
Subresources() map[string]any

// Copy returns a full copy of the Resource with all its data
Expand Down Expand Up @@ -160,14 +178,12 @@ type ListMetadata struct {
ExtraFields map[string]any `json:"extraFields"`
}

// CommonMetadata is the system-defined common metadata associated with a [Resource].
// It combines Kubernetes standard metadata with certain Grafana-specific additions.
//
// It is analogous to [k8s.io/apimachinery/pkg/apis/meta/v1.ObjectMeta] in vanilla Kubernetes.
//
// TODO generate this from the CUE definition
// TODO review this for optionality
type CommonMetadata struct {
// KubeMetadata is kubernetes object metadata. It does not directly import kubernetes go structs to avoid requiring kubernetes dependencies in projects
// which use kindsys.
// TODO: does this matter? Could we just import kubernetes' struct and attach methods we need to it?
type KubeMetadata struct {
Name string
Namespace string
// UID is the unique ID of the object. This can be used to uniquely identify objects,
// but is not guaranteed to be usable for lookups.
UID string `json:"uid"`
Expand All @@ -179,6 +195,8 @@ type CommonMetadata struct {
// Labels are string key/value pairs attached to the object. They can be used for filtering,
// or as additional metadata.
Labels map[string]string `json:"labels"`
// Annotations
Annotations map[string]string `json:"annotations"`
// CreationTimestamp indicates when the resource has been created.
CreationTimestamp time.Time `json:"creationTimestamp"`
// DeletionTimestamp indicates that the resource is pending deletion as of the provided time if non-nil.
Expand All @@ -190,6 +208,32 @@ type CommonMetadata struct {
// DeletionTimestamp is set to the time of the "delete," and the resource will continue to exist
// until the finalizers list is cleared.
Finalizers []string `json:"finalizers"`
}

func (k KubeMetadata) Copy() KubeMetadata {
n := KubeMetadata{
Name: k.Name,
Namespace: k.Namespace,
UID: k.UID,
ResourceVersion: k.ResourceVersion,
CreationTimestamp: k.CreationTimestamp,
}
if k.DeletionTimestamp != nil {
*n.DeletionTimestamp = *(k.DeletionTimestamp)
}
copy(n.Finalizers, k.Finalizers)
for key, val := range k.Annotations {
n.Annotations[key] = val
}
for key, val := range k.Labels {
k.Labels[key] = val
}
return n
}

// TODO
// On encoding to kubernetes, fields in GrafanaMetadata MUST be encoded into annotations with the name "grafana.com/X", where X is the JSON name of the field.
type GrafanaMetadata struct {
// UpdateTimestamp is the timestamp of the last update to the resource.
UpdateTimestamp time.Time `json:"updateTimestamp"`
// CreatedBy is a string which indicates the user or process which created the resource.
Expand All @@ -198,12 +242,10 @@ type CommonMetadata struct {
// UpdatedBy is a string which indicates the user or process which last updated the resource.
// Implementations may choose what this indicator should be.
UpdatedBy string `json:"updatedBy"`
}

// ExtraFields stores implementation-specific metadata.
// Not all Client implementations are required to honor all ExtraFields keys.
// Generally, this field should be shied away from unless you know the specific
// Client implementation you're working with and wish to track or mutate extra information.
ExtraFields map[string]any `json:"extraFields"`
func (g GrafanaMetadata) Copy() GrafanaMetadata {
return GrafanaMetadata{}
}

// TODO guard against skew, use indirection through an internal package
Expand All @@ -221,34 +263,70 @@ func (s SimpleCustomMetadata) MapFields() map[string]any {
// BasicMetadataObject provides a Metadata field composed of StaticMetadata and ObjectMetadata, as well as the
// ObjectMetadata(),SetObjectMetadata(), StaticMetadata(), and SetStaticMetadata() receiver functions.
type BasicMetadataObject struct {
StaticMeta StaticMetadata `json:"staticMetadata"`
CommonMeta CommonMetadata `json:"commonMetadata"`
CustomMeta SimpleCustomMetadata `json:"customMetadata"`
}

// CommonMetadata returns the object's CommonMetadata
func (b *BasicMetadataObject) CommonMetadata() CommonMetadata {
return b.CommonMeta
}

// SetCommonMetadata overwrites the ObjectMetadata.Common() supplied by BasicMetadataObject.ObjectMetadata()
func (b *BasicMetadataObject) SetCommonMetadata(m CommonMetadata) {
b.CommonMeta = m
Kind string `json:"kind"`
APIVersion string `json:"apiVersion"`
Metadata KubeMetadata `json:"metadata"`
GrafanaMeta GrafanaMetadata `json:"grafanaMetadata"`
KindMeta SimpleCustomMetadata `json:"kindMetadata"`
}

// StaticMetadata returns the object's StaticMetadata
func (b *BasicMetadataObject) StaticMetadata() StaticMetadata {
return b.StaticMeta
gv := strings.Split(b.APIVersion, "/")
g := ""
v := ""
if len(gv) > 1 {
g = gv[0]
v = gv[1]
} else if len(gv) == 1 {
// For kubernetes core resources, the group is empty and only the version appears in the APIVersion string
v = gv[0]
}
return StaticMetadata{
Kind: b.Kind,
Group: g,
Version: v,
Namespace: b.Metadata.Namespace,
Name: b.Metadata.Name,
}
}

// SetStaticMetadata overwrites the StaticMetadata supplied by BasicMetadataObject.StaticMetadata()
// Note that in implementations, this may impact the KubeMetadata, as they have overlapping information.
func (b *BasicMetadataObject) SetStaticMetadata(m StaticMetadata) {
b.StaticMeta = m
b.Kind = m.Kind
if m.Group != "" {
b.APIVersion = fmt.Sprintf("%s/%s", m.Group, m.Version)
} else {
b.APIVersion = m.Version
}
b.Metadata.Namespace = m.Namespace
b.Metadata.Name = m.Name
}

// KubeMetadata returns the object's KubeMetadata
func (b *BasicMetadataObject) KubeMetadata() KubeMetadata {
return b.Metadata
}

// SetKubeMetadata overwrites the KubeMetadata supplied by BasicMetadataObject.KubeMetadata()
func (b *BasicMetadataObject) SetKubeMetadata(m KubeMetadata) {
b.Metadata = m
}

// GrafanaMetadata returns the object's GrafanaMetadata
func (b *BasicMetadataObject) GrafanaMetadata() GrafanaMetadata {
return b.GrafanaMeta
}

// SetGrafanaMetadata overwrites the GrafanaMetadata supplied by BasicMetadataObject.GrafanaMetadata()
func (b *BasicMetadataObject) SetGrafanaMetadata(m GrafanaMetadata) {
b.GrafanaMeta = m
}

// CustomMetadata returns the object's CustomMetadata
func (b *BasicMetadataObject) CustomMetadata() CustomMetadata {
return b.CustomMeta
// KindMetadata returns the object's CustomMetadata (kind-specific metadata)
func (b *BasicMetadataObject) KindMetadata() CustomMetadata {
return b.KindMeta
}

// TODO delete these?
Expand Down
42 changes: 16 additions & 26 deletions unstructured.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ var _ Resource = &UnstructuredResource{}
// strongly typed, and lacks any user-defined methods that may exist on a
// kind-specific struct that implements [Resource].
type UnstructuredResource struct {
BasicMetadataObject
Spec map[string]any `json:"spec,omitempty"`
Status map[string]any `json:"status,omitempty"`
BasicMetadataObject `json:",inline"`
Spec map[string]any `json:"spec,omitempty"`
Status map[string]any `json:"status,omitempty"`
// TODO: is there value in storing other subresources in UnstructuredResource?
}

func (u *UnstructuredResource) SpecObject() any {
Expand All @@ -24,32 +25,21 @@ func (u *UnstructuredResource) Subresources() map[string]any {
}

func (u *UnstructuredResource) Copy() Resource {
com := CommonMetadata{
UID: u.CommonMeta.UID,
ResourceVersion: u.CommonMeta.ResourceVersion,
CreationTimestamp: u.CommonMeta.CreationTimestamp.UTC(),
UpdateTimestamp: u.CommonMeta.UpdateTimestamp.UTC(),
CreatedBy: u.CommonMeta.CreatedBy,
UpdatedBy: u.CommonMeta.UpdatedBy,
}

copy(u.CommonMeta.Finalizers, com.Finalizers)
if u.CommonMeta.DeletionTimestamp != nil {
*com.DeletionTimestamp = *(u.CommonMeta.DeletionTimestamp)
}
for k, v := range u.CommonMeta.Labels {
com.Labels[k] = v
}
com.ExtraFields = mapcopy(u.CommonMeta.ExtraFields)

cp := UnstructuredResource{
n := UnstructuredResource{
BasicMetadataObject: BasicMetadataObject{
Kind: u.Kind,
APIVersion: u.APIVersion,
Metadata: u.Metadata.Copy(),
GrafanaMeta: u.GrafanaMeta.Copy(),
KindMeta: SimpleCustomMetadata{},
},
Spec: mapcopy(u.Spec),
Status: mapcopy(u.Status),
}

cp.CommonMeta = com
cp.CustomMeta = mapcopy(u.CustomMeta)
return &cp
for k, v := range u.KindMeta {
n.KindMeta[k] = v
}
return &n
}

func mapcopy(m map[string]any) map[string]any {
Expand Down