-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP/Feedback needed] Capture more attributes to create multiple deployment #211
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
/* | ||
Copyright 2020 The Flux authors | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package v1beta2 | ||
|
||
const ( | ||
// ImageURLInvalidReason represents the fact that a given repository has an invalid image URL. | ||
ImageURLInvalidReason string = "ImageURLInvalid" | ||
) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
/* | ||
Copyright 2020 The Flux authors | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
// Package v1beta2 contains API types for the image API group, version | ||
// v1beta2. These types are concerned with reflecting metadata from | ||
// OCI image repositories into a cluster, so they can be consulted for | ||
// e.g., automation. | ||
// | ||
// +kubebuilder:object:generate=true | ||
// +groupName=image.toolkit.fluxcd.io | ||
package v1beta2 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
/* | ||
Copyright 2020 The Flux authors | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
// Package v1beta2 contains API Schema definitions for the image v1beta2 API group | ||
//+kubebuilder:object:generate=true | ||
//+groupName=image.toolkit.fluxcd.io | ||
package v1beta2 | ||
|
||
import ( | ||
"k8s.io/apimachinery/pkg/runtime/schema" | ||
"sigs.k8s.io/controller-runtime/pkg/scheme" | ||
) | ||
|
||
var ( | ||
// GroupVersion is group version used to register these objects | ||
GroupVersion = schema.GroupVersion{Group: "image.toolkit.fluxcd.io", Version: "v1beta2"} | ||
|
||
// SchemeBuilder is used to add go types to the GroupVersionKind scheme | ||
SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} | ||
|
||
// AddToScheme adds the types in this group-version to the given scheme. | ||
AddToScheme = SchemeBuilder.AddToScheme | ||
) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,174 @@ | ||
/* | ||
Copyright 2020, 2021 The Flux authors | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package v1beta2 | ||
|
||
import ( | ||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
|
||
"github.com/fluxcd/pkg/apis/meta" | ||
) | ||
|
||
const ImagePolicyKind = "ImagePolicy" | ||
|
||
// ImagePolicySpec defines the parameters for calculating the | ||
// ImagePolicy | ||
type ImagePolicySpec struct { | ||
// ImageRepositoryRef points at the object specifying the image | ||
// being scanned | ||
// +required | ||
ImageRepositoryRef meta.NamespacedObjectReference `json:"imageRepositoryRef"` | ||
// Policy gives the particulars of the policy to be followed in | ||
// selecting the most recent image | ||
// +required | ||
Policy ImagePolicyChoice `json:"policy"` | ||
// FilterTags enables filtering for only a subset of tags based on a set of | ||
// rules. If no rules are provided, all the tags from the repository will be | ||
// ordered and compared. | ||
// +optional | ||
FilterTags *TagFilter `json:"filterTags,omitempty"` | ||
} | ||
|
||
// ImagePolicyChoice is a union of all the types of policy that can be | ||
// supplied. | ||
type ImagePolicyChoice struct { | ||
// SemVer gives a semantic version range to check against the tags | ||
// available. | ||
// +optional | ||
SemVer *SemVerPolicy `json:"semver,omitempty"` | ||
// Alphabetical set of rules to use for alphabetical ordering of the tags. | ||
// +optional | ||
Alphabetical *AlphabeticalPolicy `json:"alphabetical,omitempty"` | ||
// Numerical set of rules to use for numerical ordering of the tags. | ||
// +optional | ||
Numerical *NumericalPolicy `json:"numerical,omitempty"` | ||
} | ||
|
||
// SemVerPolicy specifies a semantic version policy. | ||
type SemVerPolicy struct { | ||
// Range gives a semver range for the image tag; the highest | ||
// version within the range that's a tag yields the latest image. | ||
// +required | ||
Range string `json:"range"` | ||
} | ||
|
||
// AlphabeticalPolicy specifies a alphabetical ordering policy. | ||
type AlphabeticalPolicy struct { | ||
// Order specifies the sorting order of the tags. Given the letters of the | ||
// alphabet as tags, ascending order would select Z, and descending order | ||
// would select A. | ||
// +kubebuilder:default:="asc" | ||
// +kubebuilder:validation:Enum=asc;desc | ||
// +optional | ||
Order string `json:"order,omitempty"` | ||
} | ||
|
||
// NumericalPolicy specifies a numerical ordering policy. | ||
type NumericalPolicy struct { | ||
// Order specifies the sorting order of the tags. Given the integer values | ||
// from 0 to 9 as tags, ascending order would select 9, and descending order | ||
// would select 0. | ||
// +kubebuilder:default:="asc" | ||
// +kubebuilder:validation:Enum=asc;desc | ||
// +optional | ||
Order string `json:"order,omitempty"` | ||
} | ||
|
||
// TagFilter enables filtering tags based on a set of defined rules | ||
type TagFilter struct { | ||
// Pattern specifies a regular expression pattern used to filter for image | ||
// tags. | ||
// +optional | ||
Pattern string `json:"pattern"` | ||
// Extract allows a capture group to be extracted from the specified regular | ||
// expression pattern, useful before tag evaluation. | ||
// +optional | ||
Extract string `json:"extract"` | ||
// Discriminator allows to split the tags in multiple groups and execute the | ||
// policy against each of those groups. This value if present is a capture | ||
// group to be extracted from the regular expression pattern. | ||
// +optional | ||
Discriminator string `json:"discriminator,omitempty"` | ||
// Additional attributes we want to extract from the tag. | ||
// +optional | ||
Attributes []string `json:"attributes,omitempty"` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I kind of like this idea of having some ability to extract attribute parts from the tag value. What I'm not quite sure about is if this should actually be the responsibility of the image policy or if it should be something implemented at the level of the image automation controller. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thinking on easiest way to use this, Image automation is a global thing (process multiple policy) so configuration to extract data cannot be set on the image configuration itself |
||
} | ||
|
||
// ImagePolicyStatus defines the observed state of ImagePolicy | ||
type ImagePolicyStatus struct { | ||
// LatestImage gives the first in the list of images scanned by | ||
// the image repository, when filtered and ordered according to | ||
// the policy. | ||
LatestImage string `json:"latestImage,omitempty"` | ||
// Distribution of tags scanned by the image repository | ||
// +optional | ||
Distribution map[string]ImageAndAttributes `json:"distribution,omitempty"` | ||
// Discriminator of the latest image. | ||
// +optional | ||
LatestDiscriminator string `json:"latestDiscriminator,omitempty"` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is where I see a contradiction. If you actually have the image policy keep track of the value for what the latest discriminator would be then why would this even exist in the first place if you actually want to share this policy across multiple discriminator scopes? This is pretty much equivalent to an image policy definition like this:
Which would generate a number such as 124202112051625 which can be compared, but doesn't really help in your situation if you want to scope it to the PR identifier since it will always return whatever the latest PR + timestamp values combination would be. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The latest image is kept to be compatible with the setter policy, the latest discriminator is kept for a more practical reason. As mention before, I created a "duplication" policy on the automation controller. So from 1 kubernetes object (which have a comment using a policy), I want to create multiple object, 1 per discriminator value. But what with the original object ? is this something we can hide ? is this something we want to deploy ? ... I chose to kep the last distriminator value only on this purpose: the original object is set to the latest image ( with the sorted value) so I need to keep the latest discriminator to be able to find attributes. |
||
// len of the distribution map. Statistics only. | ||
NbDistribution int `json:"nbDistribution,omitempty"` | ||
// +optional | ||
ObservedGeneration int64 `json:"observedGeneration,omitempty"` | ||
// +optional | ||
Conditions []metav1.Condition `json:"conditions,omitempty"` | ||
} | ||
|
||
type ImageAndAttributes struct { | ||
Image string `json:"image"` | ||
Tag string `json:"tag"` | ||
// +optional | ||
Attributes map[string]string `json:"attributes,omitempty"` | ||
} | ||
|
||
func (p *ImagePolicy) GetStatusConditions() *[]metav1.Condition { | ||
return &p.Status.Conditions | ||
} | ||
|
||
// SetImageRepositoryReadiness sets the ready condition with the given status, reason and message. | ||
func SetImagePolicyReadiness(p *ImagePolicy, status metav1.ConditionStatus, reason, message string) { | ||
p.Status.ObservedGeneration = p.ObjectMeta.Generation | ||
meta.SetResourceCondition(p, meta.ReadyCondition, status, reason, message) | ||
} | ||
|
||
// +kubebuilder:storageversion | ||
// +kubebuilder:object:root=true | ||
// +kubebuilder:subresource:status | ||
// +kubebuilder:printcolumn:name="LatestImage",type=string,JSONPath=`.status.latestImage` | ||
// +kubebuilder:printcolumn:name="Nb distribution",type=string,JSONPath=`.status.nbDistribution` | ||
|
||
// ImagePolicy is the Schema for the imagepolicies API | ||
type ImagePolicy struct { | ||
metav1.TypeMeta `json:",inline"` | ||
metav1.ObjectMeta `json:"metadata,omitempty"` | ||
|
||
Spec ImagePolicySpec `json:"spec,omitempty"` | ||
// +kubebuilder:default={"observedGeneration":-1} | ||
Status ImagePolicyStatus `json:"status,omitempty"` | ||
} | ||
|
||
// +kubebuilder:object:root=true | ||
|
||
// ImagePolicyList contains a list of ImagePolicy | ||
type ImagePolicyList struct { | ||
metav1.TypeMeta `json:",inline"` | ||
metav1.ListMeta `json:"metadata,omitempty"` | ||
Items []ImagePolicy `json:"items"` | ||
} | ||
|
||
func init() { | ||
SchemeBuilder.Register(&ImagePolicy{}, &ImagePolicyList{}) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not quite sure I understand the need for a discriminator. For this use case, it is recommended to just define a different image policy, scoped to the identifier (PR id in the prefix in your documented use case). e.g.
^pr-123-(?P<ts>\d*)-(?P<sha1>.*)$
.Based on your description, you want to create ephemeral environments for each pull request, so why not just create an image policy resource as part of the automation that would build that in the first place? Having the discriminator doesn't really help as you still need to somehow tell the IUA which discriminator value you want to use for the respective setter marker.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thing is I do not want to use the setter policy, I want to have ephemeral environment, so if I have 3 open PR, I want to have 3 environments. Setter policy is just a setter on yaml file, it cannot help me there as it would create a single environment.
If I want to create 3 environments I would have to create 3 policies with associated kubernetes objects but then I feel it contradict the gitops approach as the CI would have to create deployment objet (and then why use flux to automate what is already done by the CI)
It is why there is another PR on the automation controller to be able to use one policy and duplicate some objects.
The goal would be to to be able to create a docker image on the CI (and delete some when PR is closed) and then flux on its own would be able to trigger what need to be triggered.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general, changes to code can be accompanied by changes to configuration. For example, if you add a feature to your app, you might have some new argument that needs to be supplied to the image; or, to have a feature flag set, to enable the feature. For this reason, I think of preview environments as being driven by changes to configs (YAMLs), rather than new images; i.e., automation belongs outside the rather restricted view of the image reflector.