Skip to content

Commit

Permalink
Merge branch eni-cleanup into master
Browse files Browse the repository at this point in the history 
* Call DisassociateTrunkInterface before deleting branch ENI (#372)

* Call DisassociateTrunkInterface before deleting branch ENI

* feat: Centralize leaked ENI cleanup (#374)

* feat: centralized eni cleanup

* Merge master into eni-cleanup (#385)

* fix: paginate DescribeNetworkInterfaces with deep filters (#375)

* fix: paginate DescribeNetworkInterfaces with deep filters

* update metrics and address review comments

* minor updates to address comments

* Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (#380)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump k8s.io/client-go from 0.29.1 to 0.29.2 (#377)

Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](kubernetes/client-go@v0.29.1...v0.29.2)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (#378)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.46.0...v0.49.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Repo controlled build go version (#381)

* update golang version (#383)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jason Du <jasonxdu@amazon.com>

* fix:update cluster tag name in CNINode (#386)

* fix:add node OS label in CNINode, retry get CNINode with backoff

* update protobuf to 1.33.0 (#387)

* add CNINode integration tests (#391)

* use DescribeNetworkInterfaces with deep filters

* add integration test to validate ec2 permissions

* remove DisassociateAllBranchENIs as it is not useful (#400)

* remove DisassociateAllBranchENIs as it is not useful

* skip deletion success log for NotFound ENI

* fix govulncheck

* Merge master branch into eni-cleanup (#416)

* fix: paginate DescribeNetworkInterfaces with deep filters (#375)

* fix: paginate DescribeNetworkInterfaces with deep filters

* update metrics and address review comments

* minor updates to address comments

* Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (#380)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump k8s.io/client-go from 0.29.1 to 0.29.2 (#377)

Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](kubernetes/client-go@v0.29.1...v0.29.2)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (#378)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.46.0...v0.49.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Repo controlled build go version (#381)

* update golang version (#383)

* update protobuf to 1.33.0 (#387)

* pin envtest version due to an upstream bug (#390)

* Bump k8s.io/client-go from 0.29.2 to 0.29.3 (#392)

Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.2 to 0.29.3.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](kubernetes/client-go@v0.29.2...v0.29.3)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/aws/amazon-vpc-cni-k8s from 1.16.0 to 1.17.1 (#393)

Bumps [github.com/aws/amazon-vpc-cni-k8s](https://github.com/aws/amazon-vpc-cni-k8s) from 1.16.0 to 1.17.1.
- [Release notes](https://github.com/aws/amazon-vpc-cni-k8s/releases)
- [Changelog](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/CHANGELOG.md)
- [Commits](aws/amazon-vpc-cni-k8s@v1.16.0...v1.17.1)

---
updated-dependencies:
- dependency-name: github.com/aws/amazon-vpc-cni-k8s
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.49.0 to 0.51.1 (#395)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.49.0 to 0.51.1.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.49.0...v0.51.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/aws/aws-sdk-go from 1.50.29 to 1.51.12 (#397)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.50.29 to 1.51.12.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](aws/aws-sdk-go@v1.50.29...v1.51.12)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add github action to run gosec static analysis (#398)

* add github action to run gosec static analysis

* install gosec

* update golang and dependency to fix CVE (#401)

* revert pagination and call DescribeNetworkInterfaces with vpcID or subnetID filter

* Revert "fix: paginate DescribeNetworkInterfaces with deep filters (#375)"

This reverts commit b5699de.

* call DescribeNetworkInterfaces with vpcID or subnetID filter

* update EC2 supported instance types (#402)

* remove global exclusion for G108,G114 and add nosec in code (#404)

* Update controller_auth_proxy_patch.yaml (#405)

Update the reference from gcr.io to registry.k8s.io

>  kube-rbac-proxy is moving to registry.k8s.io/kubebuilder/kube-rbac-proxy (from gcr.io/kubebuilder/kube-rbac-proxy) because GCR is being sunset. We need to update these references.

* Fix log which causes panic (#407)

* Fix log which causes panic

* Consistent key name

* consistent naming

* run go mod tidy

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jason Du <jasonxdu@amazon.com>
Co-authored-by: Hao Zhou <haouc@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <senthilx@amazon.com>
Co-authored-by: Garvin Pang <garvinpang@protonmail.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jason Du <jasonxdu@amazon.com>
Co-authored-by: Hao Zhou <haouc@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <senthilx@amazon.com>
Co-authored-by: Garvin Pang <garvinpang@protonmail.com>
  • Loading branch information
@sushrk @dependabot
@xdu31 @haouc @orsenthil @GnatorX
6 people committed May 1, 2024
1 parent c80fd41 commit b5a054e
Show file tree
Hide file tree
Showing 49 changed files with 2,097 additions and 610 deletions.
1 change: 1 addition & 0 deletions PROJECT
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ resources:
version: v1beta1
- api:
crdVersion: v1
controller: true
domain: k8s.aws
group: vpcresources
kind: CNINode
Expand Down
2 changes: 2 additions & 0 deletions apis/vpcresources/v1alpha1/cninode_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ type Feature struct {
// CNINodeSpec defines the desired state of CNINode
type CNINodeSpec struct {
Features []Feature `json:"features,omitempty"`
// Additional tag key/value added to all network interfaces provisioned by the vpc-resource-controller and VPC-CNI
Tags map[string]string `json:"tags,omitempty"`
}

// CNINodeStatus defines the managed VPC resources.
Expand Down
7 changes: 7 additions & 0 deletions apis/vpcresources/v1alpha1/zz_generated.deepcopy.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions config/crd/bases/vpcresources.k8s.aws_cninodes.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,12 @@ spec:
type: string
type: object
type: array
tags:
additionalProperties:
type: string
description: Additional tag key/value added to all network interfaces
provisioned by the vpc-resource-controller and VPC-CNI
type: object
type: object
status:
description: CNINodeStatus defines the managed VPC resources.
Expand Down
2 changes: 2 additions & 0 deletions config/rbac/role.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ rules:
- create
- get
- list
- patch
- update
- watch
- apiGroups:
- vpcresources.k8s.aws
Expand Down
12 changes: 2 additions & 10 deletions controllers/core/node_controller.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (

"github.com/aws/amazon-vpc-resource-controller-k8s/apis/vpcresources/v1alpha1"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/condition"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/config"
rcHealthz "github.com/aws/amazon-vpc-resource-controller-k8s/pkg/healthz"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/k8s"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/node/manager"
Expand All @@ -36,15 +37,6 @@ import (
"sigs.k8s.io/controller-runtime/pkg/healthz"
)

// MaxNodeConcurrentReconciles is the number of go routines that can invoke
// Reconcile in parallel. Since Node Reconciler, performs local operation
// on cache only a single go routine should be sufficient. Using more than
// one routines to help high rate churn and larger nodes groups restarting
// when the controller has to be restarted for various reasons.
const (
MaxNodeConcurrentReconciles = 10
)

// NodeReconciler reconciles a Node object
type NodeReconciler struct {
client.Client
Expand Down Expand Up @@ -117,7 +109,7 @@ func (r *NodeReconciler) SetupWithManager(mgr ctrl.Manager, healthzHandler *rcHe

return ctrl.NewControllerManagedBy(mgr).
For(&corev1.Node{}).
WithOptions(controller.Options{MaxConcurrentReconciles: MaxNodeConcurrentReconciles}).
WithOptions(controller.Options{MaxConcurrentReconciles: config.MaxNodeConcurrentReconciles}).
Owns(&v1alpha1.CNINode{}).
Complete(r)
}
Expand Down
252 changes: 252 additions & 0 deletions controllers/crds/cninode_controller.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License"). You may
// not use this file except in compliance with the License. A copy of the
// License is located at
//
// http://aws.amazon.com/apache2.0/
//
// or in the "license" file accompanying this file. This file is distributed
// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
// express or implied. See the License for the specific language governing
// permissions and limitations under the License.

package crds

import (
"context"
"time"

"github.com/aws/amazon-vpc-resource-controller-k8s/apis/vpcresources/v1alpha1"
ec2API "github.com/aws/amazon-vpc-resource-controller-k8s/pkg/aws/ec2/api"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/aws/ec2/api/cleanup"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/config"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/k8s"
"github.com/aws/amazon-vpc-resource-controller-k8s/pkg/utils"
"github.com/go-logr/logr"
"github.com/prometheus/client_golang/prometheus"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/client-go/util/retry"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller"
"sigs.k8s.io/controller-runtime/pkg/metrics"
)

var (
prometheusRegistered = false
recreateCNINodeCallCount = prometheus.NewCounter(
prometheus.CounterOpts{
Name: "recreate_cniNode_call_count",
Help: "The number of requests made by controller to recreate CNINode when node exists",
},
)
recreateCNINodeErrCount = prometheus.NewCounter(
prometheus.CounterOpts{
Name: "recreate_cniNode_err_count",
Help: "The number of requests that failed when controller tried to recreate the CNINode",
},
)
)

func prometheusRegister() {
prometheusRegistered = true

metrics.Registry.MustRegister(
recreateCNINodeCallCount,
recreateCNINodeErrCount)

prometheusRegistered = true
}

// CNINodeReconciler reconciles a CNINode object
type CNINodeReconciler struct {
client.Client
Scheme *runtime.Scheme
Context context.Context
Log logr.Logger
EC2Wrapper ec2API.EC2Wrapper
K8sAPI k8s.K8sWrapper
ClusterName string
VPCID string
FinalizerManager k8s.FinalizerManager
}

//+kubebuilder:rbac:groups=vpcresources.k8s.aws,resources=cninodes,verbs=get;list;watch;create;update;patch;

// Reconcile handles CNINode create/update/delete events
// Reconciler will add the finalizer and cluster name tag if it does not exist and finalize on CNINode on deletion to clean up leaked resource on node
func (r *CNINodeReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
cniNode := &v1alpha1.CNINode{}
if err := r.Client.Get(ctx, req.NamespacedName, cniNode); err != nil {
if errors.IsNotFound(err) {
r.Log.Info("CNINode is deleted", "CNINode", req.NamespacedName)
}
// Ignore not found error
return ctrl.Result{}, client.IgnoreNotFound(err)
}

nodeFound := true
node := &v1.Node{}
if err := r.Client.Get(ctx, req.NamespacedName, node); err != nil {
if errors.IsNotFound(err) {
nodeFound = false
} else {
r.Log.Error(err, "failed to get the node object in CNINode reconciliation, will retry")
// Requeue request so it can be retried
return ctrl.Result{}, err
}
}

if cniNode.GetDeletionTimestamp().IsZero() {
shouldPatch := false
cniNodeCopy := cniNode.DeepCopy()
// Add cluster name tag if it does not exist
val, ok := cniNode.Spec.Tags[config.CNINodeClusterNameKey]
if !ok || val != r.ClusterName {
if len(cniNodeCopy.Spec.Tags) != 0 {
cniNodeCopy.Spec.Tags[config.CNINodeClusterNameKey] = r.ClusterName
} else {
cniNodeCopy.Spec.Tags = map[string]string{
config.CNINodeClusterNameKey: r.ClusterName,
}
}
shouldPatch = true
}
// if node exists, get & add OS label if it does not exist on CNINode
if nodeFound {
nodeLabelOS := node.ObjectMeta.Labels[config.NodeLabelOS]
val, ok = cniNode.ObjectMeta.Labels[config.NodeLabelOS]
if !ok || val != nodeLabelOS {
if len(cniNodeCopy.ObjectMeta.Labels) != 0 {
cniNodeCopy.ObjectMeta.Labels[config.NodeLabelOS] = nodeLabelOS
} else {
cniNodeCopy.ObjectMeta.Labels = map[string]string{
config.NodeLabelOS: nodeLabelOS,
}
}
shouldPatch = true
}
}

if shouldPatch {
r.Log.Info("patching CNINode to add required fields Tags and Labels", "cninode", cniNode.Name)
return ctrl.Result{}, r.Client.Patch(ctx, cniNodeCopy, client.MergeFromWithOptions(cniNode, client.MergeFromWithOptimisticLock{}))
}

// Add finalizer if it does not exist
if err := r.FinalizerManager.AddFinalizers(ctx, cniNode, config.NodeTerminationFinalizer); err != nil {
r.Log.Error(err, "failed to add finalizer on CNINode, will retry", "cniNode", cniNode.Name, "finalizer", config.NodeTerminationFinalizer)
return ctrl.Result{}, err
}
return ctrl.Result{}, nil

} else { // CNINode is marked for deletion
if !nodeFound {
// node is also deleted, proceed with running the cleanup routine and remove the finalizer

// run cleanup for Linux nodes only
if val, ok := cniNode.ObjectMeta.Labels[config.NodeLabelOS]; ok && val == config.OSLinux {
r.Log.Info("running the finalizer routine on cniNode", "cniNode", cniNode.Name)
cleaner := &cleanup.NodeTerminationCleaner{
NodeName: cniNode.Name,
}
cleaner.ENICleaner = &cleanup.ENICleaner{
EC2Wrapper: r.EC2Wrapper,
Manager: cleaner,
VPCID: r.VPCID,
Log: ctrl.Log.WithName("eniCleaner").WithName("node"),
}

if err := cleaner.DeleteLeakedResources(); err != nil {
r.Log.Error(err, "failed to cleanup resources during node termination, request will be requeued")
// Return err if failed to delete leaked ENIs on node so it can be retried
return ctrl.Result{}, err
}
}

if err := r.FinalizerManager.RemoveFinalizers(ctx, cniNode, config.NodeTerminationFinalizer); err != nil {
r.Log.Error(err, "failed to remove finalizer on CNINode, will retry", "cniNode", cniNode.Name, "finalizer", config.NodeTerminationFinalizer)
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
} else {
// node exists, do not run the cleanup routine(periodic cleanup routine will anyway delete leaked ENIs), remove the finalizer
// to proceed with object deletion, and recreate similar object

// Create a copy without deletion timestamp for creation
newCNINode := &v1alpha1.CNINode{
ObjectMeta: metav1.ObjectMeta{
Name: cniNode.Name,
Namespace: "",
OwnerReferences: cniNode.OwnerReferences,
// TODO: should we include finalizers at object creation or let controller patch it on Create/Update event?
Finalizers: cniNode.Finalizers,
},
Spec: cniNode.Spec,
}

if err := r.FinalizerManager.RemoveFinalizers(ctx, cniNode, config.NodeTerminationFinalizer); err != nil {
r.Log.Error(err, "failed to remove finalizer on CNINode on node deletion, will retry")
return ctrl.Result{}, err
}
// wait till CNINode is deleted before recreation as the new object will be created with same name to avoid "object already exists" error
if err := r.waitTillCNINodeDeleted(k8s.NamespacedName(newCNINode)); err != nil {
// raise event if CNINode could not be deleted after removing the finalizer
r.K8sAPI.BroadcastEvent(cniNode, utils.CNINodeDeleteFailed, "CNINode deletion failed and object could not be recreated by the vpc-resource-controller, will retry",
v1.EventTypeWarning)
// requeue here to check if CNINode deletion is successful and retry CNINode deletion if node exists
return ctrl.Result{}, err
}

r.Log.Info("creating CNINode after it has been deleted as node still exists", "cniNode", newCNINode.Name)
recreateCNINodeCallCount.Inc()
if err := r.createCNINodeFromObj(ctx, newCNINode); err != nil {
recreateCNINodeErrCount.Inc()
// raise event on node publish warning that CNINode is deleted and could not be recreated by controller
utils.SendNodeEventWithNodeName(r.K8sAPI, node.Name, utils.CNINodeCreateFailed,
"CNINode was deleted and failed to be recreated by the vpc-resource-controller", v1.EventTypeWarning, r.Log)
// return nil as deleted and we cannot recreate the object now
return ctrl.Result{}, nil
}
r.Log.Info("successfully recreated CNINode", "cniNode", newCNINode.Name)
}
}
return ctrl.Result{}, nil
}

// SetupWithManager sets up the controller with the Manager.
func (r *CNINodeReconciler) SetupWithManager(mgr ctrl.Manager) error {
if !prometheusRegistered {
prometheusRegister()
}
return ctrl.NewControllerManagedBy(mgr).
For(&v1alpha1.CNINode{}).
WithOptions(controller.Options{MaxConcurrentReconciles: config.MaxNodeConcurrentReconciles}).
Complete(r)
}

// waitTillCNINodeDeleted waits for CNINode to be deleted with timeout and returns error
func (r *CNINodeReconciler) waitTillCNINodeDeleted(nameSpacedCNINode types.NamespacedName) error {
oldCNINode := &v1alpha1.CNINode{}

return wait.PollUntilContextTimeout(context.TODO(), 500*time.Millisecond, time.Second*3, true, func(ctx context.Context) (bool, error) {
if err := r.Client.Get(ctx, nameSpacedCNINode, oldCNINode); err != nil && errors.IsNotFound(err) {
return true, nil
}
return false, nil
})
}

// createCNINodeFromObj will create CNINode with backoff and returns error if CNINode is not recreated
func (r *CNINodeReconciler) createCNINodeFromObj(ctx context.Context, newCNINode client.Object) error {
return retry.OnError(retry.DefaultBackoff, func(error) bool { return true },
func() error {
return r.Client.Create(ctx, newCNINode)
})
}

0 comments on commit b5a054e

Please sign in to comment.