Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-1.5] feat: external load balancer garbage collection (part 2) - new gc service #3632

Merged
merged 3 commits into from
Aug 3, 2022
Merged

[release-1.5] feat: external load balancer garbage collection (part 2) - new gc service #3632

merged 3 commits into from
Aug 3, 2022

Commits on Aug 1, 2022

  1. feat: new garbage collection servive 

    This introduces a new garbage collection service which will be used later
by the controllers to do clean up of AWS resources for child/tenant
clusters that where created via the CCM. Initially we support clearing
up load balancers (classic elb, nlb, alb) and security groups.

It works in 2 phases depending on if a child cluster is being
created/update (i.e. **Reconcile**) or deleted (i.e.
**ReconcileDelete**).

When a cluster is created the cluster controllers will call **Reconcile**. Its
purpose is to deteremine if the cluster should be garbage collected and
if it should then it needs marked. If the gc feature is enabled we will
operate an opt-out model, so all clusters will be garbage collected
unless they exlicitly opt out. To opt out the infra cluster must have
the `aws.cluster.x-k8s.io/external-resource-gc` annotation with a value
of false. Otherwise a cluster is marked as requiring gc by adding the
`awsexternalresourcegc.infrastructure.cluster.x-k8s.io` finalizer to the
infra cluster.

When a cluster is deleted the cluster controllers will call
**ReconcileDelete**. The first job is to identify the AWS resources that
where created by the CCM in the child/tenant cluster. This is done by
using the AWS resource tagging api and getting resources with the
kubernetes cluster owned label. The resources that are returned are then
put into buckets for each AWS resource type (i.e. ec2,
elasticloadbalancing) and then these buckets of resource ARNs are passed
to a function for that specific AWS service which will do teh actual API
calls to clear up the AWS resources. The reason we use buckets is that
the order in which you delete services can matter, for example load
balancers must be deleted before target groups.

The service will be use by a later change to the controllers.

Signed-off-by: Richard Case <richard.case@outlook.com>
    @richardcase
    richardcase committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    a2aa639 View commit details
    Browse the repository at this point in the history

  2. refactor: gc opt in/out any time before delete 

    Changed the gc service based on the lateset proposal update & review
feedback. It now uses a model where "the user decides whether
to enable the gc at any time before they delete the cluster.".

This translates into the `ReconcileDelete` now:
- Checks to see ig the cluster should be garbage collected by looking at
  the gc annotation (if it exists).
- Does resource cleanup

Signed-off-by: Richard Case <richard.case@outlook.com>
    @richardcase
    richardcase committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    587b8bc View commit details
    Browse the repository at this point in the history

  3. refactor: cleanup functions filter resource 

    The the cleanup functions now accepted the full list of AWS resources
and then they filter which resources they want to delete themselves.

Signed-off-by: Richard Case <richard.case@outlook.com>
    @richardcase
    richardcase committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    a65ccb2 View commit details
    Browse the repository at this point in the history