Add option select-any for deployTask and globalDeployTask #831

peiranliushop · 2021-06-04T19:20:23Z

Related PR: https://github.com/Shopify/gkrane-config/pull/45

What are you trying to accomplish with this PR?
Currently option --selector for sub command krane deploy and krane global-deploy will cause validation failure if any k8s resources in templates (--filenames) does not match the label selectors. This will cause problems when we want to deploy a subset of k8s resources.

How is this accomplished?
Adding a new option selector-as-filter in boolean with default value of false. If set to true, selector validation will be skipped. Only resources with matching labels will be deployed and be prunable.
This allows us to deploy none or a subset of resources. It will help us to address canary and production deploy where the 2 sets of resources share the same namespace and same templates file directory.

What could go wrong?
The new option has default value of false, which will not change current selector behaviour. Therefore it should be backward compatible. This is not a breaking change.

krane help deploy                                                                                                                                                                                                   
Usage:
  krane deploy NAMESPACE CONTEXT

Options:
  f, [--filenames=config/deploy/production config/deploy/my-extra-resource.yml]  # Directories and files that contains the configuration to apply
      [--stdin], [--no-stdin]                                                    # [DEPRECATED] Read resources from stdin
      [--global-timeout=duration]                                                # Max duration to monitor workloads correctly deployed
                                                                                 # Default: 300s
      [--protected-namespaces=namespace1 namespace2 namespaceN]                  # Enable deploys to a list of selected namespaces; set to an empty string to disable
                                                                                 # Default: ["default", "kube-system", "kube-public"]
      [--prune], [--no-prune]                                                    # Enable deletion of resources that do not appear in the template dir
                                                                                 # Default: true
      [--selector='label=value']                                                 # Select workloads by selector(s)
      [--selector-as-filter], [--no-selector-as-filter]                          # Use --selector as a label filter to select a subset of resources to deploy
      [--verbose-log-prefix], [--no-verbose-log-prefix]                          # Add [context][namespace] to the log prefix
      [--verify-result], [--no-verify-result]                                    # Verify workloads correctly deployed
                                                                                 # Default: true

krane help global-deploy                                                                                                                                                                                             
Usage:
  krane global-deploy CONTEXT --selector='label=value'

Options:
  f, [--filenames=config/deploy/production config/deploy/my-extra-resource.yml]  # Directories and files that contains the configuration to apply
      [--stdin], [--no-stdin]                                                    # [DEPRECATED] Read resources from stdin
      [--global-timeout=duration]                                                # Max duration to monitor workloads correctly deployed
                                                                                 # Default: 300s
      [--verify-result], [--no-verify-result]                                    # Verify workloads correctly deployed
                                                                                 # Default: true
      --selector='label=value'                                                   # Select workloads owned by selector(s)
      [--selector-as-filter], [--no-selector-as-filter]                          # Use --selector as a label filter to select a subset of resources to deploy
      [--prune], [--no-prune]                                                    # Enable deletion of resources that match the provided selector and do not appear in the provided templates

timothysmith0609 · 2021-06-08T17:32:03Z

As discussed in Slack, perhaps a more appropriate name for this flag is selector-as-filter or something equally explicit that denotes the fact we are selecting a subset of the passed-in resources.

If this indeed is the most straightforward solution then I won't block it, but I strongly urge you to consider other ways in which we can accomplish this goal (e.g. filtering the resources before passing them to krane). I find it a bit unpalatable to have an option that discards some of the resources that are explicitly passed in by the user (via the -f / --filename flag). It is vastly more preferable to pass in only those files desired for deploy, rather than filter post hoc. I feel this also blurs the line on what the selector flag actually does. Contrary to what one might assume (via kubectl), in krane, the selector flag allows targeted pruning of resources, which is useful in the case where a namespace is shared between deploy stacks. Changing the semantics to suggest actual selection of resources, while a natural evolution, is somewhat orthogonal to this aim.

Again, I won't block if it's decided that this is indeed the best way forward, but it's not my first choice.

cc @karanthukral interested to know your thoughts as one of the other krane maintainers

karanthukral · 2021-06-08T19:00:15Z

After some discussion on slack, I understand the concerns around adding this as a feature to Krane but at the same time think its a valid trade off. I agree with the more explicit naming approach and setting the new feature to false by default which preserves the existing behaviour.

The deploy tooling internally is built to not care about what it's actually deploying and we don't have an appropriate area to add the extra logic prior to templates being passed to Krane. Due to this I think despite this not being the most ideal approach, it's a valid tradeoff given the requirements and constraints the team is under.

timothysmith0609

A few small nits, overall the code looks fine. We should update the README to be a bit more explicit about this feature, since it's a bit of a special case whose use isn't immediately clear. In particular, we should be crystal clear about its intent (see readme suggestions). We should also explain that it preserves the pruning behaviour of selector as well.

timothysmith0609 · 2021-06-08T19:45:38Z

README.md

@@ -117,6 +117,7 @@ Refer to `krane help` for the authoritative set of options.
 - `--global-timeout=duration`: Raise a timeout error if it takes longer than _duration_ for any
 resource to deploy.
 - `--selector`: Instructs krane to only prune resources which match the specified label selector, such as `environment=staging`. If you use this option, all resource templates must specify matching labels. See [Sharing a namespace](#sharing-a-namespace) below.
+- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels.


nit/suggestion:

Suggested change

- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels.

- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels. This is useful if you only want to deploy a subset of resources within a given YAML file.

timothysmith0609 · 2021-06-08T19:46:35Z

README.md

@@ -441,6 +442,7 @@ Refer to `krane global-deploy help` for the authoritative set of options.
 - `--filenames / -f [PATHS]`: Accepts a list of directories and/or filenames to specify the set of directories/files that will be deployed. Use `-` to specify STDIN.
 - `--no-prune`: Skips pruning of resources that are no longer in your Kubernetes template set. Not recommended, as it allows your namespace to accumulate cruft that is not reflected in your deploy directory.
 - `--selector`: Instructs krane to only prune resources which match the specified label selector, such as `environment=staging`. By using this option, all resource templates must specify matching labels. See [Sharing a namespace](#sharing-a-namespace) below.
+- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels.


nit/suggestion:

Suggested change

- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels.

- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels. This is useful if you only want to deploy a subset of resources within a given YAML file.

timothysmith0609 · 2021-06-08T19:47:20Z

lib/krane/cli/deploy_command.rb

@@ -25,6 +25,9 @@ class DeployCommand
                     default: true },
        "selector" => { type: :string, banner: "'label=value'",
                        desc: "Select workloads by selector(s)" },
+        "selector-as-filter" => { type: :boolean,
+                                  desc: "Use --selector as a label filter to select a subset of resources to deploy",


Suggested change

desc: "Use --selector as a label filter to select a subset of resources to deploy",

desc: "Use --selector as a label filter to deploy only a subset of the provided resources to deploy",

timothysmith0609 · 2021-06-08T19:48:23Z

lib/krane/cli/global_deploy_command.rb

@@ -16,6 +16,9 @@ class GlobalDeployCommand
                             desc: "Verify workloads correctly deployed" },
        "selector" => { type: :string, banner: "'label=value'", required: true,
                        desc: "Select workloads owned by selector(s)" },
+        "selector-as-filter" => { type: :boolean,
+                                  desc: "Use --selector as a label filter to select a subset of resources to deploy",


Suggested change

desc: "Use --selector as a label filter to select a subset of resources to deploy",

desc: "Use --selector as a label filter to deploy only a subset of the provided resources to deploy",

A few small nits, overall the code looks fine. We should update the README to be a bit more explicit about this feature, since it's a bit of a special case whose use isn't immediately clear. In particular, we should be crystal clear about its intent (see readme suggestions). We should also explain that it preserves the pruning behaviour of selector as well.

Added some explanation in README Sharing Namespace.

timothysmith0609 · 2021-06-08T19:51:19Z

test/integration/krane_deploy_test.rb

@@ -166,6 +166,40 @@ def test_selector
    assert_equal("master", deployments.first.metadata.labels.branch)
  end

+  def test_select_any
+    # Deploy only the resource matching the selector without valiation error


nit: valiation -> validation

lib/krane/cli/deploy_command.rb

timothysmith0609

Thanks for updating the docs. I have a few more minor comments but nothing too major.

timothysmith0609 · 2021-06-08T19:58:03Z

lib/krane/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module Krane
-  VERSION = "2.1.10"
+  VERSION = "2.1.11"


Usually I make a dedicated branch when bumping the version, so I can tie up any loose ends that are around just in case. For now, let's put this back to 2.1.10 and I can cut a release when this merges

ok, I reverted the version here. Not sure if you want me to keep the update in CHANGELOG.md.

The CHANGELOG entry is fine (and appreciated!)

lib/krane/cli/deploy_command.rb

timothysmith0609 · 2021-06-09T12:30:45Z

lib/krane/global_deploy_task.rb

@@ -143,6 +146,8 @@ def validate_resources(resources)
        end
        raise FatalDeploymentError, "Template validation failed"
      end
+
+      resources.select! { |r| r.selected?(@selector) }


Any reason we can't move this up before we call validate_definition on the resources? That way validate_definition doesn't need to worry about whether or not selector_as_filter is set.

e.g.

resources.select! { |r| r.selected?(@selector) } if @selector_as_filter Concurrency.split_across_threads(resources) do |r| r.validate_definition(kubectl: @kubectl, selector: @selector) end

This would be nicer because otherwise we are leaking the filtering concern into kubernetes_resource.

timothysmith0609

One last nit, but I think this is good

timothysmith0609 · 2021-06-09T13:50:52Z

lib/krane/deploy_task.rb

@@ -273,6 +275,7 @@ def validate_configuration(prune:)

      confirm_ejson_keys_not_prunable if prune
      @logger.info("Using resource selector #{@selector}") if @selector
+      @logger.info("Can deploy a subset of resources") if @selector && @selector_as_filter


Perhaps something a bit more explicit:

Suggested change

@logger.info("Can deploy a subset of resources") if @selector && @selector_as_filter

@logger.info("Only deploying resources that match filter: #{@selector_as_filter}") if @selector && @selector_as_filter

peiranliushop requested a review from a team as a code owner June 4, 2021 19:20

peiranliushop requested a review from timothysmith0609 June 4, 2021 21:38

timothysmith0609 requested a review from karanthukral June 8, 2021 17:32

timothysmith0609 reviewed Jun 8, 2021

View reviewed changes

lib/krane/cli/deploy_command.rb Show resolved Hide resolved

peiranliushop added 4 commits June 8, 2021 20:36

add option select-any for deploy task and global deploy task

9866c1c

bump version

87be32c

rename the option

f5d64f4

update

1a3371d

peiranliushop force-pushed the select-any branch from 52beee3 to 1a3371d Compare June 9, 2021 00:37

test

848d714

timothysmith0609 reviewed Jun 9, 2021

View reviewed changes

back version to 2.1.10

01ce0fa

timothysmith0609 approved these changes Jun 9, 2021

View reviewed changes

update log

af83245

peiranliushop merged commit 41bc7e5 into master Jun 9, 2021

shopify-shipit bot temporarily deployed to rubygems June 9, 2021 19:07 Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add option select-any for deployTask and globalDeployTask #831

Add option select-any for deployTask and globalDeployTask #831

peiranliushop commented Jun 4, 2021 •

edited

Loading

timothysmith0609 commented Jun 8, 2021 •

edited

Loading

karanthukral commented Jun 8, 2021

timothysmith0609 left a comment

timothysmith0609 Jun 8, 2021

timothysmith0609 Jun 8, 2021

timothysmith0609 Jun 8, 2021

timothysmith0609 Jun 8, 2021

peiranliushop Jun 9, 2021

timothysmith0609 Jun 8, 2021

timothysmith0609 left a comment

timothysmith0609 Jun 8, 2021

peiranliushop Jun 9, 2021

timothysmith0609 Jun 9, 2021

timothysmith0609 Jun 9, 2021

timothysmith0609 left a comment

timothysmith0609 Jun 9, 2021

	- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels.
	- `--selector-as-filter`: Instructs krane to only deploy resources that are filtered by the specified labels in `--selector`. The deploy will not fail if not all resources match the labels. This is useful if you only want to deploy a subset of resources within a given YAML file.

	desc: "Use --selector as a label filter to select a subset of resources to deploy",
	desc: "Use --selector as a label filter to deploy only a subset of the provided resources to deploy",

	@logger.info("Can deploy a subset of resources") if @selector && @selector_as_filter
	@logger.info("Only deploying resources that match filter: #{@selector_as_filter}") if @selector && @selector_as_filter

Add option select-any for deployTask and globalDeployTask #831

Add option select-any for deployTask and globalDeployTask #831

Conversation

peiranliushop commented Jun 4, 2021 • edited Loading

timothysmith0609 commented Jun 8, 2021 • edited Loading

karanthukral commented Jun 8, 2021

timothysmith0609 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timothysmith0609 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timothysmith0609 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peiranliushop commented Jun 4, 2021 •

edited

Loading

timothysmith0609 commented Jun 8, 2021 •

edited

Loading