Add support for Krane annotations

CHANGELOG.md

@@ -8,6 +8,7 @@
 - We've added a new Krane cli. This code is in alpha. We are providing
 no warranty at this time and reserve the right to make major breaking changes including
 removing it entirely at any time. ([#256](https://github.com/Shopify/kubernetes-deploy/issues/256))
+- Deprecate `kubernetes-deploy.shopify.io` annotations in favour of `krane.shopify.io`.
 
 ## 0.27.0
 *Enhancements*

README.md

@@ -1,3 +1,7 @@
+# krane
+
+As this project approaches the v1.0 milestone, we're excited to announce that `kubernetes-deploy` will be [officially renamed as `krane`](https://github.com/Shopify/kubernetes-deploy/issues/30#issuecomment-468750341). Follow the [1.0 requirement label](https://github.com/Shopify/kubernetes-deploy/issues?q=is%3Aissue+is%3Aopen+label%3A%22%3Arocket%3A+1.0+requirement%22) to keep up with the progress.
+
 # kubernetes-deploy [![Build status](https://badge.buildkite.com/61937e40a1fc69754d9d198be120543d6de310de2ba8d3cb0e.svg?branch=master)](https://buildkite.com/shopify/kubernetes-deploy) [![codecov](https://codecov.io/gh/Shopify/kubernetes-deploy/branch/master/graph/badge.svg)](https://codecov.io/gh/Shopify/kubernetes-deploy)
 
 `kubernetes-deploy` is a command line tool that helps you ship changes to a Kubernetes namespace and understand the result. At Shopify, we use it within our much-beloved, open-source [Shipit](https://github.com/Shopify/shipit-engine#kubernetes) deployment app.
@@ -228,10 +232,10 @@ This is a limitation of the current implementation.
 
 
 ### Customizing behaviour with annotations
-- `kubernetes-deploy.shopify.io/timeout-override`: Override the tool's hard timeout for one specific resource. Both full ISO8601 durations and the time portion of ISO8601 durations are valid. Value must be between 1 second and 24 hours.
+- `krane.shopify.io/timeout-override`: Override the tool's hard timeout for one specific resource. Both full ISO8601 durations and the time portion of ISO8601 durations are valid. Value must be between 1 second and 24 hours.
   - _Example values_: 45s / 3m / 1h / PT0.25H
   - _Compatibility_: all resource types
-- `kubernetes-deploy.shopify.io/required-rollout`: Modifies how much of the rollout needs to finish
+- `krane.shopify.io/required-rollout`: Modifies how much of the rollout needs to finish
 before the deployment is considered successful.
   - _Compatibility_: Deployment
   - `full`: The deployment is successful when all pods in the new `replicaSet` are ready.
@@ -242,11 +246,11 @@ before the deployment is considered successful.
   that use the `RollingUpdate` strategy.
   - Percent (e.g. 90%): The deploy is successful when the number of new pods that are ready is equal to
   `spec.replicas` * Percent.
-- `kubernetes-deploy.shopify.io/prunable`: Allows a Custom Resource to be pruned during deployment.
+- `krane.shopify.io/prunable`: Allows a Custom Resource to be pruned during deployment.
   - _Compatibility_: Custom Resource Definition
   - `true`: The custom resource will be pruned if the resource is not in the deploy directory.
   - All other values: The custom resource will not be pruned.
-- `kubernetes-deploy.shopify.io/predeployed`: Causes a Custom Resource to be deployed in the pre-deploy phase.
+- `krane.shopify.io/predeployed`: Causes a Custom Resource to be deployed in the pre-deploy phase.
   - _Compatibility_: Custom Resource Definition
   - _Default_: `true`
   - `true`: The custom resource will be deployed in the pre-deploy phase.
@@ -335,12 +339,12 @@ This feature is only available on clusters running Kubernetes 1.11+ since it rel
 *Requirements:*
 
 * The custom resource must expose a `status` subresource with an `observedGeneration` field.
-* The `kubernetes-deploy.shopify.io/instance-rollout-conditions` annotation must be present on the CRD that defines the custom resource.
-* (optional) The `kubernetes-deploy.shopify.io/instance-timeout` annotation can be added to the CRD that defines the custom resource to override the global default timeout for all instances of that resource. This annotation can use ISO8601 format or unprefixed ISO8601 time components (e.g. '1H', '60S').
+* The `krane.shopify.io/instance-rollout-conditions` annotation must be present on the CRD that defines the custom resource.
+* (optional) The `krane.shopify.io/instance-timeout` annotation can be added to the CRD that defines the custom resource to override the global default timeout for all instances of that resource. This annotation can use ISO8601 format or unprefixed ISO8601 time components (e.g. '1H', '60S').
 
 #### Specifying pass/fail conditions
 
-The presence of a valid `kubernetes-deploy.shopify.io/instance-rollout-conditions` annotation on a CRD will cause kubernetes-deploy to monitor the rollout of all instances of that custom resource. Its value can either be `"true"` (giving you the defaults described in the next section) or a valid JSON string with the following format:
+The presence of a valid `krane.shopify.io/instance-rollout-conditions` annotation on a CRD will cause kubernetes-deploy to monitor the rollout of all instances of that custom resource. Its value can either be `"true"` (giving you the defaults described in the next section) or a valid JSON string with the following format:
 ```
 '{
   "success_conditions": [
@@ -364,7 +368,7 @@ You **must** ensure that your custom resource controller sets `.status.observedG
 
 #### Example
 
-As an example, the following is the default configuration that will be used if you set `kubernetes-deploy.shopify.io/instance-rollout-conditions: "true"` on the CRD that defines the custom resources you wish to monitor:
+As an example, the following is the default configuration that will be used if you set `krane.shopify.io/instance-rollout-conditions: "true"` on the CRD that defines the custom resources you wish to monitor:
 
 ```
 '{

lib/kubernetes-deploy/deploy_task.rb

@@ -255,6 +255,11 @@ def validate_resources(resources)
       KubernetesDeploy::Concurrency.split_across_threads(resources) do |r|
         r.validate_definition(kubectl, selector: @selector)
       end
+
+      resources.select(&:has_warnings?).each do |resource|
+        record_warnings(warning: resource.validation_warning_msg, filename: File.basename(resource.file_path))
+      end
+
       failed_resources = resources.select(&:validation_failed?)
       return unless failed_resources.present?
 
@@ -319,6 +324,12 @@ def record_invalid_template(err:, filename:, content: nil)
       @logger.summary.add_paragraph(debug_msg)
     end
 
+    def record_warnings(warning:, filename:)
+      warn_msg = "Template warning: #{filename}\n"
+      warn_msg += "> Warning message:\n#{FormattedLogger.indent_four(warning)}"
+      @logger.summary.add_paragraph(ColorizedString.new(warn_msg).yellow)
+    end
+
     def validate_configuration(allow_protected_ns:, prune:)
       errors = []
       errors += kubeclient_builder.validate_config_files

lib/kubernetes-deploy/kubernetes_resource.rb

@@ -30,7 +30,9 @@ class KubernetesResource
       If you have reason to believe it will succeed, retry the deploy to continue to monitor the rollout.
       MSG
 
-    TIMEOUT_OVERRIDE_ANNOTATION = "kubernetes-deploy.shopify.io/timeout-override"
+    TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX = "timeout-override"
+    TIMEOUT_OVERRIDE_ANNOTATION_DEPRECATED = "kubernetes-deploy.shopify.io/#{TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX}"
+    TIMEOUT_OVERRIDE_ANNOTATION = "krane.shopify.io/#{TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX}"
     LAST_APPLIED_ANNOTATION = "kubectl.kubernetes.io/last-applied-configuration"
     SENSITIVE_TEMPLATE_CONTENT = false
 
@@ -93,7 +95,8 @@ def timeout
 
     def timeout_override
       return @timeout_override if defined?(@timeout_override)
-      @timeout_override = DurationParser.new(timeout_annotation).parse!.to_i
+
+      @timeout_override = DurationParser.new(krane_annotation_value(TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX)).parse!.to_i
     rescue DurationParser::ParsingError
       @timeout_override = nil
     end
@@ -113,6 +116,7 @@ def initialize(namespace:, context:, definition:, logger:, statsd_tags: [])
       @statsd_report_done = false
       @disappeared = false
       @validation_errors = []
+      @validation_warnings = []
       @instance_data = {}
     end
 
@@ -122,12 +126,22 @@ def to_kubeclient_resource
 
     def validate_definition(kubectl, selector: nil)
       @validation_errors = []
+      @validation_warnings = []
       validate_selector(selector) if selector
       validate_timeout_annotation
+      validate_annotation_version
       validate_spec_with_kubectl(kubectl)
       @validation_errors.present?
     end
 
+    def validation_warning_msg
+      @validation_warnings.join("\n")
+    end
+
+    def has_warnings?
+      @validation_warnings.present?
+    end
+
     def validation_error_msg
       @validation_errors.join("\n")
     end
@@ -396,20 +410,44 @@ def global?
     private
 
     def validate_timeout_annotation
-      return if timeout_annotation.nil?
+      timeout_override_value = krane_annotation_value(TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX)
+      timeout_annotation_key = krane_annotation_key(TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX)
+      return if timeout_override_value.nil?
 
-      override = DurationParser.new(timeout_annotation).parse!
+      override = DurationParser.new(timeout_override_value).parse!
       if override <= 0
-        @validation_errors << "#{TIMEOUT_OVERRIDE_ANNOTATION} annotation is invalid: Value must be greater than 0"
+        @validation_errors << "#{timeout_annotation_key} annotation is invalid: Value must be greater than 0"
       elsif override > 24.hours
-        @validation_errors << "#{TIMEOUT_OVERRIDE_ANNOTATION} annotation is invalid: Value must be less than 24h"
+        @validation_errors << "#{timeout_annotation_key} annotation is invalid: Value must be less than 24h"
       end
     rescue DurationParser::ParsingError => e
-      @validation_errors << "#{TIMEOUT_OVERRIDE_ANNOTATION} annotation is invalid: #{e}"
+      @validation_errors << "#{timeout_annotation_key} annotation is invalid: #{e}"
+    end
+
+    def validate_annotation_version
+      return if validation_warning_msg.include?("annotations is deprecated")
+      annotation_keys = @definition.dig("metadata", "annotations")&.keys
+      annotation_keys&.each do |annotation|
+        if annotation.include?("kubernetes-deploy.shopify.io")
+          annotation_prefix = annotation.split('/').first
+          @validation_warnings << "#{annotation_prefix} as a prefix for annotations is deprecated: "\
+            "Use the 'krane.shopify.io' annotation prefix instead"
+          return
+        end
+      end
     end
 
-    def timeout_annotation
-      @definition.dig("metadata", "annotations", TIMEOUT_OVERRIDE_ANNOTATION)
+    def krane_annotation_value(suffix)
+      @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/#{suffix}") ||
+        @definition.dig("metadata", "annotations", "krane.shopify.io/#{suffix}")
+    end
+
+    def krane_annotation_key(suffix)
+      if @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/#{suffix}")
+        "kubernetes-deploy.shopify.io/#{suffix}"
+      elsif @definition.dig("metadata", "annotations", "krane.shopify.io/#{suffix}")
+        "krane.shopify.io/#{suffix}"
+      end
     end
 
     def validate_selector(selector)

lib/kubernetes-deploy/kubernetes_resource/custom_resource_definition.rb

@@ -2,8 +2,9 @@
 module KubernetesDeploy
   class CustomResourceDefinition < KubernetesResource
     TIMEOUT = 2.minutes
-    ROLLOUT_CONDITIONS_ANNOTATION = "kubernetes-deploy.shopify.io/instance-rollout-conditions"
-    TIMEOUT_FOR_INSTANCE_ANNOTATION = "kubernetes-deploy.shopify.io/instance-timeout"
+    ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX = "instance-rollout-conditions"
+    ROLLOUT_CONDITIONS_ANNOTATION = "krane.shopify.io/#{ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX}"
+    TIMEOUT_FOR_INSTANCE_ANNOTATION = "krane.shopify.io/instance-timeout"
     GLOBAL = true
 
     def deploy_succeeded?
@@ -19,7 +20,7 @@ def timeout_message
     end
 
     def timeout_for_instance
-      timeout = @definition.dig("metadata", "annotations", TIMEOUT_FOR_INSTANCE_ANNOTATION)
+      timeout = krane_annotation_value("instance-timeout")
       DurationParser.new(timeout).parse!.to_i
     rescue DurationParser::ParsingError
       nil
@@ -50,20 +51,20 @@ def name
     end
 
     def prunable?
-      prunable = @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/prunable")
+      prunable = krane_annotation_value("prunable")
       prunable == "true"
     end
 
     def predeployed?
-      predeployed = @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/predeployed")
+      predeployed = krane_annotation_value("predeployed")
       predeployed.nil? || predeployed == "true"
     end
 
     def rollout_conditions
       return @rollout_conditions if defined?(@rollout_conditions)
 
-      @rollout_conditions = if rollout_conditions_annotation
-        RolloutConditions.from_annotation(rollout_conditions_annotation)
+      @rollout_conditions = if krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX)
+        RolloutConditions.from_annotation(krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX))
       end
     rescue RolloutConditionsError
       @rollout_conditions = nil
@@ -74,12 +75,13 @@ def validate_definition(*)
 
       validate_rollout_conditions
     rescue RolloutConditionsError => e
-      @validation_errors << "Annotation #{ROLLOUT_CONDITIONS_ANNOTATION} on #{name} is invalid: #{e}"
+      @validation_errors << "Annotation #{krane_annotation_key(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX)} "\
+        "on #{name} is invalid: #{e}"
     end
 
     def validate_rollout_conditions
-      if rollout_conditions_annotation && @rollout_conditions_validated.nil?
-        conditions = RolloutConditions.from_annotation(rollout_conditions_annotation)
+      if krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX) && @rollout_conditions_validated.nil?
+        conditions = RolloutConditions.from_annotation(krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX))
         conditions.validate!
       end
 
@@ -96,9 +98,5 @@ def names_accepted_condition
     def names_accepted_status
       names_accepted_condition["status"]
     end
-
-    def rollout_conditions_annotation
-      @definition.dig("metadata", "annotations", ROLLOUT_CONDITIONS_ANNOTATION)
-    end
   end
 end

lib/kubernetes-deploy/kubernetes_resource/deployment.rb

@@ -4,7 +4,9 @@
 module KubernetesDeploy
   class Deployment < KubernetesResource
     TIMEOUT = 7.minutes
-    REQUIRED_ROLLOUT_ANNOTATION = 'kubernetes-deploy.shopify.io/required-rollout'
+    REQUIRED_ROLLOUT_ANNOTATION_SUFFIX = "required-rollout"
+    REQUIRED_ROLLOUT_ANNOTATION_DEPRECATED = "kubernetes-deploy.shopify.io/#{REQUIRED_ROLLOUT_ANNOTATION_SUFFIX}"
+    REQUIRED_ROLLOUT_ANNOTATION = "krane.shopify.io/#{REQUIRED_ROLLOUT_ANNOTATION_SUFFIX}"
     REQUIRED_ROLLOUT_TYPES = %w(maxUnavailable full none).freeze
     DEFAULT_REQUIRED_ROLLOUT = 'full'
 
@@ -103,8 +105,8 @@ def validate_definition(*)
 
       strategy = @definition.dig('spec', 'strategy', 'type').to_s
       if required_rollout.downcase == 'maxunavailable' && strategy.present? && strategy.downcase != 'rollingupdate'
-        @validation_errors << "'#{REQUIRED_ROLLOUT_ANNOTATION}: #{required_rollout}' is incompatible "\
-        "with strategy '#{strategy}'"
+        @validation_errors << "'#{krane_annotation_key(REQUIRED_ROLLOUT_ANNOTATION_SUFFIX)}: #{required_rollout}' "\
+          "is incompatible with strategy '#{strategy}'"
       end
 
       @validation_errors.empty?
@@ -148,7 +150,7 @@ def progress_deadline
     end
 
     def rollout_annotation_err_msg
-      "'#{REQUIRED_ROLLOUT_ANNOTATION}: #{required_rollout}' is invalid. "\
+      "'#{krane_annotation_key(REQUIRED_ROLLOUT_ANNOTATION_SUFFIX)}: #{required_rollout}' is invalid. "\
         "Acceptable values: #{REQUIRED_ROLLOUT_TYPES.join(', ')}"
     end
 
@@ -201,7 +203,7 @@ def max_unavailable
     end
 
     def required_rollout
-      @definition.dig('metadata', 'annotations', REQUIRED_ROLLOUT_ANNOTATION).presence || DEFAULT_REQUIRED_ROLLOUT
+      krane_annotation_value("required-rollout") || DEFAULT_REQUIRED_ROLLOUT
     end
 
     def percent?(value)

test/fixtures/crd/widgets.yml

@@ -3,7 +3,7 @@ kind: CustomResourceDefinition
 metadata:
   name: widgets.stable.example.io
   annotations:
-    kubernetes-deploy.shopify.io/prunable: "true"
+    krane.shopify.io/prunable: "true"
 spec:
   group: stable.example.io
   version: v1

test/fixtures/crd/widgets_deprecated.yml

@@ -0,0 +1,13 @@
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: widgets.stable.example.io
+  annotations:
+    kubernetes-deploy.shopify.io/prunable: "true"
+spec:
+  group: stable.example.io
+  version: v1
+  names:
+    kind: Widget
+    plural: widgets
+  scope: Namespaced

test/fixtures/crd/with_custom_conditions.yml

@@ -3,7 +3,7 @@ kind: CustomResourceDefinition
 metadata:
   name: customizeds.stable.example.io
   annotations:
-    kubernetes-deploy.shopify.io/instance-rollout-conditions: '{
+    krane.shopify.io/instance-rollout-conditions: '{
       "success_conditions": [
         {
           "path": "$.status.condition",

test/fixtures/crd/with_default_conditions.yml

@@ -3,7 +3,7 @@ kind: CustomResourceDefinition
 metadata:
   name: parameterizeds.stable.example.io
   annotations:
-    kubernetes-deploy.shopify.io/instance-rollout-conditions: "true"
+    krane.shopify.io/instance-rollout-conditions: "true"
 spec:
   group: stable.example.io
   names:

test/fixtures/crd/with_default_conditions_deprecated.yml

@@ -0,0 +1,15 @@
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: parameterizeds.stable.example.io
+  annotations:
+    kubernetes-deploy.shopify.io/instance-rollout-conditions: "true"
+spec:
+  group: stable.example.io
+  names:
+    kind: Parameterized
+    listKind: ParameterizedList
+    plural: parameterizeds
+    singular: parameterized
+  scope: Namespaced
+  version: v1

test/fixtures/hello-cloud/unmanaged-pod-1.yml.erb

@@ -3,7 +3,7 @@ kind: Pod
 metadata:
   name: unmanaged-pod-1-<%= deployment_id %>
   annotations:
-    kubernetes-deploy.shopify.io/timeout-override: 60s
+    krane.shopify.io/timeout-override: 60s
   labels:
     type: unmanaged-pod
     name: unmanaged-pod-1-<%= deployment_id %>

test/fixtures/hello-cloud/unmanaged-pod-2.yml.erb

@@ -3,7 +3,7 @@ kind: Pod
 metadata:
   name: unmanaged-pod-2-<%= deployment_id %>
   annotations:
-    kubernetes-deploy.shopify.io/timeout-override: "60s"
+    krane.shopify.io/timeout-override: "60s"
   labels:
     type: unmanaged-pod
     name: unmanaged-pod-2-<%= deployment_id %>

test/fixtures/pvc/pod.yml

@@ -2,7 +2,7 @@ apiVersion: v1
 kind: Pod
 metadata:
   annotations:
-    kubernetes-deploy.shopify.io/timeout-override: 10s
+    krane.shopify.io/timeout-override: 10s
   name: pvc
   labels:
     type: unmanaged-pod