Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.13: [Bug]: build flag of deploy action has no effect #4506

Closed
stefreak opened this issue Jun 1, 2023 · 3 comments · Fixed by #4516 or #4846
Closed

0.13: [Bug]: build flag of deploy action has no effect #4506

stefreak opened this issue Jun 1, 2023 · 3 comments · Fixed by #4516 or #4846
Assignees

Comments

@stefreak
Copy link
Member

stefreak commented Jun 1, 2023

Garden Bonsai (0.13) Bug

Current Behavior

When running garden deploy

✖ deploy.deploy        → Failed resolving status for Deploy type=kubernetes name=deploy (took 0.03 sec). Here is the output:

ENOENT: no such file or directory, open '/Users/steffen/repro/generated-manifest.yml'
  • Build action has not been triggered (directory .garden/build/generate/ does not exist)
  • It looks for generated-manifest.yml in the wrong directory

When running garden build it actually generates the file

✔ build.generate       → Done (took 0 sec)

Done! ✔️
% ls .garden/build/generate/
generated-manifest.yml

Expected behavior

  1. it should execute the build action when running garden deploy, as it is listed in dependencies
  2. Successful deployment of the generated manifest

Reproducible example

kind: Project
apiVersion: garden.io/v1
name: repro
environments:
 - name: default
providers:
 - name: local-kubernetes
---
kind: Build
type: exec
name: generate
spec:
  shell: true
  command: ["echo -n 'apiVersion: v1\nkind: ConfigMapList' > generated-manifest.yml"]

---
kind: Deploy
type: kubernetes
name: deploy
dependencies:
 - build.generate
build: generate
spec:
  files:
    - generated-manifest.yml

Workaround

Unknown

Suggested solution(s)

Use the original solution (a5f5092) from #4516 and make sure there is no regression like we had in #4811. Here is the patch with the original changes:

Index: core/src/plugins/kubernetes/status/status.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/core/src/plugins/kubernetes/status/status.ts b/core/src/plugins/kubernetes/status/status.ts
--- a/core/src/plugins/kubernetes/status/status.ts	(revision 409899599c81409852f08b4de1e4f44acce1e84f)
+++ b/core/src/plugins/kubernetes/status/status.ts	(date 1689325155940)
@@ -12,31 +12,36 @@
 import { PluginContext } from "../../../plugin-context"
 import { KubeApi } from "../api"
 import { getAppNamespace } from "../namespace"
-import { KubernetesResource, KubernetesServerResource, BaseResource, KubernetesWorkload } from "../types"
-import { zip, isArray, isPlainObject, pickBy, mapValues, flatten, cloneDeep, omit, isEqual, keyBy } from "lodash"
-import { KubernetesProvider, KubernetesPluginContext } from "../config"
+import {
+  BaseResource,
+  KubernetesResource,
+  KubernetesServerResource,
+  KubernetesWorkload,
+  SyncableResource,
+} from "../types"
+import { cloneDeep, flatten, isArray, isEqual, isPlainObject, keyBy, mapValues, omit, pickBy } from "lodash"
+import { KubernetesPluginContext, KubernetesProvider } from "../config"
 import { isSubset } from "../../../util/is-subset"
 import { Log } from "../../../logger/log-entry"
 import {
-  V1ReplicationController,
-  V1ReplicaSet,
-  V1Pod,
-  V1PersistentVolumeClaim,
-  V1Service,
-  V1Container,
   KubernetesObject,
+  V1Container,
   V1Job,
+  V1PersistentVolumeClaim,
+  V1Pod,
+  V1ReplicaSet,
+  V1ReplicationController,
+  V1Service,
 } from "@kubernetes/client-node"
-import dedent = require("dedent")
 import { getPods, getResourceKey, hashManifest } from "../util"
 import { checkWorkloadStatus } from "./workload"
 import { checkWorkloadPodStatus } from "./pod"
 import { deline, gardenAnnotationKey, stableStringify } from "../../../util/string"
-import { SyncableResource } from "../types"
 import { ActionMode } from "../../../actions/types"
 import { deepMap } from "../../../util/objects"
-import { DeployState, combineStates } from "../../../types/service"
+import { combineStates, DeployState } from "../../../types/service"
 import { isTruthy, sleep } from "../../../util/util"
+import dedent = require("dedent")
 
 export interface ResourceStatus<T extends BaseResource | KubernetesObject = BaseResource> {
   state: DeployState
@@ -174,18 +179,14 @@
   log: Log
   waitForJobs?: boolean
 }) {
-  const handler = objHandlers[manifest.kind]
-
   if (manifest.metadata?.namespace) {
     namespace = manifest.metadata.namespace
   }
 
   let resource: KubernetesServerResource
-  let resourceVersion: number | undefined
 
   try {
     resource = await api.readBySpec({ namespace, manifest, log })
-    resourceVersion = parseInt(resource.metadata.resourceVersion!, 10)
   } catch (err) {
     if (err.statusCode === 404) {
       return { state: <DeployState>"missing", resource: manifest }
@@ -194,15 +195,41 @@
     }
   }
 
-  let status: ResourceStatus
+  return resolveResourceStatus({ api, namespace, resource, log, waitForJobs })
+}
+
+export async function resolveResourceStatus(
+  params: Omit<StatusHandlerParams, "resourceVersion">
+): Promise<ResourceStatus> {
+  const handler = objHandlers[params.resource.kind]
+
   if (handler) {
-    status = await handler({ api, namespace, resource, log, resourceVersion, waitForJobs })
+    const resourceVersion = parseInt(params.resource.metadata.resourceVersion!, 10)
+    return handler({ ...params, resourceVersion })
   } else {
     // if there is no explicit handler to check the status, we assume there's no rollout phase to wait for
-    status = { state: "ready", resource: manifest }
+    return { state: "ready", resource: params.resource }
   }
+}
 
-  return status
+export function resolveResourceStatuses(log: Log, statuses: ResourceStatus[]) {
+  const deployedStates = statuses.map((s) => s.state)
+  const state = combineStates(deployedStates)
+
+  if (state !== "ready") {
+    const descriptions = statuses
+      .filter((s) => s.state !== "ready")
+      .map((s) => `${getResourceKey(s.resource)}: "${s.state}"`)
+      .join("\n")
+
+    log.silly(
+      dedent`
+      Resource(s) with non-ready status found in the cluster:
+
+      ${descriptions}` + "\n"
+    )
+  }
+  return state
 }
 
 interface WaitParams {
@@ -370,10 +397,7 @@
   manifests = flatten(manifests.map((r: any) => (r.apiVersion === "v1" && r.kind === "List" ? r.items : [r])))
 
   // Check if any resources are missing from the cluster.
-  const maybeDeployedObjects = await Bluebird.map(manifests, (resource) =>
-    getDeployedResource(ctx, ctx.provider, resource, log)
-  )
-  const deployedResources = <KubernetesResource[]>maybeDeployedObjects.filter((o) => o !== null)
+  const deployedResources = await getDeployedResources({ ctx, log, manifests })
   const manifestsMap = keyBy(manifests, (m) => getResourceKey(m))
   const manifestKeys = Object.keys(manifestsMap)
   const deployedMap = keyBy(deployedResources, (m) => getResourceKey(m))
@@ -405,24 +429,13 @@
   log.debug(`Getting currently deployed resource statuses...`)
 
   const deployedObjectStatuses: ResourceStatus[] = await Bluebird.map(deployedResources, async (resource) =>
-    checkResourceStatus({ api, namespace, manifest: resource, log })
+    resolveResourceStatus({ api, namespace, resource, log })
   )
 
-  const deployedStates = deployedObjectStatuses.map((s) => s.state)
-  if (deployedStates.find((s) => s !== "ready")) {
-    const descriptions = zip(deployedResources, deployedStates)
-      .filter(([_, s]) => s !== "ready")
-      .map(([o, s]) => `${logDescription(o!)}: "${s}"`)
-      .join("\n")
-
-    log.silly(
-      dedent`
-      Resource(s) with non-ready status found in the cluster:
+  const resolvedState = resolveResourceStatuses(log, deployedObjectStatuses)
 
-      ${descriptions}` + "\n"
-    )
-
-    result.state = combineStates(deployedStates)
+  if (resolvedState !== "ready") {
+    result.state = resolvedState
     return result
   }
 
@@ -595,9 +608,6 @@
   }
 }
 
-/**
- * Fetches matching deployed resources from the cluster for the provided array of manifests.
- */
 export async function getDeployedResources<ResourceKind extends KubernetesObject>({
   ctx,
   manifests,
@@ -606,8 +616,8 @@
   ctx: KubernetesPluginContext
   manifests: KubernetesResource<ResourceKind>[]
   log: Log
-}): Promise<KubernetesResource<ResourceKind>[]> {
-  const maybeDeployedObjects = await Bluebird.map(manifests, async (resource) =>
+}) {
+  const maybeDeployedObjects = await Bluebird.map(manifests, (resource) =>
     getDeployedResource(ctx, ctx.provider, resource, log)
   )
   return maybeDeployedObjects.filter(isTruthy)
Index: core/src/plugins/kubernetes/kubernetes-type/handlers.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/core/src/plugins/kubernetes/kubernetes-type/handlers.ts b/core/src/plugins/kubernetes/kubernetes-type/handlers.ts
--- a/core/src/plugins/kubernetes/kubernetes-type/handlers.ts	(revision 409899599c81409852f08b4de1e4f44acce1e84f)
+++ b/core/src/plugins/kubernetes/kubernetes-type/handlers.ts	(date 1689324546625)
@@ -9,7 +9,7 @@
 import Bluebird from "bluebird"
 import { isEmpty, omit, partition, uniq } from "lodash"
 import type { ModuleActionHandlers } from "../../../plugin/plugin"
-import { ServiceStatus } from "../../../types/service"
+import { DeployState, ForwardablePort, ServiceStatus } from "../../../types/service"
 import { gardenAnnotationKey } from "../../../util/string"
 import { KubeApi } from "../api"
 import type { KubernetesPluginContext } from "../config"
@@ -19,16 +19,28 @@
 import { getActionNamespace, getActionNamespaceStatus } from "../namespace"
 import { getForwardablePorts, killPortForwards } from "../port-forward"
 import { getK8sIngresses } from "../status/ingress"
-import { compareDeployedResources, waitForResources } from "../status/status"
+import {
+  getDeployedResource,
+  resolveResourceStatus,
+  resolveResourceStatuses,
+  ResourceStatus,
+  waitForResources,
+} from "../status/status"
 import type { BaseResource, KubernetesResource, KubernetesServerResource, SyncableResource } from "../types"
-import { convertServiceResource, gardenNamespaceAnnotationValue, getManifests } from "./common"
+import {
+  convertServiceResource,
+  gardenNamespaceAnnotationValue,
+  getManifests,
+  getMetadataManifest,
+  parseMetadataResource,
+} from "./common"
 import { configureKubernetesModule, KubernetesModule } from "./module-config"
 import { configureLocalMode, startServiceInLocalMode } from "../local-mode"
 import type { ExecBuildConfig } from "../../exec/build"
 import type { KubernetesActionConfig, KubernetesDeployAction, KubernetesDeployActionConfig } from "./config"
 import type { DeployActionHandler } from "../../../plugin/action-types"
 import type { ActionLog } from "../../../logger/log-entry"
-import type { Resolved } from "../../../actions/types"
+import type { ActionMode, Resolved } from "../../../actions/types"
 import { deployStateToActionState } from "../../../plugin/handlers/Deploy/get-status"
 
 export const kubernetesHandlers: Partial<ModuleActionHandlers<KubernetesModule>> = {
@@ -169,42 +181,69 @@
     provider,
     skipCreate: true,
   })
-  const namespace = namespaceStatus.namespaceName
+  const defaultNamespace = namespaceStatus.namespaceName
   const api = await KubeApi.factory(log, ctx, k8sCtx.provider)
 
-  // FIXME: We're currently reading the manifests from the module source dir (instead of build dir)
-  // because the build may not have been staged.
-  // This means that manifests added via the `build.dependencies[].copy` field will not be included.
-  const manifests = await getManifests({ ctx, api, log, action, defaultNamespace: namespace, readFromSrcDir: true })
-  const prepareResult = await configureSpecialModesForManifests({
-    ctx: k8sCtx,
-    log,
-    action,
-    manifests,
-  })
-  const preparedManifests = prepareResult.manifests
+  // Note: This is analogous to how we version check Helm charts, i.e. we don't check every resource individually.
+  // Users can always force deploy, much like with Helm Deploys.
+  const metadataManifest = getMetadataManifest(action, defaultNamespace, [])
 
-  let {
-    state,
-    remoteResources,
-    mode: deployedMode,
-  } = await compareDeployedResources({
-    ctx: k8sCtx,
-    api,
-    namespace,
-    manifests: preparedManifests,
-    log,
-  })
+  let deployedMode: ActionMode = "default"
+  let state: DeployState = "ready"
+  let remoteResources: KubernetesResource[] = []
+  let forwardablePorts: ForwardablePort[] | undefined
 
-  // Local mode has its own port-forwarding configuration
-  const forwardablePorts = deployedMode === "local" ? [] : getForwardablePorts(remoteResources, action)
+  const remoteMetadataResource = await getDeployedResource(ctx, provider, metadataManifest, log)
 
-  if (state === "ready") {
-    // Local mode always takes precedence over sync mode
-    if (mode === "local" && spec.localMode && deployedMode !== "local") {
+  if (!remoteMetadataResource) {
+    state = "missing"
+  } else {
+    const deployedMetadata = parseMetadataResource(log, remoteMetadataResource)
+    deployedMode = deployedMetadata.mode
+
+    if (deployedMetadata.resolvedVersion !== action.versionString()) {
+      state = "outdated"
+    } else if (mode === "local" && spec.localMode && deployedMode !== "local") {
       state = "outdated"
     } else if (mode === "sync" && spec.sync?.paths && deployedMode !== "sync") {
       state = "outdated"
+    } else if (mode === "default" && deployedMode !== mode) {
+      state = "outdated"
+    }
+
+    const manifestMetadata = Object.values(deployedMetadata.manifestMetadata)
+
+    if (manifestMetadata.length > 0) {
+      try {
+        const maybeDeployedResources = await Bluebird.map(manifestMetadata, async (m) => {
+          return [m, await api.readOrNull({ log, ...m })]
+        })
+
+        const statuses: ResourceStatus[] = await Bluebird.map(maybeDeployedResources, async ([m, resource]) => {
+          if (!resource) {
+            return {
+              state: "missing" as const,
+              resource: { apiVersion: m.apiVersion, kind: m.kind, metadata: { name: m.name, namespace: m.namespace } },
+            }
+          }
+          remoteResources.push(resource)
+          return resolveResourceStatus({ api, namespace: defaultNamespace, resource, log })
+        })
+
+        state = resolveResourceStatuses(log, statuses)
+      } catch (error) {
+        log.debug({ msg: `Failed querying for remote resources: ${error.message}`, error })
+        state = "unknown"
+      }
+    }
+
+    // Note: Local mode has its own port-forwarding configuration
+    if (deployedMode !== "local" && remoteResources && remoteResources.length > 0) {
+      try {
+        forwardablePorts = getForwardablePorts(remoteResources, action)
+      } catch (error) {
+        log.debug({ msg: `Unable to extract forwardable ports: ${error.message}`, error })
+      }
     }
   }
 
@@ -216,7 +255,7 @@
       version: state === "ready" ? action.versionString() : undefined,
       detail: { remoteResources },
       mode: deployedMode,
-      ingresses: getK8sIngresses(remoteResources),
+      ingresses: getK8sIngresses(remoteResources || []),
     },
     // TODO-0.13.1
     outputs: {},
@@ -397,7 +436,7 @@
   }
 
   return {
-    state: "ready",
+    state: "not-ready",
     detail: status,
     outputs: {},
   }
Index: core/src/plugins/kubernetes/kubernetes-type/common.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/core/src/plugins/kubernetes/kubernetes-type/common.ts b/core/src/plugins/kubernetes/kubernetes-type/common.ts
--- a/core/src/plugins/kubernetes/kubernetes-type/common.ts	(revision 409899599c81409852f08b4de1e4f44acce1e84f)
+++ b/core/src/plugins/kubernetes/kubernetes-type/common.ts	(date 1689324546621)
@@ -9,13 +9,13 @@
 import { resolve } from "path"
 import { readFile } from "fs-extra"
 import Bluebird from "bluebird"
-import { flatten, set } from "lodash"
+import { flatten, keyBy, set } from "lodash"
 import { loadAll } from "js-yaml"
 
 import { KubernetesModule } from "./module-config"
 import { KubernetesResource } from "../types"
 import { KubeApi } from "../api"
-import { gardenAnnotationKey } from "../../../util/string"
+import { gardenAnnotationKey, stableStringify } from "../../../util/string"
 import { Log } from "../../../logger/log-entry"
 import { PluginContext } from "../../../plugin-context"
 import { ConfigurationError, PluginError } from "../../../exceptions"
@@ -24,9 +24,10 @@
 import { KubernetesDeployAction } from "./config"
 import { CommonRunParams } from "../../../plugin/handlers/Run/run"
 import { runAndCopy } from "../run"
-import { getTargetResource, getResourcePodSpec, getResourceContainer, makePodName } from "../util"
-import { Resolved } from "../../../actions/types"
+import { getTargetResource, getResourcePodSpec, getResourceContainer, makePodName, getResourceKey } from "../util"
+import { ActionMode, Resolved } from "../../../actions/types"
 import { KubernetesPodRunAction, KubernetesPodTestAction } from "./kubernetes-pod"
+import { V1ConfigMap } from "@kubernetes/client-node"
 import { glob } from "glob"
 
 /**
@@ -39,16 +40,14 @@
   log,
   action,
   defaultNamespace,
-  readFromSrcDir = false,
 }: {
   ctx: PluginContext
   api: KubeApi
   log: Log
   action: Resolved<KubernetesDeployAction | KubernetesPodRunAction | KubernetesPodTestAction>
   defaultNamespace: string
-  readFromSrcDir?: boolean
 }): Promise<KubernetesResource[]> {
-  const rawManifests = (await readManifests(ctx, action, log, readFromSrcDir)) as KubernetesResource[]
+  const rawManifests = (await readManifests(ctx, action, log)) as KubernetesResource[]
 
   // remove *List objects
   const manifests = rawManifests.flatMap((manifest) => {
@@ -71,6 +70,11 @@
     return manifest
   })
 
+  if (action.kind === "Deploy") {
+    // Add metadata ConfigMap to aid quick status check
+    manifests.push(getMetadataManifest(action, defaultNamespace, manifests))
+  }
+
   return Bluebird.map(manifests, async (manifest) => {
     // Ensure a namespace is set, if not already set, and if required by the resource type
     if (!manifest.metadata?.namespace) {
@@ -106,26 +110,80 @@
   })
 }
 
+export interface ManifestMetadata {
+  key: string
+  apiVersion: string
+  kind: string
+  name: string
+  namespace: string
+}
+
+export interface ParsedMetadataManifestData {
+  resolvedVersion: string
+  mode: ActionMode
+  manifestMetadata: { [key: string]: ManifestMetadata }
+}
+
+export function getMetadataManifest(
+  action: Resolved<KubernetesDeployAction>,
+  defaultNamespace: string,
+  manifests: KubernetesResource[]
+): KubernetesResource<V1ConfigMap> {
+  const manifestMetadata: ManifestMetadata[] = manifests.map((m) => ({
+    key: getResourceKey(m),
+    apiVersion: m.apiVersion,
+    kind: m.kind,
+    name: m.metadata.name,
+    namespace: m.metadata.namespace || defaultNamespace,
+  }))
+
+  return {
+    apiVersion: "v1",
+    kind: "ConfigMap",
+    metadata: {
+      name: `garden-meta-${action.kind.toLowerCase()}-${action.name}`,
+    },
+    data: {
+      resolvedVersion: action.versionString(),
+      mode: action.mode(),
+      manifestMetadata: stableStringify(keyBy(manifestMetadata, "key")),
+    },
+  }
+}
+
+export function parseMetadataResource(log: Log, resource: KubernetesResource<V1ConfigMap>): ParsedMetadataManifestData {
+  // TODO: validate schema here
+  const output: ParsedMetadataManifestData = {
+    resolvedVersion: resource.data?.resolvedVersion || "",
+    mode: (resource.data?.mode || "default") as ActionMode,
+    manifestMetadata: {},
+  }
+
+  const manifestMetadata = resource.data?.manifestMetadata
+
+  if (manifestMetadata) {
+    try {
+      // TODO: validate by schema
+      output.manifestMetadata = JSON.parse(manifestMetadata)
+    } catch (error) {
+      log.debug({ msg: `Failed querying for remote resources: ${error.message}`, error })
+    }
+  }
+
+  return output
+}
+
 const disallowedKustomizeArgs = ["-o", "--output", "-h", "--help"]
 
 /**
  * Read the manifests from the module config, as well as any referenced files in the config.
- *
- * @param module The kubernetes module to read manifests for.
- * @param readFromSrcDir Whether or not to read the manifests from the module build dir or from the module source dir.
- * In general we want to read from the build dir to ensure that manifests added via the `build.dependencies[].copy`
- * field will be included. However, in some cases, e.g. when getting the service status, we can't be certain that
- * the build has been staged and we therefore read the manifests from the source.
- *
- * TODO: Remove this once we're checking for kubernetes module service statuses with version hashes.
  */
 export async function readManifests(
   ctx: PluginContext,
   action: Resolved<KubernetesDeployAction | KubernetesPodRunAction | KubernetesPodTestAction>,
-  log: Log,
-  readFromSrcDir = false
+  log: Log
 ) {
-  const manifestPath = readFromSrcDir ? action.basePath() : action.getBuildPath()
+  const manifestPath = action.getBuildPath()
 
   const spec = action.getSpec()
 
Index: core/test/integ/src/plugins/kubernetes/kubernetes-type/handlers.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/core/test/integ/src/plugins/kubernetes/kubernetes-type/handlers.ts b/core/test/integ/src/plugins/kubernetes/kubernetes-type/handlers.ts
--- a/core/test/integ/src/plugins/kubernetes/kubernetes-type/handlers.ts	(revision 409899599c81409852f08b4de1e4f44acce1e84f)
+++ b/core/test/integ/src/plugins/kubernetes/kubernetes-type/handlers.ts	(date 1689324546649)
@@ -14,7 +14,11 @@
 import { TestGarden } from "../../../../../helpers"
 import { getKubernetesTestGarden } from "./common"
 import { DeployTask } from "../../../../../../src/tasks/deploy"
-import { getManifests } from "../../../../../../src/plugins/kubernetes/kubernetes-type/common"
+import {
+  getManifests,
+  getMetadataManifest,
+  parseMetadataResource,
+} from "../../../../../../src/plugins/kubernetes/kubernetes-type/common"
 import { KubeApi } from "../../../../../../src/plugins/kubernetes/api"
 import { ActionLog, createActionLog, Log } from "../../../../../../src/logger/log-entry"
 import { KubernetesPluginContext, KubernetesProvider } from "../../../../../../src/plugins/kubernetes/config"
@@ -127,7 +131,143 @@
     }
   })
 
-  describe("getServiceStatus", () => {
+  describe("getKubernetesDeployStatus", () => {
+    it("returns ready status and correct detail for a ready deployment", async () => {
+      const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
+      const action = await garden.resolveAction<KubernetesDeployAction>({
+        action: graph.getDeploy("module-simple"),
+        log: garden.log,
+        graph,
+      })
+      const deployParams = {
+        ctx,
+        log: actionLog,
+        action,
+        force: false,
+      }
+
+      await kubernetesDeploy(deployParams)
+
+      const status = await getKubernetesDeployStatus(deployParams)
+      expect(status.state).to.equal("ready")
+
+      expect(status.detail?.mode).to.equal("default")
+      expect(status.detail?.forwardablePorts).to.eql([
+        {
+          name: undefined,
+          protocol: "TCP",
+          targetName: "Deployment/busybox-deployment",
+          targetPort: 80,
+        },
+      ])
+
+      const remoteResources = status.detail?.detail?.remoteResources
+      expect(remoteResources).to.exist
+      expect(remoteResources.length).to.equal(1)
+      expect(remoteResources[0].kind).to.equal("Deployment")
+    })
+
+    it("should return missing status when metadata ConfigMap is missing", async () => {
+      const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
+      const action = await garden.resolveAction<KubernetesDeployAction>({
+        action: graph.getDeploy("module-simple"),
+        log: garden.log,
+        graph,
+      })
+      const deployParams = {
+        ctx,
+        log: actionLog,
+        action,
+        force: false,
+      }
+
+      await kubernetesDeploy(deployParams)
+
+      const namespace = await getActionNamespace({
+        ctx,
+        log,
+        action,
+        provider: ctx.provider,
+        skipCreate: true,
+      })
+
+      const metadataManifest = getMetadataManifest(action, namespace, [])
+
+      await api.deleteBySpec({ log, namespace, manifest: metadataManifest })
+
+      const status = await getKubernetesDeployStatus(deployParams)
+      expect(status.state).to.equal("not-ready")
+      expect(status.detail?.state).to.equal("missing")
+    })
+
+    it("should return outdated status when metadata ConfigMap has different version", async () => {
+      const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
+      const action = await garden.resolveAction<KubernetesDeployAction>({
+        action: graph.getDeploy("module-simple"),
+        log: garden.log,
+        graph,
+      })
+      const deployParams = {
+        ctx,
+        log: actionLog,
+        action,
+        force: false,
+      }
+
+      await kubernetesDeploy(deployParams)
+
+      const namespace = await getActionNamespace({
+        ctx,
+        log,
+        action,
+        provider: ctx.provider,
+        skipCreate: true,
+      })
+
+      const metadataManifest = getMetadataManifest(action, namespace, [])
+      metadataManifest.data!.resolvedVersion = "v-foo"
+
+      await api.replace({ log, namespace, resource: metadataManifest })
+
+      const status = await getKubernetesDeployStatus(deployParams)
+      expect(status.state).to.equal("not-ready")
+      expect(status.detail?.state).to.equal("outdated")
+    })
+
+    it("should return outdated status when metadata ConfigMap has different mode", async () => {
+      const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
+      const action = await garden.resolveAction<KubernetesDeployAction>({
+        action: graph.getDeploy("module-simple"),
+        log: garden.log,
+        graph,
+      })
+      const deployParams = {
+        ctx,
+        log: actionLog,
+        action,
+        force: false,
+      }
+
+      await kubernetesDeploy(deployParams)
+
+      const namespace = await getActionNamespace({
+        ctx,
+        log,
+        action,
+        provider: ctx.provider,
+        skipCreate: true,
+      })
+
+      const metadataManifest = getMetadataManifest(action, namespace, [])
+      metadataManifest.data!.mode = "sync"
+
+      await api.replace({ log, namespace, resource: metadataManifest })
+
+      const status = await getKubernetesDeployStatus(deployParams)
+      expect(status.state).to.equal("not-ready")
+      expect(status.detail?.state).to.equal("outdated")
+    })
+
     it("should return not-ready status for a manifest with a missing resource type", async () => {
       const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
       const action = await garden.resolveAction<KubernetesDeployAction>({
@@ -183,11 +323,30 @@
         log,
         action: resolvedAction,
         defaultNamespace: namespace,
-        readFromSrcDir: true,
       })
       return { deployParams, manifests }
     }
 
+    it("gets the correct manifests when `build` is set", async () => {
+      const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
+      const action = graph.getDeploy("with-build-action")
+      const deployParams = {
+        ctx,
+        log: actionLog,
+        action: await garden.resolveAction<KubernetesDeployAction>({ action, log: garden.log, graph }),
+        force: false,
+      }
+
+      const status = await kubernetesDeploy(deployParams)
+      expect(status.state).to.eql("ready")
+
+      const remoteResources = status.detail?.detail?.remoteResources
+      expect(remoteResources).to.exist
+      expect(remoteResources.length).to.equal(1)
+      expect(remoteResources[0].kind).to.equal("Deployment")
+      expect(remoteResources[0].metadata?.name).to.equal("busybox-deployment")
+    })
+
     it("should successfully deploy when serviceResource doesn't have a containerModule", async () => {
       const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
       const action = graph.getDeploy("module-simple")
@@ -207,6 +366,48 @@
       expect(namespaceStatus!.namespaceName).to.eql("kubernetes-type-test-default")
     })
 
+    it("creates a metadata ConfigMap describing what was last deployed", async () => {
+      const graph = await garden.getConfigGraph({ log: garden.log, emit: false })
+      const action = graph.getDeploy("module-simple")
+      const resolvedAction = await garden.resolveAction<KubernetesDeployAction>({ action, log: garden.log, graph })
+      const deployParams = {
+        ctx,
+        log: actionLog,
+        action: resolvedAction,
+        force: false,
+      }
+      const status = await kubernetesDeploy(deployParams)
+      expect(status.state).to.eql("ready")
+
+      const namespace = await getActionNamespace({
+        ctx,
+        log,
+        action: resolvedAction,
+        provider: ctx.provider,
+        skipCreate: true,
+      })
+
+      const emptyManifest = getMetadataManifest(resolvedAction, namespace, [])
+
+      const metadataResource = await api.readBySpec({ log, namespace, manifest: emptyManifest })
+
+      expect(metadataResource).to.exist
+
+      const parsedMetadata = parseMetadataResource(log, metadataResource)
+
+      expect(parsedMetadata.resolvedVersion).to.equal(resolvedAction.versionString())
+      expect(parsedMetadata.mode).to.equal("default")
+      expect(parsedMetadata.manifestMetadata).to.eql({
+        "Deployment/busybox-deployment": {
+          apiVersion: "apps/v1",
+          key: "Deployment/busybox-deployment",
+          kind: "Deployment",
+          name: "busybox-deployment",
+          namespace: "kubernetes-type-test-default",
+        },
+      })
+    })
+
     it("should toggle sync mode", async () => {
       const syncData = await getTestData("with-source-module", {
         sync: ["deploy.with-source-module"],
@@ -322,8 +523,9 @@
     it("should successfully deploy List manifest kinds", async () => {
       const configMapList = await getTestData("config-map-list", {})
 
-      // this should be 3, and not 1, as we transform *List objects to separate manifests
-      expect(configMapList.manifests.length).to.be.equal(3)
+      // this should be 4, and not 2 (including the metadata manifest),
+      // as we transform *List objects to separate manifests
+      expect(configMapList.manifests.length).to.be.equal(4)
 
       // test successful deploy
       await kubernetesDeploy(configMapList.deployParams)
Index: core/test/integ/src/plugins/kubernetes/kubernetes-type/common.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/core/test/integ/src/plugins/kubernetes/kubernetes-type/common.ts b/core/test/integ/src/plugins/kubernetes/kubernetes-type/common.ts
--- a/core/test/integ/src/plugins/kubernetes/kubernetes-type/common.ts	(revision 409899599c81409852f08b4de1e4f44acce1e84f)
+++ b/core/test/integ/src/plugins/kubernetes/kubernetes-type/common.ts	(date 1689324546646)
@@ -60,7 +60,7 @@
       action["_config"].spec.kustomize!.extraArgs = ["--output", "foo"]
 
       await expectError(
-        () => readManifests(ctx, action, garden.log, false),
+        () => readManifests(ctx, action, garden.log),
         (err) => expect(err.message).to.equal(expectedErr)
       )
     })
@@ -69,7 +69,7 @@
       action["_config"].spec.kustomize!.extraArgs = ["-o", "foo"]
 
       await expectError(
-        () => readManifests(ctx, action, garden.log, false),
+        () => readManifests(ctx, action, garden.log),
         (err) => expect(err.message).to.equal(expectedErr)
       )
     })
@@ -78,7 +78,7 @@
       action["_config"].spec.kustomize!.extraArgs = ["-h"]
 
       await expectError(
-        () => readManifests(ctx, action, garden.log, false),
+        () => readManifests(ctx, action, garden.log),
         (err) => expect(err.message).to.equal(expectedErr)
       )
     })
@@ -87,20 +87,20 @@
       action["_config"].spec.kustomize!.extraArgs = ["--help"]
 
       await expectError(
-        () => readManifests(ctx, action, garden.log, false),
+        () => readManifests(ctx, action, garden.log),
         (err) => expect(err.message).to.equal(expectedErr)
       )
     })
 
     it("runs kustomize build in the given path", async () => {
-      const result = await readManifests(ctx, action, garden.log, true)
+      const result = await readManifests(ctx, action, garden.log)
       const kinds = result.map((r) => r.kind)
       expect(kinds).to.have.members(["ConfigMap", "Service", "Deployment"])
     })
 
     it("adds extraArgs if specified to the build command", async () => {
       action["_config"].spec.kustomize!.extraArgs = ["--reorder", "none"]
-      const result = await readManifests(ctx, action, garden.log, true)
+      const result = await readManifests(ctx, action, garden.log)
       const kinds = result.map((r) => r.kind)
       expect(kinds).to.eql(["Deployment", "Service", "ConfigMap"])
     })

Additional context

Your environment

  • OS: macOS
  • How I'm running Kubernetes: Docker desktop

garden version 0.13.0

@stefreak
Copy link
Member Author

stefreak commented Jun 1, 2023

This might be a duplicate of #4505

@vvagaytsev
Copy link
Collaborator

This was fixed in 0.13.7 (see #4516), but that PR was partially reverted in 0.13.8. Reopening this as a valid one.

@vvagaytsev
Copy link
Collaborator

Closing as a duplicate of #4931

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Status: Done
3 participants