feat(ko): Inferred file sync for ko static assets (#7728)

This change adds inferred file sync mode to the ko builder.

Skaffold syncs static assets from the workspace to a running
container, based on the `<importpath>/kodata/` convention:
https://github.com/google/ko#static-assets

Tracking: #7131

docs-v2/content/en/docs/pipeline-stages/builders/ko.md

@@ -389,7 +389,46 @@ To learn more about how Skaffold debugs Go applications, read the
 
 ### File sync
 
-File `sync` is not supported while the ko builder feature is in Alpha.
+The `ko` builder can
+[sync files to a running container]({{< relref "/docs/pipeline-stages/filesync" >}})
+when you run `skaffold dev`.
+
+The sync feature for the `ko` builder only works for 
+[static assets bundled with the container image](https://github.com/google/ko#static-assets).
+
+Use `infer` mode to specify patterns for the files you want to sync. The
+infer patterns are relative to the `context` directory.
+
+For instance, if your main package is in the `context` directory, you can use
+this configuration to sync all the static files bundled with the container
+image:
+
+```yaml
+    sync:
+      infer:
+      - kodata/**/*
+```
+
+Note that the file sync feature requires the `tar` command to be available in
+the container. The default `ko` builder base image does not include the `tar`
+command. Use the `fromImage` field in the `ko` builder configuration in your
+`skaffold.yaml` file to specify a base image that contains the `tar` command,
+such as `gcr.io/distroless/base:debug`.
+
+You can use [profiles]({{< relref "/docs/environment/profiles" >}}) with
+activation by command to override the `fromImage` value only when running
+`skaffold dev`, such as in this example:
+
+```yaml
+profiles:
+- name: sync
+  activation:
+  - command: dev
+  patches:
+  - op: add
+    path: /build/artifacts/0/ko/fromImage
+    value: gcr.io/distroless/base:debug
+```
 
 ### Remote builds
 

docs-v2/content/en/docs/pipeline-stages/filesync.md

@@ -14,11 +14,12 @@ This tar file is sent to and extracted on the corresponding containers.
 Multiple types of sync are supported by Skaffold:
 
  + `manual`: The user must specify both the files in their local workspace and the destination in the running container.
-   This is supported by every type of artifact.
+   This sync mode is supported by every type of artifact.
 
  + `infer`: The destinations for each changed file is inferred from the builder.
    The docker and kaniko builders examine instructions in a Dockerfile.
    This inference is also supported for custom artifacts that **explicitly declare a dependency on a Dockerfile.**
+   The ko builder can sync static content using this sync mode.
 
 + `auto`: Skaffold automatically configures the sync.  This mode is only supported by Jib and Buildpacks artifacts.
    Auto sync mode is enabled by default for Buildpacks artifacts.
@@ -46,13 +47,18 @@ The following example showcases manual filesync:
 
 ### Inferred sync mode
 
-For docker artifacts, Skaffold knows how to infer the desired destination from the artifact's `Dockerfile`
+For Docker artifacts, Skaffold knows how to infer the desired destination from the artifact's `Dockerfile`
 by examining the `ADD` and `COPY` instructions.
-To enable syncing, you only need to specify which files are eligible for syncing in the sync rules.
-The sync rules for inferred sync mode is just a list of glob patterns.
-The following example showcases this filesync mode:
 
-Given a Dockerfile such as
+For Ko artifacts, Skaffold infers the destination from the structure of your
+codebase.
+
+To enable syncing, you specify which files are eligible for syncing in the sync rules.
+The sync rules for inferred sync mode is a list of glob patterns.
+
+The following example showcases this filesync mode for Docker artifacts:
+
+Given this Dockerfile:
 
 ```Dockerfile
 FROM hugo
@@ -74,12 +80,15 @@ And a `skaffold.yaml` with the following sync configuration:
 - The last rule enables synchronization for all `md` files below the `content/en`.
   For example, `content/en/sub/index.md` ↷ `content/sub/index.md` but _not_ `content/en_GB/index.md`.
 
-Inferred sync mode only applies to modified and added files.
-File deletion will always cause a complete rebuild.
+For Docker artifacts, inferred sync mode only applies to modified and added
+files; file deletion will cause a complete rebuild.
 
 For multi-stage Dockerfiles, Skaffold only examines the last stage.
 Use manual sync rules to sync file copies from other stages.
 
+[Ko artifacts supports syncing static content]({{<relref "/docs/pipeline-stages/builders/ko#file-sync">}}),
+and the sync rules apply to added, modified, and deleted files.
+
 ### Auto sync mode
 
 In auto sync mode, Skaffold automatically generates sync rules for known file types. 

docs/content/en/docs/pipeline-stages/builders/ko.md

@@ -389,7 +389,46 @@ To learn more about how Skaffold debugs Go applications, read the
 
 ### File sync
 
-File `sync` is not supported while the ko builder feature is in Alpha.
+The `ko` builder can
+[sync files to a running container]({{< relref "/docs/pipeline-stages/filesync" >}})
+when you run `skaffold dev`.
+
+The sync feature for the `ko` builder only works for 
+[static assets bundled with the container image](https://github.com/google/ko#static-assets).
+
+Use `infer` mode to specify patterns for the files you want to sync. The
+infer patterns are relative to the `context` directory.
+
+For instance, if your main package is in the `context` directory, you can use
+this configuration to sync all the static files bundled with the container
+image:
+
+```yaml
+    sync:
+      infer:
+      - kodata/**/*
+```
+
+Note that the file sync feature requires the `tar` command to be available in
+the container. The default `ko` builder base image does not include the `tar`
+command. Use the `fromImage` field in the `ko` builder configuration in your
+`skaffold.yaml` file to specify a base image that contains the `tar` command,
+such as `gcr.io/distroless/base:debug`.
+
+You can use [profiles]({{< relref "/docs/environment/profiles" >}}) with
+activation by command to override the `fromImage` value only when running
+`skaffold dev`, such as in this example:
+
+```yaml
+profiles:
+- name: sync
+  activation:
+  - command: dev
+  patches:
+  - op: add
+    path: /build/artifacts/0/ko/fromImage
+    value: gcr.io/distroless/base:debug
+```
 
 ### Remote builds
 

docs/content/en/docs/pipeline-stages/filesync.md

@@ -14,11 +14,12 @@ This tar file is sent to and extracted on the corresponding containers.
 Multiple types of sync are supported by Skaffold:
 
  + `manual`: The user must specify both the files in their local workspace and the destination in the running container.
-   This is supported by every type of artifact.
+   This sync mode is supported by every type of artifact.
 
  + `infer`: The destinations for each changed file is inferred from the builder.
    The docker and kaniko builders examine instructions in a Dockerfile.
    This inference is also supported for custom artifacts that **explicitly declare a dependency on a Dockerfile.**
+   The ko builder can sync static content using this sync mode.
 
 + `auto`: Skaffold automatically configures the sync.  This mode is only supported by Jib and Buildpacks artifacts.
    Auto sync mode is enabled by default for Buildpacks artifacts.
@@ -46,13 +47,18 @@ The following example showcases manual filesync:
 
 ### Inferred sync mode
 
-For docker artifacts, Skaffold knows how to infer the desired destination from the artifact's `Dockerfile`
+For Docker artifacts, Skaffold knows how to infer the desired destination from the artifact's `Dockerfile`
 by examining the `ADD` and `COPY` instructions.
-To enable syncing, you only need to specify which files are eligible for syncing in the sync rules.
-The sync rules for inferred sync mode is just a list of glob patterns.
-The following example showcases this filesync mode:
 
-Given a Dockerfile such as
+For Ko artifacts, Skaffold infers the destination from the structure of your
+codebase.
+
+To enable syncing, you specify which files are eligible for syncing in the sync rules.
+The sync rules for inferred sync mode is a list of glob patterns.
+
+The following example showcases this filesync mode for Docker artifacts:
+
+Given this Dockerfile:
 
 ```Dockerfile
 FROM hugo
@@ -74,12 +80,15 @@ And a `skaffold.yaml` with the following sync configuration:
 - The last rule enables synchronization for all `md` files below the `content/en`.
   For example, `content/en/sub/index.md` ↷ `content/sub/index.md` but _not_ `content/en_GB/index.md`.
 
-Inferred sync mode only applies to modified and added files.
-File deletion will always cause a complete rebuild.
+For Docker artifacts, inferred sync mode only applies to modified and added
+files; file deletion will cause a complete rebuild.
 
 For multi-stage Dockerfiles, Skaffold only examines the last stage.
 Use manual sync rules to sync file copies from other stages.
 
+[Ko artifacts supports syncing static content]({{<relref "/docs/pipeline-stages/builders/ko#file-sync">}}),
+and the sync rules apply to added, modified, and deleted files.
+
 ### Auto sync mode
 
 In auto sync mode, Skaffold automatically generates sync rules for known file types. 

integration/examples/ko-sync-infer/README.md

@@ -0,0 +1,18 @@
+### Example: inferred file sync using the ko builder
+
+This example uses
+[inferred file sync](https://skaffold.dev/docs/pipeline-stages/filesync/#inferred-sync-mode)
+for static assets with the
+[`ko` builder](https://skaffold.dev/docs/pipeline-stages/builders/ko/)
+for a Go web app.
+
+To observe the behavior of file sync, run this command:
+
+```shell
+skaffold dev
+```
+
+Try changing the HTML file in the `kodata` directory to see how Skaffold
+syncs the file.
+
+If change the the `main.go` file, Skaffold will rebuild and redeploy the image.

integration/examples/ko-sync-infer/go.mod

@@ -0,0 +1,17 @@
+// Copyright 2022 The Skaffold Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+module github.com/GoogleContainerTools/skaffold/examples/ko-sync-infer
+
+go 1.13

integration/examples/ko-sync-infer/k8s/example.yaml

@@ -0,0 +1,45 @@
+# Copyright 2022 The Skaffold Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: v1
+kind: Service
+metadata:
+  name: ko-sync-infer
+spec:
+  ports:
+  - name: http
+    port: 80
+    targetPort: 8080
+  selector:
+    app: ko-sync-infer
+  type: ClusterIP
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ko-sync-infer
+spec:
+  selector:
+    matchLabels:
+      app: ko-sync-infer
+  template:
+    metadata:
+      labels:
+        app: ko-sync-infer
+    spec:
+      containers:
+      - image: skaffold-ko-sync-infer
+        name: ko-sync-infer
+        ports:
+        - containerPort: 8080

integration/examples/ko-sync-infer/kodata/index.html

@@ -0,0 +1,27 @@
+<!DOCTYPE html>
+<!--
+Copyright 2022 The Skaffold Authors
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width">
+  <title>Skaffold ko builder inferred file sync example</title>
+  <link rel="icon" href="data:,">
+</head>
+<body>
+  <p>Try changing this text in dev mode.</p>
+</body>
+</html>

integration/examples/ko-sync-infer/main.go

@@ -0,0 +1,43 @@
+/*
+Copyright 2022 The Skaffold Authors
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package main
+
+import (
+	"fmt"
+	"log"
+	"net/http"
+	"os"
+)
+
+const defaultStaticPath = "kodata" // the value to use for local development
+
+func main() {
+	if err := run(); err != nil {
+		fmt.Fprintf(os.Stderr, "%+v", err)
+		os.Exit(1)
+	}
+}
+
+func run() error {
+	staticPath := os.Getenv("KO_DATA_PATH")
+	if staticPath == "" {
+		staticPath = defaultStaticPath
+	}
+	http.Handle("/", http.FileServer(http.Dir(staticPath)))
+	log.Println("Listening on port 8080")
+	return http.ListenAndServe(":8080", nil)
+}