Support for generated files

[fsaintjacques]

Is it possible to have ko invoke commands to generate files, notably protoc? This seems to be a very common thing to do with go services.

[imjasonh]

It's possible, but I think it would be against ko's (and go's) philosophy, so we'd probably need a very strong motivation to make it part of ko proper.

The usual recommendation is to use go generate to generate files, then go build etc to build them, and I think ko publish should follow that pattern in place of go build. I'd be hesitant to have ko publish encompass both go generate and go build.

[fsaintjacques]

I understand the will to follow philosophies, but the reality is that go micro-services using protobuf definition is not an exception. GoRelease supports the pre/post hooks, are you open to a patch?

[imjasonh]

I'd much prefer to discuss an abstract design before asking you to write code for us to review. But even then, I don't think I'd be open to accepting a proposal to add pre/post hooks into ko without a stronger motivation than has been put forward so far.

Is there a reason go generate && ko publish is not suitable for your use case?

[jonjohnsonjr]

GoRelease supports the pre/post hooks, are you open to a patch?

I think the ideal end-state is that make it possible (or the default) to support everything goreleaser does. Right now, it's unclear to me what kind of relationship makes the most sense between the projects. I don't think it make sense to just vendor all of goreleaser or reimplement everything, but there may be some way to make this more pluggable...

[mattmoor]

My $0.02 tl;dr is that we're based on Go idioms, not goreleaser.

The overarching Go guidance around codegen (for better or worse) is to check it in. Things like go {get,install} or even just vendoring libraries simply won't work if you don't check in the codegen.

The real place where I see this being useful is more inner-loop development where juggling tools is annoying, but generally things like proto definitions (or K8s API types) change orders of magnitude less often than the surrounding code.

Personally, from a security standpoint I really like the property that ko resolve doesn't execute any user-sourced code, which will become an increasingly important property as folks start to reason more about hermetic builds (and what should/shouldn't be run in the network jail).

[halvards]

On inner-loop development, Skaffold supports ko natively and provides lifecycle hooks.

By combining these hooks with profiles, you can run go generate locally as part of the inner dev loop, while skipping this step in a CI build (so the CI build uses only checked-in files).

An example config to achieve this:

apiVersion: skaffold/v2beta26
kind: Config
build:
  artifacts:
  - image: go-generate-build-hook-via-profile
    ko: {}
profiles:
- name: inner-loop
  activation:
  - command: debug
  - command: dev
  - command: run
  patches:
  - op: add
    path: /build/artifacts/0/hooks
    value:
      before:
      - command: ["go", "generate", "./..."]

With this config, Skaffold executes go generate when you run commands that are aimed at local development: skaffold run, skaffold dev, or skaffold debug.

Other commands that are aimed at CI build scenarios, such as skaffold build, skip the go generate step.

[jonjohnsonjr]

I'm liking this pattern of skaffold supporting use cases that feel inappropriate for ko ❤️