kubernetes: Feature Request: Step Pod Annotations

[benwilson512]

Use Case

Specific use case: Enabling AWS IAM roles for each pod step.

Kube2IAM works by looking at annotations on a given pod to figure out which role to provide info for. Before, we could just annotate the agent pods, but now each step is running in a new pod and we need a way to annotate that.

In general though there are a variety of K8s features that use annotations.

API

From an API perspective, we could treat this a bit like limits were handled in: #29 . This has the perk that you could apply very specific roles to each step of the pipeline.

[srhopkins]

Is there a work around for this currently? We are blocked as we're unable to access s3 buckets that we use kiam roles with.

[kevtaylor]

@srhopkins We have an opensource project that uses mutatingwebhooks to apply rules onto namespaces, which might be an option for you https://github.com/HotelsDotCom/kube-graffiti

[willejs]

@bradrydzewski there are two PRs that address this, can you review them?

[willejs]

@kevtaylor thats a really cool project. I don't think this will work with the way drone k8s jobs currently work though? The jobs expose info via annotations instead of labels, and the only labels that it sets in the spec that you could use field selectors is job-name, which would be a bit nasty... What are you guys doing? I am curious.
I think the answer is to either fix the runtime annotations as per this PR or use kube-graffiti when there are proper labels added?

[kevtaylor]

@willejs kube-grafitti can do labels and annotations - we use it to decorate namespaces / kiam policies in general - but appreciate it may be tricky if annotations are dynamic

[willejs]

@kevtaylor yeah, you can paint labels or annotations, but you can only match on labels or field selectors (name, namespace) so i don't think it's going to work well in the current state?
Adding the annotation to the job would work, but I am also thinking that this might not even be the way to go considering the other runtime changes...
#69

[Sluggerman]

Hello Drone team,

Firstly thank you for the great tool and all your hard work!

We are also in need of being able to set kube2iam pod annotations, so jobs could natively authenticate in AWS with number of use cases such as ECR, S3, Vault AWS auth etc.

We would really appreciate if this could be added to Drone sooner.

[bradrydzewski]

there is a new (still experimental) kubernetes runner that supports annotations:
https://github.com/drone-runners/drone-runner-kube

kind: pipeline
type: kubernetes

metadata:
  annotations:
    foo: bar

steps: [ ... ]