Allow adding postStartScript lifecycle hook to all deployments

[cognifloyd]

This builds on the postStart lifecycle hook from #219, but allows adding custom postStartScript (via values) to each deployment pod.

In my old 1ppc-based st2 cluster, the entrypoint script had some cluster-specific
customizations. With the current docker images, we don't use an entrypoint script.
Some of those customizations do not translate well to cacheable Dockerfile commands.

The postStart lifecycle event provides something similar in that I can add those
cluster-specific customizations when I can't bake the changes directly into custom
docker images.

For example, in the actionrunner pods, I have a script that:

• configures ansible,
• clones a git repo with ansible-related content,
• configures ssmtp in the pod for sending email,
• and runs a sanity check on some perl modules used by an internal pack
  (the perl modules themselves are baked into the docker image).
  And in the chatops pod, our image has a slightly customized version of hubot that
  needs an additional config file. That config file should not be baked into the image.

We also use the init script for rapid testing of changes in our development st2 cluster
before we bake them into the docker images (where baking the changes in makes sense).

Though these could potentially be done via initContainers, adding custom initContainers
becomes much more complex when you consider the required additional volumes and volumeMounts.

see:
https://kubernetes.io/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/
https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/

[cognifloyd]

Hmm. I should probably add a checksum annotation that covers the configmap template and the postStartScript value.
edit: done

[cognifloyd]

@armab edit: please checkout #219 next, which was separated from this PR I think this is the next one to review (please, when you have time).

It fixes the stanley_rsa file permissions issue using lifecycle.postStart and includes a test to make sure that the permissions stay fixed. It also allows for custom postStart scripts for the rest of the pods.

You might want to checkout the k8s docs in the PR description. :)

[cognifloyd]

K. I split the stanley_rsa fix into #219. This is now only about adding the postStartScript to all the rest of the pods.

#206 builds on #219 now, so please review and merge #219 first. edit: merged

[arm4b]

I think it will solve one of the use cases that many requested, - installing packages for some of the st2 actions like, say, ansible instead of providing their own forked pre-baked image, both approaches with their pros/cons.

Oerall any customization before starting the st2 cluster makes sense and brings a lot of configuration flexibility 👍

[cognifloyd]

@armab I'm glad you like the idea

[arm4b]

Nice addition for customizing the ST2 cluster environment.
Thanks for that!