-
Notifications
You must be signed in to change notification settings - Fork 691
Kubernetes rules #82
Comments
I've been playing around a bit here, and have a prototype of some rules for managing a k8s I've been playing around with using these to deploy different environments from a single template in my bazel-grpc "Hello World" app. You can explore the bazel run :dev.replace At least for this relatively simple app, if I make some edits the above command takes <10 seconds to have the new app running on my cluster (including C++ compilation, image packaging, image pushing, and It is notable that there is very little Errata / TODO / Stuff I still don't like:
|
FYI I see |
FWIW, I had a demo bug (I hadn't dropped |
I'd like to be able to build a container, push it to the docker daemon running in a minikube cluster, and then create a deployment using that container. I've tried out the following rule from mattmoor/rules_k8s (with the address to the minikube VM hardcoded while testing):
Running eval $(minikube docker-env) followed by this rule gives me the following error:
which I assume suggests that the DOCKER environment variables with the path to certs etc is currently not picked up. Omitting the "images" in the rules, the deployment is successfully created in minikube (however it of course fails to fetch an image). |
Indeed, I have not fully worked out the appropriate interaction with minikube in my prototype. Frankly, I am glad it worked as nearly as you describe! However, minikube is clearly one of the core scenarios I'd like this to support in a real For minikube, one of the options I'd considered previously was side-loading the containers into the Docker daemon (e.g. via The |
We don't have a docker registry by default, but it's possible to run one in minikube. You then need to make sure your pods all reference the in-cluster registry namespace for containers, though. The end vars probably make the most sense. |
As far as I know, minikube only has a docker daemon running by default (see https://github.com/kubernetes/minikube/blob/master/docs/reusing_the_docker_daemon.md). You can then use the
Using |
@niclaslockner Sure, what I'd meant was that I'd want Considering we already have users specify the What I'm thinking is something like: k8s_defaults(
name = "k8s_local_deploy",
kind = "deployment",
minikube = True, # Use minikube CLI to determine the rest.
) If juggling multiple minikubes is a thing (and they are distinguished by cluster name) then perhaps a better interface would be a parallel minikube_defaults(
name = "k8s_local_deploy",
kind = "deployment",
cluster = "my-local-cluster",
) |
I wanted to surface my current thinking around how these rules will manifest in the near term, and solicit feedback. My current prototype conflates three things:
I think that the most immediate value of these rules is delivering on I want to punt on substitution in v1 for a few reasons:
So in the immediate term, I think the surface I will target (with each bullet as an increment of functionality) is:
How important do folks think templating is? Does my logic here make sense? I'd appreciate any feedback here. |
Would this also build any images referenced in the yaml?
I'm not sure I understand the difference here. Is this about whether :foo is an image, or if it references one?
Is the idea to completely wrap kubectl with these extra actions? |
@dlorenc I'm not sure that in the first increment I'll even expose the kwarg, but once it's there it would because they'd be runfiles of the executable version. The difference is:
Technically, with multiple image references (and a partial We don't need to fully wrap Regarding templating, I think that if we head down that route (in a further increment), we should adhere to this accepted K8s design proposal. |
I have created the repo: https://github.com/bazelbuild/rules_k8s I will start adding some of the elements of my prototype there as I break off pieces and clean them up. I have enabled issues on that repo, so let's discuss further topics there. |
Part of the issues in bazelbuild#82
I am opening this issue to track discussions around what shape
rules_k8s
might take, and to enumerate the kinds of scenarios folks would like to seerules_k8s
cover.The text was updated successfully, but these errors were encountered: