generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Configure Tilt for rapid iterative development.
- Loading branch information
1 parent
ddfc6ae
commit 660374c
Showing
18 changed files
with
706 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
load('ext://kubebuilder', 'kubebuilder') | ||
|
||
docker_build('dtr.thefacebook.com/k8s/aqueduct', '.', | ||
dockerfile='Dockerfile') | ||
|
||
yaml = kustomize('config/default/') | ||
k8s_yaml(yaml) | ||
|
||
k8s_resource('aqueduct-controller-manager', | ||
port_forwards=8000) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
{ | ||
"Extensions": [ | ||
{ | ||
"Name": "kubebuilder", | ||
"ExtensionRegistry": "https://github.com/tilt-dev/tilt-extensions", | ||
"TimeFetched": "2021-10-26T12:39:48.923065-07:00" | ||
}, | ||
{ | ||
"Name": "restart_process", | ||
"ExtensionRegistry": "https://github.com/tilt-dev/tilt-extensions", | ||
"TimeFetched": "2021-10-26T12:40:00.050342-07:00" | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
# Kubebuilder | ||
|
||
Authors: [Ellen Körbes](https://github.com/ellenkorbes), [Dan Miller](https://github.com/jazzdan/) | ||
|
||
Enable live-update for developing [Kubebuilder](https://github.com/kubernetes-sigs/kubebuilder) projects. | ||
|
||
## Usage | ||
|
||
To use this extension, create a new directory, import the extension into your Tiltfile, and call the `kubebuilder()` function. E.g. | ||
|
||
``` python | ||
load('ext://kubebuilder', 'kubebuilder') | ||
kubebuilder("domain.tilt.dev", "tiltgroup", "v1", "CustomKind") | ||
``` | ||
|
||
The arguments are domain, group, version, and kind for the operator to be created. | ||
|
||
It is required that kubebuilder be installed in the system and present on PATH. If you don't have Kubebuilder installed, check out Kubebuilder's [Quick Start](https://book.kubebuilder.io/quick-start.html#installation) guide. | ||
|
||
## In Depth | ||
|
||
This extension automates live-update setup for CRD changes, controller code changes, and sample YAML changes. | ||
|
||
### CRD Changes | ||
|
||
Let's say I have a `CustomKind` CRD: | ||
|
||
``` | ||
$ kubectl get CustomKind | ||
NAME AGE | ||
customkind-sample 5m41s | ||
``` | ||
|
||
I can change the CRD definition by adding the code below, which changes the columns printed on kubectl output: | ||
|
||
``` | ||
// +kubebuilder:printcolumn:name="Foo",type=string,JSONPath=`.spec.foo` | ||
``` | ||
|
||
And the output gets instantly updated: | ||
|
||
``` | ||
$ kubectl get CustomKind | ||
NAME FOO | ||
customkind-sample bar | ||
``` | ||
|
||
### Sample YAML Changes | ||
|
||
Changes to files in the `./config/samples` files also get automatically updated. E.g. by changing the following in my sample YAML... | ||
|
||
``` | ||
spec: | ||
# Add fields here | ||
foo: newvalue | ||
``` | ||
|
||
I now get the following output: | ||
|
||
``` | ||
$ kubectl get CustomKind | ||
NAME FOO | ||
customkind-sample newvalue | ||
``` | ||
|
||
### Controller Code Changes | ||
|
||
Lastly, any changes to controller code also update automatically. For example, by adding the following to `./controllers/customkind_controller.go`... | ||
|
||
``` | ||
log := r.Log.WithValues("customkind", req.NamespacedName) | ||
log.Info("CONTROLLER SAYS HELLO!") | ||
``` | ||
|
||
Tilt's logs immediately update to display the following output: | ||
|
||
``` | ||
[manager] 2020-11-30T14:47:31.550Z INFO controllers.CustomKind CONTROLLER SAYS HELLO! {"customkind": "default/customkind-sample"} | ||
``` | ||
|
||
## Clean Up | ||
|
||
To delete all Kubebuilder resources from your cluster, use Tilt's `tilt down` command. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
load('ext://restart_process', 'docker_build_with_restart') | ||
|
||
def kubebuilder(DOMAIN, GROUP, VERSION, KIND, IMG='controller:latest', CONTROLLERGEN='crd:trivialVersions=true rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases;'): | ||
|
||
DOCKERFILE = '''FROM golang:alpine | ||
WORKDIR / | ||
COPY ./bin/manager / | ||
CMD ["/manager"] | ||
''' | ||
|
||
def yaml(): | ||
return local('cd config/manager; kustomize edit set image controller=' + IMG + '; cd ../..; kustomize build config/default') | ||
|
||
def manifests(): | ||
return 'controller-gen ' + CONTROLLERGEN | ||
|
||
def generate(): | ||
return 'controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./...";' | ||
|
||
def vetfmt(): | ||
return 'go vet ./...; go fmt ./...' | ||
|
||
def binary(): | ||
return 'CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -o bin/manager main.go' | ||
|
||
installed = local("which kubebuilder") | ||
print("kubebuilder is present:", installed) | ||
|
||
DIRNAME = os.path.basename(os. getcwd()) | ||
# if kubebuilder | ||
if os.path.exists('go.mod') == False: | ||
local("go mod init %s" % DIRNAME) | ||
|
||
if os.path.exists('PROJECT') == False: | ||
local("kubebuilder init --domain %s" % DOMAIN) | ||
|
||
if os.path.exists('api') == False: | ||
local("kubebuilder create api --resource --controller --group %s --version %s --kind %s" % (GROUP, VERSION, KIND)) | ||
|
||
local(manifests() + generate()) | ||
|
||
local_resource('CRD', manifests() + 'kustomize build config/crd | kubectl apply -f -', deps=["api"]) | ||
|
||
k8s_yaml(yaml()) | ||
|
||
deps = ['controllers', 'main.go'] | ||
deps.append('api') | ||
|
||
local_resource('Watch&Compile', generate() + binary(), deps=deps, ignore=['*/*/zz_generated.deepcopy.go']) | ||
|
||
local_resource('Sample YAML', 'kubectl apply -f ./config/samples', deps=["./config/samples"], resource_deps=[DIRNAME + "-controller-manager"]) | ||
|
||
docker_build_with_restart(IMG, '.', | ||
dockerfile_contents=DOCKERFILE, | ||
entrypoint='/manager', | ||
only=['./bin/manager'], | ||
live_update=[ | ||
sync('./bin/manager', '/manager'), | ||
] | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
tilt-restart-wrapper |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
FROM alpine/git | ||
|
||
RUN apk update && apk add make | ||
RUN apk add build-base | ||
|
||
RUN git clone https://github.com/eradman/entr.git /entr | ||
WORKDIR /entr | ||
RUN git checkout c564e6bdca1dfe2177d1224363cad734158863ad | ||
RUN ./configure; CFLAGS="-static" make install | ||
|
||
FROM scratch | ||
|
||
COPY --from=0 /usr/local/bin/entr / | ||
|
||
ADD tilt-restart-wrapper / |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,181 @@ | ||
# Restart Process | ||
|
||
This extension helps create images that can restart on `live_update`: | ||
|
||
- `docker_build_with_restart`: wraps a `docker_build` call | ||
- `custom_build_with_restart`: wraps a `custom_build` call | ||
|
||
At the end of a `live_update`, the container's process will rerun itself. | ||
|
||
(Use it in place of the `restart_container()` Live Update step, which has been deprecated for Kubernetes resources.) | ||
|
||
## When to Use | ||
Use this extension when you have an image and you want to re-execute its entrypoint/command as part of a `live_update`. | ||
|
||
E.g. if your app is a static binary, you'll probably need to re-execute the binary for any changes you made to take effect. | ||
|
||
(If your app has hot reloading capabilities--i.e. it can detect and incorporate changes to its source code without needing to restart--you probably don't need this extension.) | ||
|
||
### Unsupported Cases | ||
This extension does NOT support process restarts for: | ||
- Images built with `custom_build` using any of the `skips_local_docker`, `disable_push`, or `tag` parameters. | ||
- Images run in Docker Compose resources (use the [`restart_container()`](https://docs.tilt.dev/api.html#api.restart_container) builtin instead) | ||
- Images without a shell (e.g. `scratch`, `distroless`) | ||
- Container commands specified as `command` in Kubernetes YAML will be overridden by this extension. | ||
- However, the `args` field is still available; [reach out](https://tilt.dev/contact) if you need help navigating the interplay between Tilt and these YAML values | ||
- CRDs | ||
|
||
If this extension doesn't work for your use case, [see our docs for alternatives](https://docs.tilt.dev/live_update_reference.html#restarting-your-process). | ||
|
||
Run into a bug? Need a use case that we don't yet support? Let us know--[open an issue](https://github.com/tilt-dev/tilt-extensions/issues) or [contact us](https://tilt.dev/contact). | ||
|
||
## How to Use | ||
|
||
Import this extension by putting the following at the top of your Tiltfile: | ||
```python | ||
load('ext://restart_process', 'docker_build_with_restart') | ||
``` | ||
|
||
For the image that needs the process restart, replace your existing `docker_build` call: | ||
```python | ||
docker_build( | ||
'foo-image', | ||
'./foo', | ||
arg1=val1, | ||
arg2=val2, | ||
live_update=[x, y, z...] | ||
) | ||
``` | ||
with a `docker_build_with_restart` call: | ||
```python | ||
docker_build_with_restart( | ||
'foo-image', | ||
'./foo', | ||
entrypoint='/go/bin/foo', | ||
arg1=val1, | ||
arg2=val2, | ||
live_update=[x, y, z...] | ||
) | ||
``` | ||
The call above looks just like the initial `docker_build` call except for one added parameter, `entrypoint` (in this example, `/go/bin/foo`). This is the command that you want to run on container start and _re-run_ on Live Update. | ||
|
||
A custom_build call looks similar: | ||
|
||
```python | ||
load('ext://restart_process', 'custom_build_with_restart') | ||
|
||
custom_build_with_restart( | ||
'foo-image', | ||
'docker build -t $EXPECTED_REF ./foo', | ||
deps=['./foo'], | ||
live_update=[sync(...)] | ||
) | ||
``` | ||
|
||
### Troubleshooting | ||
#### `failed running [touch /tmp/.restart-proc']` | ||
If you see an error of the form: | ||
``` | ||
ERROR: Build Failed: ImageBuild: executor failed running [touch /tmp/.restart-proc']: exit code: 1 | ||
``` | ||
this often means that your Dockerfile user ([see docs](https://docs.docker.com/engine/reference/builder/#user)) doesn't have permission to write to the file we use to signal a process restart. Use the `restart_file` parameter to specify a file that your Dockerfile user definitely has write access to. | ||
|
||
### API | ||
```python | ||
def docker_build_with_restart(ref: str, context: str, | ||
entrypoint: Union[str, List[str]], | ||
live_update: List[LiveUpdateStep], | ||
base_suffix: str = '-base', | ||
restart_file: str = '/.restart-proc', | ||
trigger: Union[str, List[str]] = [], | ||
**kwargs | ||
): | ||
"""Args: | ||
ref: name for this image (e.g. 'myproj/backend' or 'myregistry/myproj/backend'); as the parameter of the same name in docker_build | ||
context: path to use as the Docker build context; as the parameter of the same name in docker_build | ||
entrypoint: the command to be (re-)executed when the container starts or when a live_update is run | ||
live_update: set of steps for updating a running container; as the parameter of the same name in docker_build | ||
base_suffix: suffix for naming the base image, applied as {ref}{base_suffix} | ||
restart_file: file that Tilt will update during a live_update to signal the entrypoint to rerun | ||
trigger: (optional) list of local paths. If specified, the process will ONLY be restarted when there are changes | ||
to the given file(s); as the parameter of the same name in the LiveUpdate `run` step. | ||
**kwargs: will be passed to the underlying `docker_build` call | ||
""" | ||
|
||
|
||
def custom_build_with_restart(ref: str, command: str, deps: List[str], entrypoint, | ||
|
||
entrypoint: Union[str, List[str]], | ||
live_update: List[LiveUpdateStep], | ||
base_suffix: str = '-base', | ||
restart_file: str = '/.restart-proc', | ||
trigger: Union[str, List[str]] = [], | ||
, **kwargs | ||
): | ||
""" | ||
Args: | ||
ref: name for this image (e.g. 'myproj/backend' or 'myregistry/myproj/backend'); as the parameter of the same name in custom_build | ||
command: build command for building your image | ||
deps: source dependencies of the custom build | ||
entrypoint: the command to be (re-)executed when the container starts or when a live_update is run | ||
live_update: set of steps for updating a running container; as the parameter of the same name in custom_build | ||
base_suffix: suffix for naming the base image, applied as {ref}{base_suffix} | ||
restart_file: file that Tilt will update during a live_update to signal the entrypoint to rerun | ||
trigger: (optional) list of local paths. If specified, the process will ONLY be restarted when there are changes | ||
to the given file(s); as the parameter of the same name in the LiveUpdate `run` step. | ||
**kwargs: will be passed to the underlying `custom_build` call | ||
""" | ||
``` | ||
|
||
## What's Happening Under the Hood | ||
*If you're a casual user/just want to get your app running, you can stop reading now. However, if you want to dig deep and know exactly what's going on, or are trying to debug weird behavior, read on.* | ||
|
||
This extension wraps commands in `tilt-restart-wrapper`, which makes use of [`entr`](https://github.com/eradman/entr/) | ||
to run arbitrary commands whenever a specified file changes. Specifically, we override the container's entrypoint with the following: | ||
|
||
``` | ||
/tilt-restart-wrapper --watch_file='/.restart-proc' <entrypoint> | ||
``` | ||
|
||
This invocation says: | ||
- when the container starts, run <entrypoint> | ||
- whenever the `/.restart-proc` file changes, re-execute <entrypoint> | ||
|
||
We also set the following as the last `live_update` step: | ||
```python | ||
run('date > /.restart-proc') | ||
``` | ||
|
||
Because `tilt-restart-wrapper` will re-execute the entrypoint whenever `/.restart-proc'` changes, the above `run` step will cause the entrypoint to re-run. | ||
|
||
#### Provide `tilt-restart-wrapper` | ||
For this all to work, the `entr` binary must be available on the Docker image. The easiest solution would be to call e.g. `apt-get install entr` in the Dockerfile, but different base images will have different package managers; rather than grapple with that, we've made a statically linked binary available on Docker image: [`tiltdev/entr`](https://hub.docker.com/repository/docker/tiltdev/entr). | ||
|
||
To build `image-foo`, this extension will: | ||
- build your image as normal (via `docker_build`, with all of your specified args/kwargs) but with the name `image-foo-base` | ||
- build `image-foo` (the actual image that will be used in your resource) as a _child_ of `image-foo-base`, with the `tilt-process-wrapper` and its dependencies available | ||
|
||
Thus, the final image produced is tagged `image-foo` and has all the properties of your original `docker_build`, plus access to the `tilt-restart-wrapper` binary. | ||
|
||
#### Why a Wrapper? | ||
Why bother with `tilt-restart-wrapper` rather than just calling `entr` directly? | ||
|
||
Because in its canonical invocation, `entr` requires that the file(s) to watch be piped via stdin, i.e. it is invoked like: | ||
``` | ||
echo "/.restart-proc" | entr -rz /bin/my-app | ||
``` | ||
|
||
When specified as a `command` in Kubernetes or Docker Compose YAML (this is how Tilt overrides entrypoints), the above would therefore need to be executed as shell: | ||
``` | ||
/bin/sh -c 'echo "/.restart-proc" | entr -rz /bin/my-app' | ||
``` | ||
Any `args` specified in Kubernetes/Docker Compose are attached to the end of this call, and therefore in this case would apply TO THE `/bin/sh -c` CALL, rather than to the actual command run by `entr`; that is, any `args` specified by the user would be effectively ignored. | ||
|
||
In order to make `entr` usable without a shell, this extension uses [a simple binary](/restart_process/tilt-restart-wrapper.go) that invokes `entr` and writes to its stdin. | ||
|
||
Note: ideally `entr` could accept files-to-watch via flag instead of stdin, but (for a number of good reasons) this feature isn't likely to be added any time soon (see [entr#33](https://github.com/eradman/entr/issues/33)). | ||
|
||
## For Maintainers: Releasing | ||
If you have push access to the `tiltdev` repository on DockerHub, you can release a new version of the binaries used by this extension like so: | ||
1. run `release.sh` (builds `tilt-restart-wrapper` from source, builds and pushes a Docker image with the new binary and a fresh binary of `entr` also installed from source) | ||
2. update the image tag in the [Tiltfile](/restart_process/Tiltfile) with the tag you just pushed (you'll find the image referenced in the Dockerfile contents of the child image--look for "FROM tiltdev/restart-helper") |
Oops, something went wrong.