Add a Tiltfile for fast LiteLLM Proxy development with Tilt

README.md

@@ -368,6 +368,67 @@ Step 4: Submit a PR with your changes! 🚀
 - push your fork to your GitHub repo
 - submit a PR from there
 
+### Developing LiteLLM Proxy with Tilt
+
+You can use [Tilt](https://tilt.dev/) to automatically build a Docker image and
+deploy it to a Kubernetes environment (typically a small development-only
+Kubernetes environment running on your development machine) when you change
+files.
+
+#### Create a local Kubernetes cluster for Tilt to deploy to
+
+If you don't yet have a local Kubernetes environment, here is one way to get a
+Tilt-friendly local Kubernetes environment on a Mac:
+
+```shell
+brew install ctlptl kind
+ctlptl create cluster kind --registry=ctlptl-registry
+```
+
+If this works then it will add a `kind-kind` context to your `~/.kube/config`
+and select it as the current context - e.g.:
+
+```shell
+$ kubectl config get-contexts
+CURRENT   NAME                   CLUSTER                AUTHINFO               NAMESPACE
+...
+*         kind-kind              kind-kind              kind-kind              
+...
+```
+
+#### Use Tilt
+
+A `Tiltfile` is provided, so just run:
+
+```shell
+tilt up
+```
+
+and then hit the space bar to open the Tilt UI in your browser.
+
+Tilt will watch for changes and will rebuild the Docker image and redeploy
+LiteLLM Proxy and Postgres (via the Helm chart at `deploy/charts/litellm-helm`)
+to whatever Kubernetes context you have configured. If you're running a local
+Kubernetes cluster (arguably the most convenient way to use Tilt), then you can
+probably access LiteLLM proxy at http://localhost:4000. The `Tiltfile` sets the LiteLLM Proxy master key to `sk-master` so you can likely do something like this:
+
+```shell
+curl -sSL \
+    --request POST \
+    --url 'http://localhost:4000/chat/completions' \
+    --header "Authorization: Bearer sk-master" \
+    --header 'Content-Type: application/json' \
+    --data '{
+        "model": "fake-openai-endpoint",
+        "messages": [
+            {
+                "role": "user",
+                "content": "Kinda pointless as we are accessing a fake endpoint :-)"
+            }
+        ]
+    }'
+```
+
 ### Building LiteLLM Docker Image 
 
 Follow these instructions if you want to build / run the LiteLLM Docker Image yourself.

Tiltfile

@@ -0,0 +1,47 @@
+# Tiltfile
+#
+# for LiteLLM Proxy
+#
+# [Tilt](https://tilt.dev/) is a local development platform that makes it
+# easy to develop applications for Kubernetes. Tilt automates all the
+# steps from a code change to a new container image to a fresh pod.
+#
+# Tiltfiles are written in Starlark, a Python-inspired language.
+
+load('ext://uibutton', 'cmd_button', 'text_input', 'choice_input', 'location')
+
+secret_settings(disable_scrub=True)
+
+docker_build(
+    ref='litellm:some-tag',
+    dockerfile='docker/Dockerfile.non_root',
+    context='.',
+)
+
+# The helm built-in function lets you load from a chart on your filesystem.
+#
+# Calling helm() runs helm template on a chart directory and returns a blob of
+# the Kubernetes YAML, which you can then deploy with k8s_yaml.
+#
+yaml = helm(
+    './deploy/charts/litellm-helm',
+    name='litellm-proxy',
+    set=[
+        "image.repository=litellm",
+        "image.tag=some-tag",
+        "masterkey=sk-master",
+    ],
+)
+k8s_yaml(yaml)
+
+k8s_resource(
+    'litellm-proxy',
+    # map one or more local ports to ports on your Pod; first number is local port, second is container port
+    port_forwards=['4000:4000'],
+)
+
+k8s_resource(
+    'litellm-proxy-postgresql',
+    # map one or more local ports to ports on your Pod; first number is local port, second is container port
+    port_forwards=['5432:5432'],
+)