fix(desktop-backend): fix prod crash and add missing env vars#6042
Conversation
RESEND_API_KEY is not used anywhere in the Rust backend code but was referenced in the default values.yaml pointing to prod-omi-backend-secrets. This key never existed in the K8s secret, causing CreateContainerConfigError whenever the pod restarts (triggered today by GKE node pool upgrade). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The default values.yaml was missing critical env vars (GEMINI_API_KEY, OPENAI_API_KEY, DEEPGRAM_API_KEY, ENCRYPTION_SECRET, REDIS, PINECONE) that the dev_values.yaml had. This adds a proper prod values file matching the dev setup but pointing to prod-omi-backend-secrets. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Greptile SummaryThis PR fixes a production outage on Key changes:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant GKE as GKE Node Pool Upgrade
participant Pod as desktop-backend Pod
participant K8s as Kubernetes Secret<br/>(prod-omi-backend-secrets)
participant App as Rust App (desktop-backend)
GKE->>Pod: Evict pod (node pool upgrade)
Pod->>K8s: Fetch env vars from secret
Note over K8s: Before fix: RESEND_API_KEY key<br/>did not exist in secret
K8s-->>Pod: CreateContainerConfigError ❌
Note over Pod,K8s: After this PR
Pod->>K8s: Fetch env vars (no RESEND_API_KEY)
K8s-->>Pod: GEMINI_API_KEY, OPENAI_API_KEY,<br/>DEEPGRAM_API_KEY, ENCRYPTION_SECRET,<br/>REDIS_DB_HOST, REDIS_DB_PASSWORD,<br/>PINECONE_API_KEY ✅
Pod->>App: Start with all required env vars
App-->>Pod: Healthy ✅
|
| @@ -0,0 +1,106 @@ | |||
| # Production environment values for desktop-backend | |||
There was a problem hiding this comment.
OPENAI_API_KEY injected but not consumed by the Rust backend
OPENAI_API_KEY is included in the secret env vars (line 22–26) but has no corresponding field in config.rs — the Rust binary never reads it. The proxy routes only cover Gemini and Deepgram; no OpenAI client is wired up. This is harmless today but could create confusion (e.g., someone assumes OpenAI calls are forwarded when they are not). If it's intentionally reserved for a future feature, a comment to that effect would help. Otherwise it can be removed to keep secrets minimal.
| - name: PINECONE_API_KEY | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: prod-omi-backend-secrets | ||
| key: PINECONE_API_KEY |
There was a problem hiding this comment.
PINECONE_HOST missing alongside PINECONE_API_KEY
dev_values.yaml sets both PINECONE_API_KEY and PINECONE_HOST as secrets. prod_values.yaml includes the API key but omits the host. In config.rs, pinecone_host is loaded from PINECONE_HOST; without it the field stays None and any Pinecone-backed feature (knowledge graph / vector search) will silently fail in production without any error at startup (the config validation does not warn on a missing Pinecone host).
Consider adding a PINECONE_HOST entry pointing to prod-omi-backend-secrets to match the dev configuration.
…ardware#6042) ## Summary - Removed unused `RESEND_API_KEY` from Helm `values.yaml` — it was never used in the Rust backend code but referenced a nonexistent K8s secret key, causing `CreateContainerConfigError` on pod restart - Added `prod_values.yaml` with all required env vars (GEMINI_API_KEY, OPENAI_API_KEY, DEEPGRAM_API_KEY, ENCRYPTION_SECRET, REDIS, PINECONE) — previously only dev had a proper values file ## Context `desktop-api.omi.me` went down today after a GKE node pool upgrade by thainguyensunya@gmail.com evicted the desktop-backend pod. The replacement pod couldn't start due to the missing `RESEND_API_KEY` in the secret. This was a latent bug since the initial Helm chart import (Feb 12). Also updated the expired Gemini API key in GCP Secret Manager and restarted affected deployments (desktop-backend + backend-listen). **Already deployed to prod** — this PR syncs the repo with what's live. ## Test plan - [x] `desktop-api.omi.me/health` returns healthy - [x] Gemini proxy returns 401 (auth required), not 502 - [x] Verified new Gemini key works via direct API call - [x] Verified backend-listen pods picked up new key 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Summary
RESEND_API_KEYfrom Helmvalues.yaml— it was never used in the Rust backend code but referenced a nonexistent K8s secret key, causingCreateContainerConfigErroron pod restartprod_values.yamlwith all required env vars (GEMINI_API_KEY, OPENAI_API_KEY, DEEPGRAM_API_KEY, ENCRYPTION_SECRET, REDIS, PINECONE) — previously only dev had a proper values fileContext
desktop-api.omi.mewent down today after a GKE node pool upgrade by thainguyensunya@gmail.com evicted the desktop-backend pod. The replacement pod couldn't start due to the missingRESEND_API_KEYin the secret. This was a latent bug since the initial Helm chart import (Feb 12).Also updated the expired Gemini API key in GCP Secret Manager and restarted affected deployments (desktop-backend + backend-listen).
Already deployed to prod — this PR syncs the repo with what's live.
Test plan
desktop-api.omi.me/healthreturns healthy🤖 Generated with Claude Code