K8S Deploy Updates: Combine Celery+Julia, Cron Restart of Pods #658
Conversation
This will log requests hitting the Julia HTTP server making it a little more obvious what's happening in the logs.
- Add some missing variables needed even for this basic restart task. - Wait for rollout restarts to complete so we know if they've been successful or not.
Combined Celery+Julia Pods and Cron-job rolling restarts
There was a problem hiding this comment.
Pull Request Overview
This PR updates the Kubernetes deployment configuration to combine Celery and Julia containers into a single pod and introduces a cron job for automated pod restarts. The changes optimize resource allocation and improve deployment management.
Key changes:
- Combines Celery and Julia services into a single deployment pod
- Reduces resource allocation across multiple configuration files
- Adds automated Jenkins pipeline for periodic pod restarts
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| julia_src/http.jl | Adds access logging to Julia HTTP server |
| config/gunicorn.conf.py | Reduces worker count from 8 to 4 for K8S deployments |
| Jenkinsfile-restart-celery-julia | New Jenkins pipeline for automated pod restarts via cron |
| .helm/values.yaml | Updates resource requests and reduces Julia replicas |
| .helm/values.staging.yaml | Removes redundant Julia CPU configuration |
| .helm/values.production.yaml | Reduces Julia replicas and memory allocation |
| .helm/templates/celery-deployment.yaml | Adds Julia container to Celery deployment pod |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| env: | ||
| - name: JULIA_HOST | ||
| value: "localhost" |
There was a problem hiding this comment.
Setting JULIA_HOST to 'localhost' hardcodes the assumption that Julia runs in the same pod. Consider using a configurable value or documenting this coupling to avoid confusion during future maintenance.
| env: | |
| - name: JULIA_HOST | |
| value: "localhost" | |
| value: {{ quote .Values.juliaHost }} | |
| # NOTE: The default value assumes Julia runs in the same pod as Celery. | |
| # Change .Values.juliaHost in values.yaml if Julia is deployed elsewhere. |
| HTTP.register!(ROUTER, "GET", "/get_ashp_defaults", get_ashp_defaults) | ||
| HTTP.register!(ROUTER, "GET", "/pv_cost_defaults", pv_cost_defaults) | ||
| HTTP.serve(ROUTER, "0.0.0.0", 8081, reuseaddr=true) | ||
| HTTP.serve(ROUTER, "0.0.0.0", 8081, reuseaddr=true, access_log=combined_logfmt) |
There was a problem hiding this comment.
The variable 'combined_logfmt' is referenced but not defined in this diff. Ensure this variable is properly imported or defined elsewhere in the file to avoid runtime errors.
PR: