Skip to content

feat: add Prometheus metrics and Grafana dashboard for observability #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alez007 merged 2 commits into from
Apr 7, 2026
Merged

feat: add Prometheus metrics and Grafana dashboard for observability #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5f87889
feat: add Prometheus metrics and Grafana dashboard for observability
Apr 6, 2026
cfdfef3
fix: correct metric names in dashboard and route vLLM metrics through…
Apr 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .devcontainer/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@
"--shm-size=8g",
"--gpus=all",
"-p", "0.0.0.0:8000:8000",
"-p", "0.0.0.0:8265:8265"
"-p", "0.0.0.0:8265:8265",
"-p", "0.0.0.0:8079:8079"
],
"forwardPorts": [8000, 8265],
"forwardPorts": [8000, 8265, 8079],
"portsAttributes": {
"8000": { "label": "API" },
"8265": { "label": "Ray Dashboard" }
Expand All @@ -31,7 +32,8 @@
},
"remoteEnv": {
"HF_TOKEN": "${localEnv:HF_TOKEN}",
"YASHA_PLUGINS": "${localEnv:YASHA_PLUGINS}"
"YASHA_PLUGINS": "${localEnv:YASHA_PLUGINS}",
"YASHA_METRICS": "true"
},
"customizations": {
"vscode": {
Expand Down
8 changes: 7 additions & 1 deletion Dockerfile.dev
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ ENV RAY_CLUSTER_ADDRESS=0.0.0.0
ENV RAY_HEAD_CPU_NUM=2
ENV RAY_HEAD_GPU_NUM=1
ENV YASHA_USE_EXISTING_RAY_CLUSTER=false
ENV YASHA_METRICS=true
ENV RAY_METRICS_EXPORT_PORT=8079
RUN uv venv

ARG PYTHON_VERSION
Expand All @@ -78,7 +80,11 @@ uv sync --project /yasha --locked \$EXTRAS
cd /yasha && uv run pre-commit install

if [ "\${YASHA_USE_EXISTING_RAY_CLUSTER}" != "true" ]; then
cd /yasha && ray start --head --port=\${RAY_REDIS_PORT} --dashboard-host=0.0.0.0 --num-cpus=\${RAY_HEAD_CPU_NUM} --num-gpus=\${RAY_HEAD_GPU_NUM} --disable-usage-stats
METRICS_FLAG=""
if [ "\${YASHA_METRICS}" = "true" ]; then
METRICS_FLAG="--metrics-export-port=\${RAY_METRICS_EXPORT_PORT}"
fi
cd /yasha && ray start --head --port=\${RAY_REDIS_PORT} --dashboard-host=0.0.0.0 --num-cpus=\${RAY_HEAD_CPU_NUM} --num-gpus=\${RAY_HEAD_GPU_NUM} --disable-usage-stats \${METRICS_FLAG}
if ! ray status --address=\${RAY_CLUSTER_ADDRESS}:\${RAY_REDIS_PORT}; then
echo "ray cluster failed to start"
exit 1
Expand Down
8 changes: 7 additions & 1 deletion Dockerfile.prod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ ENV RAY_CLUSTER_ADDRESS=0.0.0.0
ENV RAY_HEAD_CPU_NUM=2
ENV RAY_HEAD_GPU_NUM=1
ENV YASHA_USE_EXISTING_RAY_CLUSTER=false
ENV YASHA_METRICS=true
ENV RAY_METRICS_EXPORT_PORT=8079
RUN uv venv

ARG PYTHON_VERSION
Expand All @@ -77,7 +79,11 @@ fi
uv sync --project /yasha --locked \$EXTRAS

if [ "\${YASHA_USE_EXISTING_RAY_CLUSTER}" != "true" ]; then
cd /yasha && ray start --head --port=\${RAY_REDIS_PORT} --dashboard-host=0.0.0.0 --num-cpus=\${RAY_HEAD_CPU_NUM} --num-gpus=\${RAY_HEAD_GPU_NUM} --disable-usage-stats
METRICS_FLAG=""
if [ "\${YASHA_METRICS}" = "true" ]; then
METRICS_FLAG="--metrics-export-port=\${RAY_METRICS_EXPORT_PORT}"
fi
cd /yasha && ray start --head --port=\${RAY_REDIS_PORT} --dashboard-host=0.0.0.0 --num-cpus=\${RAY_HEAD_CPU_NUM} --num-gpus=\${RAY_HEAD_GPU_NUM} --disable-usage-stats \${METRICS_FLAG}
if ! ray status --address=\${RAY_CLUSTER_ADDRESS}:\${RAY_REDIS_PORT}; then
echo "ray cluster failed to start"
exit 1
Expand Down
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,11 @@ For a full guide on writing your own plugin, see [Plugin Development](docs/plugi
- [Architecture](docs/architecture.md) — system design, request lifecycle, plugin loading
- [Plugin Development](docs/plugins.md) — writing custom TTS backends
- [Home Assistant Integration](docs/home-assistant.md) — Wyoming protocol setup for voice automation
- [Monitoring](docs/monitoring.md) — Prometheus metrics, Grafana dashboard, health checks

## Monitoring

Yasha exposes Prometheus metrics (Ray cluster, Ray Serve, vLLM, and custom `yasha:*` metrics) through a single port. Enable with `YASHA_METRICS=true` and scrape port 8079. A pre-built Grafana dashboard is included. See [Monitoring](docs/monitoring.md) for setup details.

## Future Work

Expand Down
Loading
Loading