observability.md

Observability

Health

Readiness and liveness endpoints are available on /readyz and /livez to be consumed by Kubernetes or your cloud health checker. Additionally, a tool is located in hack/healthcheck.js to be consumed by Docker's native health check feature.

Logs

All logs are on STDOUT. They can be scraped by collectors. The log level can be changed with the environment variable LOG_LEVEL, take a look at the pino docs for available values.

Metrics

All metrics are accessible on /metrics. They can be scraped by collectors.

Traces

Traces are sent to a Jaeger instance or any compatible agent. You can use the environment variable OTEL_EXPORTER_JAEGER_ENDPOINT to change the Jaeger endpoint (eg. http://jaeger:14268/api/traces).

It is very easy to replace Jaeger with Zipkin or another exporter in ./src/observability/tracer.js.

Tools

This is a non-exhaustive list of OSS tools you can use for observability, there is a more exhaustive list.

Observability is a hot topic and many tools evolve rapidly towards supporting all three of metrics, logs, traces.

Collection

• Fluent Bit metrics logs
• Grafana Agent metrics logs traces
• Jaeger traces
• Logstash logs
• OpenTelemetry Collector metrics logs traces
• Prometheus metrics
• Promtail logs
• Telegraf metrics logs traces
• Vector metrics logs traces

Storage

• Elasticsearch metrics logs traces
• Jaeger traces
• Loki logs
• Prometheus metrics
• Tempo traces

Visualization

• Grafana metrics logs traces
• Jaeger traces
• Kibana metrics logs traces

---

Develop - Release - Deploy - Observe