This application is used for demonstrating the application instrumentation, especially trace and metrics.
To enable new relic instrumentation:
-
Create free newrelic account and grab the license key.
-
Set these following env var:
OTEL_NEW_RELIC_EXPORTER_OTLP_ENDPOINT
tohttps://otlp.nr-data.net:4317
OTEL_NEW_RELIC_EXPORTER_API_KEY
to ingest license key from newrelic account
-
Create GCP service account and ensure that service account has (at minimum)
roles/monitoring.metricWriter
androles/cloudtrace.agent
role. -
Update value of
exporters.googlecloud.project
in otel-config file to your google cloud project id/
```shell
docker-compose up
```
- Jaeger Tracing Dashboard: http://127.0.0.1:16686/
- Prometheus Dashboard: http://127.0.0.1:9090/
- Grafana Dashboard: http://127.0.0.1:3000/
- Asynqmon Dashboard: http://127.0.0.1:8011/
- Newrelic dashboard (optional)
- Google Cloud Monitoring
- Google Cloud Tracing
-
Install hey
-
Run load generator
hey -z 1m -c 2 -q 2 -m POST -d '{"show_id": "b9b0d5da-98a4-4b09-b5f5-83dc0c3b9964"}' http://localhost:9999/api/v1/booking hey -z 1m -c 2 -q 2 -m POST -d '{"show_id": "b9b0d5da-98a4-4b09-b5f5-83dc0c3b9964"}' http://localhost:9999/api/v2/booking
WARNING: This setup is not properly documented.
-
Setup CloudSQL Postgres Instance. Update
secret.yaml.example
,app.yaml
, andworker.yaml
with correct value for database config.docker run -d \ -v ${PWD}/hack/observability/secrets/psql-local.json:/config/key.json \ -p 127.0.0.1:5432:5432 \ gcr.io/cloudsql-docker/gce-proxy:1.31.0 /cloud_sql_proxy \ -instances=io-extended-2022:asia-southeast1:booking=tcp:0.0.0.0:5432 -credential_file=/config/key.json
-
Install bitnami/helm chart
helm repo add bitnami https://charts.bitnami.com/bitnami helm install redis --namespace redis --values redis-values.yml bitnami/redis
-
Apply all manifest in
k8s
directory. -
Port forward prometheus dashboard
kubectl -n gmp-test port-forward svc/frontend 9091:9090
-
Run load test
k apply -f k8s/hey.yaml
hey -n 1 -c 1 -q 1 -m POST -d '{"show_id": "b9b0d5da-98a4-4b09-b5f5-83dc0c3b9964"}' http://localhost:9999/api/v1/booking
hey -z 5s -c 100 -m POST -d '{"show_id": "b9b0d5da-98a4-4b09-b5f5-83dc0c3b9964"}' http://localhost:9999/api/v1/booking