Add observability to the rescue example (#2736)

* Adding initial APIs for the example * Adding code for payment-executor-api * Adding full example * Adding trace-based tests * Updating README * Apply suggestions from code review Co-authored-by: Adnan Rahić <adnan@kubeshop.io> * Adding documentation to the example * fix(backend): matching trigger span duration and trigger time durations (#2740) * fix(frontend): adding default value for data store connection types (#2742) * fix(backend): fixing response status code for grpc request (#2741) * feat: add upsert method and enable it for environments (#2746) * add upsert method and enable it for environments * handle error when getting entity * PR suggestions --------- Co-authored-by: Adnan Rahić <adnan@kubeshop.io> Co-authored-by: Oscar Reyes <oscar-rreyes1@hotmail.com> Co-authored-by: Matheus Nogueira <matheus.nogueira2008@gmail.com>
kubeshop · Jun 16, 2023 · 592188a · 592188a
1 parent c519fff
commit 592188a
Show file tree

Hide file tree

Showing 29 changed files with 6,518 additions and 0 deletions.
diff --git a/examples/observability-to-the-rescue/README.md b/examples/observability-to-the-rescue/README.md
@@ -0,0 +1,82 @@
+# Observability to the Rescue! - Conference talk at DeveloperWeek Latin America 2023 by [Daniel Dias](https://github.com/danielbdias)
+
+This repo contains the sample app for the presentation "Observability to the Rescue! Monitoring and testing APIs with OpenTelemetry" at DeveloperWeek Latin America 2023.
+
+Run this example with:
+```sh
+docker compose up
+```
+
+### Requests that you can run this example
+
+Valid payment without risk analysis scenario:
+```sh
+curl --location 'http://localhost:10013/executePaymentOrder' \
+--header 'Content-Type: application/json' \
+--data '{
+    "walletId": 2,
+    "yearsAsACustomer": 1
+}'
+
+# Output
+# {
+#     "status": "executed"
+# }
+```
+
+Valid payment with risk analysis scenario:
+```sh
+curl --location 'http://localhost:10013/executePaymentOrder' \
+--header 'Content-Type: application/json' \
+--data '{
+    "walletId": 4,
+    "yearsAsACustomer": 1
+}'
+
+# Output
+# {
+#     "status": "executed"
+# }
+```
+
+Denied payment scenario:
+```sh
+curl --location 'http://localhost:10013/executePaymentOrder' \
+--header 'Content-Type: application/json' \
+--data '{
+    "walletId": 5,
+    "yearsAsACustomer": 1
+}'
+
+# Output
+# {
+#     "status": "denied"
+# }
+```
+
+Request with error scenario
+```sh
+curl --location 'http://localhost:10013/executePaymentOrder' \
+--header 'Content-Type: application/json' \
+--data '{
+    "walletId": 4,
+    "yearsAsACustomer": 0
+}'
+
+# Output
+# internal error!
+```
+
+## Trace-based tests that you can run
+
+There are two tests that you can do to check how these APIs are working, one is the `test-with-error`, that calls `your-api` passing the `yearsAsACustomer` field as zero, causing a error propagation into the API calls:
+
+```sh
+tracetest test run -w -d ./tracetest/tests/test-with-error.yaml
+```
+
+The second one is `test-with-success`, with the field `yearsAsACustomer` greater than 0, causing the services to behave normally:
+
+```sh
+tracetest test run -w -d ./tracetest/tests/test-with-success.yaml
+```
diff --git a/examples/observability-to-the-rescue/docker-compose.yaml b/examples/observability-to-the-rescue/docker-compose.yaml
@@ -0,0 +1,122 @@
+version: '3'
+
+services:
+  your-api:
+    build: ./your-api
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    environment:
+      - YOUR_API_PORT=10013
+      - OTEL_SERVICE_NAME=your-api
+      - OTEL_EXPORTER_GRPC_URL=http://otel-collector:4317
+      - WALLET_API_ENDPOINT=http://wallet:10010/wallet
+      - PAYMENT_EXECUTOR_API_ENDPOINT=http://payment-executor:10012/payment/execute
+    ports:
+      - 10013:10013
+    depends_on:
+      - otel-collector
+      - payment-executor
+      - wallet
+
+  payment-executor:
+    build: ./payment-executor-api
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    environment:
+      - PAYMENT_EXECUTOR_API_PORT=10012
+      - OTEL_SERVICE_NAME=payment-executor-api
+      - OTEL_EXPORTER_HTTP_URL=http://otel-collector:4318/v1/traces
+      - RISK_ANALYSIS_URL=http://risk-analysis:10011/computeRisk
+    ports:
+      - 10012:10012
+    depends_on:
+      - otel-collector
+      - risk-analysis
+
+  risk-analysis:
+    build: ./risk-analysis-api
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    environment:
+      - RISK_ANALYSIS_API_PORT=10011
+      - OTEL_SERVICE_NAME=risk-analysis-api
+      - OTEL_EXPORTER_GRPC_URL=otel-collector:4317
+    ports:
+      - 10011:10011
+    depends_on:
+      - otel-collector
+
+  wallet:
+    build: ./wallet-api
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    environment:
+      - WALLET_API_PORT=10010
+      - OTEL_SERVICE_NAME=wallet-api
+      - OTEL_EXPORTER_GRPC_URL=http://otel-collector:4317
+    ports:
+      - 10010:10010
+    depends_on:
+      - otel-collector
+
+  tracetest:
+    image: kubeshop/tracetest:latest
+    platform: linux/amd64
+    volumes:
+      - type: bind
+        source: ./tracetest/tracetest-config.yaml
+        target: /app/tracetest.yaml
+      - type: bind
+        source: ./tracetest/tracetest-provision.yaml
+        target: /app/provisioning.yaml
+    ports:
+      - 11633:11633
+    command: --provisioning-file /app/provisioning.yaml
+    depends_on:
+      postgres:
+        condition: service_healthy
+      otel-collector:
+        condition: service_started
+      your-api:
+        condition: service_started
+    healthcheck:
+      test: ["CMD", "wget", "--spider", "localhost:11633"]
+      interval: 1s
+      timeout: 3s
+      retries: 60
+    environment:
+      TRACETEST_DEV: ${TRACETEST_DEV}
+
+  postgres:
+    image: postgres:14
+    environment:
+      POSTGRES_PASSWORD: postgres
+      POSTGRES_USER: postgres
+    healthcheck:
+      test: pg_isready -U "$$POSTGRES_USER" -d "$$POSTGRES_DB"
+      interval: 1s
+      timeout: 5s
+      retries: 60
+
+  otel-collector:
+    image: otel/opentelemetry-collector-contrib:0.59.0
+    command:
+      - "--config"
+      - "/otel-local-config.yaml"
+    volumes:
+      - ./tracetest/collector.config.yaml:/otel-local-config.yaml
+    depends_on:
+      - jaeger
+
+  jaeger:
+    image: jaegertracing/all-in-one:latest
+    restart: unless-stopped
+    ports:
+      - 16686:16686
+    environment:
+      - COLLECTOR_OTLP_ENABLED=true
+    healthcheck:
+      test: ["CMD", "wget", "--spider", "localhost:16686"]
+      interval: 1s
+      timeout: 3s
+      retries: 60
diff --git a/examples/observability-to-the-rescue/payment-executor-api/Dockerfile b/examples/observability-to-the-rescue/payment-executor-api/Dockerfile
@@ -0,0 +1,12 @@
+FROM ruby:3.1.2-alpine
+WORKDIR /usr/src/app
+
+COPY Gemfile Gemfile.lock .
+RUN apk update && apk add make gcc musl-dev && bundle install
+
+COPY . .
+RUN chmod 666 ./Gemfile.lock
+
+EXPOSE 8080
+
+CMD [ "bundle", "exec", "ruby", "/usr/src/app/api.rb" ]
diff --git a/examples/observability-to-the-rescue/payment-executor-api/Gemfile b/examples/observability-to-the-rescue/payment-executor-api/Gemfile
@@ -0,0 +1,9 @@
+source "https://rubygems.org"
+
+gem "puma",     "~> 6.2.1"
+gem "sinatra",  "~> 3.0.5"
+
+# Adding support for Open Telemetry
+gem 'opentelemetry-sdk'
+gem 'opentelemetry-exporter-otlp'
+gem 'opentelemetry-instrumentation-all'
diff --git a/examples/observability-to-the-rescue/payment-executor-api/Gemfile.lock b/examples/observability-to-the-rescue/payment-executor-api/Gemfile.lock
@@ -0,0 +1,65 @@
+GEM
+  remote: https://rubygems.org/
+  specs:
+    google-protobuf (3.22.2-x86_64-darwin)
+    googleapis-common-protos-types (1.5.0)
+      google-protobuf (~> 3.14)
+    mustermann (3.0.0)
+      ruby2_keywords (~> 0.0.1)
+    nio4r (2.5.9)
+    opentelemetry-api (1.1.0)
+    opentelemetry-common (0.19.6)
+      opentelemetry-api (~> 1.0)
+    opentelemetry-exporter-otlp (0.24.0)
+      google-protobuf (~> 3.19)
+      googleapis-common-protos-types (~> 1.3)
+      opentelemetry-api (~> 1.1)
+      opentelemetry-common (~> 0.19.6)
+      opentelemetry-sdk (~> 1.2)
+      opentelemetry-semantic_conventions
+    opentelemetry-instrumentation-base (0.21.1)
+      opentelemetry-api (~> 1.0)
+      opentelemetry-registry (~> 0.1)
+    opentelemetry-instrumentation-rack (0.22.1)
+      opentelemetry-api (~> 1.0)
+      opentelemetry-common (~> 0.19.3)
+      opentelemetry-instrumentation-base (~> 0.21.0)
+    opentelemetry-instrumentation-sinatra (0.21.5)
+      opentelemetry-api (~> 1.0)
+      opentelemetry-common (~> 0.19.3)
+      opentelemetry-instrumentation-base (~> 0.21.0)
+      opentelemetry-instrumentation-rack (~> 0.21)
+    opentelemetry-registry (0.2.0)
+      opentelemetry-api (~> 1.1)
+    opentelemetry-sdk (1.2.0)
+      opentelemetry-api (~> 1.1)
+      opentelemetry-common (~> 0.19.3)
+      opentelemetry-registry (~> 0.2)
+      opentelemetry-semantic_conventions
+    opentelemetry-semantic_conventions (1.8.0)
+      opentelemetry-api (~> 1.0)
+    puma (6.2.1)
+      nio4r (~> 2.0)
+    rack (2.2.6.4)
+    rack-protection (3.0.5)
+      rack
+    ruby2_keywords (0.0.5)
+    sinatra (3.0.5)
+      mustermann (~> 3.0)
+      rack (~> 2.2, >= 2.2.4)
+      rack-protection (= 3.0.5)
+      tilt (~> 2.0)
+    tilt (2.1.0)
+
+PLATFORMS
+  x86_64-darwin-21
+
+DEPENDENCIES
+  opentelemetry-exporter-otlp (~> 0.24)
+  opentelemetry-instrumentation-sinatra (~> 0.21)
+  opentelemetry-sdk (~> 1.2)
+  puma (~> 6.2.1)
+  sinatra (~> 3.0.5)
+
+BUNDLED WITH
+   2.3.12
diff --git a/examples/observability-to-the-rescue/payment-executor-api/api.rb b/examples/observability-to-the-rescue/payment-executor-api/api.rb
@@ -0,0 +1,84 @@
+require "sinatra"
+require "json"
+require "uri"
+require "net/http"
+
+require "opentelemetry/sdk"
+require "opentelemetry/exporter/otlp"
+require "opentelemetry/instrumentation/all"
+
+require "./config"
+
+config = current_config()
+
+set :port, config.port
+set :bind, '0.0.0.0'
+
+OpenTelemetry::SDK.configure do |c|
+  otel_exporter = OpenTelemetry::Exporter::OTLP::Exporter.new(endpoint: config.otel_exporter_http_url)
+  processor = OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(otel_exporter)
+
+  c.service_name = config.otel_service_name
+  c.add_span_processor(processor)
+
+  c.use_all()
+end
+
+error do
+  OpenTelemetry::Trace.current_span.record_exception(env['sinatra.error'])
+end
+
+post '/payment/execute' do
+  content_type :json # should return json
+
+  payment_data = JSON.parse(request.body.read)
+
+  amount = payment_data["amount"]
+  age = payment_data["age"]
+
+  if amount < 10000
+    # don't need to analyze risk
+    execute_payment(amount)
+    return { status: "executed" }.to_json
+  end
+
+  score = call_risk_api(amount, age)
+
+  if score < 0
+    raise "This case should not be happening"
+  end
+
+  if score > 50000
+    return { status: "denied" }.to_json
+  end
+
+  execute_payment(amount)
+  return { status: "executed" }.to_json
+end
+
+def call_risk_api(amount, age)
+  config = current_config()
+  url = URI(config.risk_analysis_url)
+
+  http = Net::HTTP.new(url.host, url.port);
+  request = Net::HTTP::Post.new(url)
+  request["Content-Type"] = "application/json"
+  request.body = JSON.dump({
+    "amount": amount,
+    "age": age
+  })
+
+  response = http.request(request)
+  output = JSON.parse(response.read_body)
+
+  output["score"]
+end
+
+def execute_payment(amount)
+  tracer = OpenTelemetry.tracer_provider.tracer('tracer')
+
+  tracer.in_span("execute_payment", attributes: { "amount" => amount }) do |span|
+    # simulate payment being execuded
+    sleep(0.05) # 50 milliseconds
+  end
+end
diff --git a/examples/observability-to-the-rescue/payment-executor-api/config.rb b/examples/observability-to-the-rescue/payment-executor-api/config.rb
@@ -0,0 +1,10 @@
+Config = Struct.new(:port, :otel_service_name, :otel_exporter_http_url, :risk_analysis_url)
+
+def current_config
+  Config.new(
+    ENV["PAYMENT_EXECUTOR_API_PORT"],
+    ENV["OTEL_SERVICE_NAME"],
+    ENV["OTEL_EXPORTER_HTTP_URL"],
+    ENV["RISK_ANALYSIS_URL"],
+  )
+end