diff --git a/assets/workflows/observability/tilebox-jaeger.png b/assets/workflows/observability/tilebox-jaeger.png new file mode 100644 index 0000000..fcef7b9 Binary files /dev/null and b/assets/workflows/observability/tilebox-jaeger.png differ diff --git a/workflows/observability/open-telemetry.mdx b/workflows/observability/open-telemetry.mdx index 49b2bce..02416f5 100644 --- a/workflows/observability/open-telemetry.mdx +++ b/workflows/observability/open-telemetry.mdx @@ -34,10 +34,107 @@ Tilebox Workflows currently supports OpenTelemetry for tracing and logging, with ## Integrations -Tilebox exports telemetry data using the [OpenTelemetry Protocol](https://opentelemetry.io/docs/specs/otlp/). +Tilebox exports telemetry data using the [OpenTelemetry Protocol](https://opentelemetry.io/docs/specs/otlp/). This allows you to send telemetry data to any OpenTelemetry-compatible backend, such as Axiom or Jaeger. + +### Axiom Tilebox is pre-integrated with Axiom -Axiom, a cloud-based observability and telemetry platform, supports this protocol. Tilebox Workflows has built-in support for Axiom, and the examples and screenshots in this section come from this integration. +Tilebox Workflows has built-in support for Axiom, a cloud-based observability and telemetry platform. The examples and screenshots in this section come from this integration. + +To get started, [sign up for a free Axiom account](https://axiom.co/), create an axiom dataset for traces and logs, and generate an API key with ingest permissions for those datasets. + +You can then configure Tilebox to export traces and logs to Axiom using [configure_otel_tracing_axiom](/workflows/observability/tracing#axiom) and [configure_otel_logging_axiom](/workflows/observability/logging#axiom). + +### Jaeger + +Tilebox works well with Jaeger + +Another popular option is [Jaeger](https://jaegertracing.io/), a popular distributed tracing system. You can use the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) to collect telemetry data from Tilebox Workflows and export it to Jaeger. + +An all-in-one Jaeger environment can be spun up using Docker: + +```bash +docker run --rm --name jaeger \ + -p 5778:5778 \ + -p 16686:16686 \ + -p 4318:4318 \ + jaegertracing/jaeger:2.9.0 +``` + +You can then configure Tilebox to export traces to Jaeger using `configure_otel_tracing`. + + +```python Python +from tilebox.workflows import Client +from tilebox.workflows.observability.tracing import configure_otel_tracing + +# your own workflow: +from my_workflow import MyTask + +def main(): + configure_otel_tracing( + # export traces to the local Jaeger instance + endpoint="http://localhost:4318", + ) + + # the following task runner generates traces for executed tasks and + # exports trace and span data to the specified endpoint + client = Client() + runner = client.runner(tasks=[MyTask]) + runner.run_forever() + +if __name__ == "__main__": + main() +``` +```go Go +package main + +import ( + "context" + "log/slog" + + "github.com/tilebox/tilebox-go/examples/workflows/opentelemetry" + "github.com/tilebox/tilebox-go/observability" + "github.com/tilebox/tilebox-go/observability/tracer" + "github.com/tilebox/tilebox-go/workflows/v1" + "go.opentelemetry.io/otel" +) + +// specify a service name and version to identify the instrumenting application in traces and logs +var service = &observability.Service{Name: "task-runner", Version: "dev"} + +func main() { + ctx := context.Background() + + endpoint := "http://localhost:4318" + + // Setup an OpenTelemetry trace span processor, exporting traces and spans to an OTEL compatible trace endpoint + tileboxTracerProvider, shutdown, err := tracer.NewOtelProvider(ctx, service, tracer.WithEndpointURL(endpoint)) + defer shutdown(ctx) + if err != nil { + slog.Error("failed to set up otel span processor", slog.Any("error", err)) + return + } + otel.SetTracerProvider(tileboxTracerProvider) // set the tilebox tracer provider as the global OTEL tracer provider + + client := workflows.NewClient() + + taskRunner, err := client.NewTaskRunner(ctx) + if err != nil { + slog.Error("failed to create task runner", slog.Any("error", err)) + return + } + + err = taskRunner.RegisterTasks(&MyTask{}) + if err != nil { + slog.Error("failed to register tasks", slog.Any("error", err)) + return + } + + taskRunner.RunForever(ctx) +} +``` + -Additionally, any other OpenTelemetry-compatible backend, such as OpenTelemetry Collector or Jaeger, can be used to collect telemetry data generated by Tilebox Workflows. +The generated workflow traces can then be viewed in the Jaeger UI at [http://localhost:16686](http://localhost:16686).