From caf7923b744acb302783cb2691f52f51746c2785 Mon Sep 17 00:00:00 2001 From: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com> Date: Thu, 5 Feb 2026 10:21:51 -0600 Subject: [PATCH 1/3] Added contributor docs for SigNoz. --- .../configuration-for-contributors.md | 2 + tools/README.md | 1 + tools/signoz/README.md | 93 +++++++++++++++++++ tools/telemetry/README.md | 4 + 4 files changed, 100 insertions(+) create mode 100644 tools/signoz/README.md diff --git a/docs/Contributing/reference/configuration-for-contributors.md b/docs/Contributing/reference/configuration-for-contributors.md index 69f564b7d74..b0d343d5abf 100644 --- a/docs/Contributing/reference/configuration-for-contributors.md +++ b/docs/Contributing/reference/configuration-for-contributors.md @@ -229,6 +229,8 @@ By default, OpenTelemetry is used. Set `tracing_type` to `elasticapm` only if yo Enables exporting logs to an OpenTelemetry collector in addition to stderr output. When enabled, logs are sent to the OTLP endpoint configured via the standard `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable. Logs are automatically correlated with traces via `trace_id` and `span_id` attributes. +> **Note:** All log levels, including debug, are always sent to the OpenTelemetry collector regardless of the `logging.debug` setting. The `logging.debug` flag only controls what appears in stderr output. + > **Note:** This option requires `logging.tracing_enabled` to be set to `true`. Fleet will fail to start if `otel_logs_enabled` is `true` but `tracing_enabled` is `false`. - Default value: `false` diff --git a/tools/README.md b/tools/README.md index 1e308f8dfc9..9ae29becb55 100644 --- a/tools/README.md +++ b/tools/README.md @@ -242,6 +242,7 @@ go run ./tools/run-scripts -scripts-disabled -content 'echo "Test"' | `percona/` | Percona testing | Percona MySQL testing configs - See [percona/test/README.md](percona/test/README.md) | | `sentry-self-hosted/` | Self-hosted Sentry | See [sentry-self-hosted/README.md](sentry-self-hosted/README.md) | | `smtp4dev/` | Local SMTP testing | SMTP4Dev server with TLS certs for email testing | +| `signoz/` | SigNoz for traces, metrics, and logs | See [signoz/README.md](signoz/README.md) | | `telemetry/` | Jaeger + Prometheus for tracing | `docker compose up` - See [telemetry/README.md](telemetry/README.md) | | `terraform/` | Terraform provider for Fleet teams | `make install && make apply` - See [terraform/README.md](terraform/README.md) | | **MDM Tools** | | | diff --git a/tools/signoz/README.md b/tools/signoz/README.md new file mode 100644 index 00000000000..d2af85bf355 --- /dev/null +++ b/tools/signoz/README.md @@ -0,0 +1,93 @@ +# Running SigNoz locally with Fleet + +[SigNoz](https://signoz.io/) is an open-source observability platform that provides traces, metrics, and logs in a single UI. This guide explains how to run SigNoz locally for Fleet development with optimized settings for reduced latency. + +## Prerequisites + +- Docker and Docker Compose +- A locally-built Fleet server (see [Testing and local development](../../docs/Contributing/getting-started/testing-and-local-development.md)) + +## Setup + +1. Clone the SigNoz repository at a specific release: + +```bash +git clone --branch v0.110.1 --depth 1 https://github.com/SigNoz/signoz.git +cd signoz/deploy +``` + +2. Modify the SigNoz UI port to avoid conflict with Fleet (which uses port 8080): + +In `docker/docker-compose.yaml`, change the signoz service port mapping: + +```yaml +services: + signoz: + ports: + - "8085:8080" # Changed from 8080:8080 to avoid conflict with Fleet +``` + +3. (Optional) For reduced latency during development, modify `docker/otel-collector-config.yaml`: + +```yaml +processors: + batch: + send_batch_size: 10000 + send_batch_max_size: 11000 + timeout: 200ms # reduced from 10s for dev + # ... + signozspanmetrics/delta: + # ... + metrics_flush_interval: 5s # reduced from 60s for dev +``` + +4. Start SigNoz: + +```bash +cd docker +docker compose up -d +``` + +Give it a minute for all services to initialize. The SigNoz UI will be available at http://localhost:8085. + +## Configuring Fleet + +Start the Fleet server with OpenTelemetry tracing and logging enabled: + +```bash +export FLEET_LOGGING_TRACING_ENABLED=true +export FLEET_LOGGING_OTEL_LOGS_ENABLED=true +export OTEL_SERVICE_NAME=fleet +export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 + +./build/fleet serve --dev +``` + +> **Note:** All log levels (including debug) are always sent to SigNoz regardless of the `--logging_debug` flag. That flag only controls stderr output. + +### Low-latency configuration (optional) + +For faster feedback during development, you can reduce the batch processing delays on the Fleet side: + +```bash +# Tracing and logging +export FLEET_LOGGING_TRACING_ENABLED=true +export FLEET_LOGGING_OTEL_LOGS_ENABLED=true +export OTEL_SERVICE_NAME=fleet +export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 + +# Batch span processor delay (default 5000ms) +export OTEL_BSP_SCHEDULE_DELAY=1000 + +# Log batch processor settings +export OTEL_BLRP_EXPORT_TIMEOUT=1000 +export OTEL_BLRP_SCHEDULE_DELAY=500 +export OTEL_BLRP_MAX_EXPORT_BATCH_SIZE=1 + +./build/fleet serve --dev +``` + +## Using SigNoz + +After starting Fleet with the above configuration, you should start seeing traces, logs, and metrics in SigNoz UI at http://localhost:8085. + diff --git a/tools/telemetry/README.md b/tools/telemetry/README.md index 940c8f267c7..6dfc6b75608 100644 --- a/tools/telemetry/README.md +++ b/tools/telemetry/README.md @@ -20,3 +20,7 @@ OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" \ ``` Afterwards, you can navigate to http://localhost:16686/ to access the Jaeger UI. + +## Alternatives + +For a more full-featured observability platform that includes traces, metrics, and logs in a single UI, see [SigNoz](../signoz/README.md). From 641b9b2f4bb08f33cc25c4ff753642876ede900d Mon Sep 17 00:00:00 2001 From: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com> Date: Thu, 5 Feb 2026 10:23:14 -0600 Subject: [PATCH 2/3] Minor cleanup --- tools/signoz/README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tools/signoz/README.md b/tools/signoz/README.md index d2af85bf355..80e83bb52a6 100644 --- a/tools/signoz/README.md +++ b/tools/signoz/README.md @@ -60,7 +60,7 @@ export FLEET_LOGGING_OTEL_LOGS_ENABLED=true export OTEL_SERVICE_NAME=fleet export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 -./build/fleet serve --dev +./build/fleet serve ``` > **Note:** All log levels (including debug) are always sent to SigNoz regardless of the `--logging_debug` flag. That flag only controls stderr output. @@ -84,10 +84,9 @@ export OTEL_BLRP_EXPORT_TIMEOUT=1000 export OTEL_BLRP_SCHEDULE_DELAY=500 export OTEL_BLRP_MAX_EXPORT_BATCH_SIZE=1 -./build/fleet serve --dev +./build/fleet serve ``` ## Using SigNoz After starting Fleet with the above configuration, you should start seeing traces, logs, and metrics in SigNoz UI at http://localhost:8085. - From 25111465c8a8733f1ac00c782172ec2babe5986f Mon Sep 17 00:00:00 2001 From: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com> Date: Thu, 5 Feb 2026 10:24:15 -0600 Subject: [PATCH 3/3] Minor cleanup --- tools/signoz/README.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/tools/signoz/README.md b/tools/signoz/README.md index 80e83bb52a6..8514591b3dc 100644 --- a/tools/signoz/README.md +++ b/tools/signoz/README.md @@ -70,12 +70,6 @@ export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 For faster feedback during development, you can reduce the batch processing delays on the Fleet side: ```bash -# Tracing and logging -export FLEET_LOGGING_TRACING_ENABLED=true -export FLEET_LOGGING_OTEL_LOGS_ENABLED=true -export OTEL_SERVICE_NAME=fleet -export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 - # Batch span processor delay (default 5000ms) export OTEL_BSP_SCHEDULE_DELAY=1000