Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,8 @@ By default, OpenTelemetry is used. Set `tracing_type` to `elasticapm` only if yo

Enables exporting logs to an OpenTelemetry collector in addition to stderr output. When enabled, logs are sent to the OTLP endpoint configured via the standard `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable. Logs are automatically correlated with traces via `trace_id` and `span_id` attributes.

> **Note:** All log levels, including debug, are always sent to the OpenTelemetry collector regardless of the `logging.debug` setting. The `logging.debug` flag only controls what appears in stderr output.

> **Note:** This option requires `logging.tracing_enabled` to be set to `true`. Fleet will fail to start if `otel_logs_enabled` is `true` but `tracing_enabled` is `false`.

- Default value: `false`
Expand Down
1 change: 1 addition & 0 deletions tools/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,7 @@ go run ./tools/run-scripts -scripts-disabled -content 'echo "Test"'
| `percona/` | Percona testing | Percona MySQL testing configs - See [percona/test/README.md](percona/test/README.md) |
| `sentry-self-hosted/` | Self-hosted Sentry | See [sentry-self-hosted/README.md](sentry-self-hosted/README.md) |
| `smtp4dev/` | Local SMTP testing | SMTP4Dev server with TLS certs for email testing |
| `signoz/` | SigNoz for traces, metrics, and logs | See [signoz/README.md](signoz/README.md) |
| `telemetry/` | Jaeger + Prometheus for tracing | `docker compose up` - See [telemetry/README.md](telemetry/README.md) |
| `terraform/` | Terraform provider for Fleet teams | `make install && make apply` - See [terraform/README.md](terraform/README.md) |
| **MDM Tools** | | |
Expand Down
86 changes: 86 additions & 0 deletions tools/signoz/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Running SigNoz locally with Fleet

[SigNoz](https://signoz.io/) is an open-source observability platform that provides traces, metrics, and logs in a single UI. This guide explains how to run SigNoz locally for Fleet development with optimized settings for reduced latency.

## Prerequisites

- Docker and Docker Compose
- A locally-built Fleet server (see [Testing and local development](../../docs/Contributing/getting-started/testing-and-local-development.md))

## Setup

1. Clone the SigNoz repository at a specific release:

```bash
git clone --branch v0.110.1 --depth 1 https://github.com/SigNoz/signoz.git
cd signoz/deploy
```

2. Modify the SigNoz UI port to avoid conflict with Fleet (which uses port 8080):

In `docker/docker-compose.yaml`, change the signoz service port mapping:

```yaml
services:
signoz:
ports:
- "8085:8080" # Changed from 8080:8080 to avoid conflict with Fleet
```

3. (Optional) For reduced latency during development, modify `docker/otel-collector-config.yaml`:

```yaml
processors:
batch:
send_batch_size: 10000
send_batch_max_size: 11000
timeout: 200ms # reduced from 10s for dev
# ...
signozspanmetrics/delta:
# ...
metrics_flush_interval: 5s # reduced from 60s for dev
```

4. Start SigNoz:

```bash
cd docker
docker compose up -d
```

Give it a minute for all services to initialize. The SigNoz UI will be available at http://localhost:8085.

## Configuring Fleet

Start the Fleet server with OpenTelemetry tracing and logging enabled:

```bash
export FLEET_LOGGING_TRACING_ENABLED=true
export FLEET_LOGGING_OTEL_LOGS_ENABLED=true
export OTEL_SERVICE_NAME=fleet
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

./build/fleet serve
```

> **Note:** All log levels (including debug) are always sent to SigNoz regardless of the `--logging_debug` flag. That flag only controls stderr output.

### Low-latency configuration (optional)

For faster feedback during development, you can reduce the batch processing delays on the Fleet side:

```bash
# Batch span processor delay (default 5000ms)
export OTEL_BSP_SCHEDULE_DELAY=1000

# Log batch processor settings
export OTEL_BLRP_EXPORT_TIMEOUT=1000
export OTEL_BLRP_SCHEDULE_DELAY=500
export OTEL_BLRP_MAX_EXPORT_BATCH_SIZE=1

./build/fleet serve
```

## Using SigNoz

After starting Fleet with the above configuration, you should start seeing traces, logs, and metrics in SigNoz UI at http://localhost:8085.
4 changes: 4 additions & 0 deletions tools/telemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,7 @@ OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" \
```

Afterwards, you can navigate to http://localhost:16686/ to access the Jaeger UI.

## Alternatives

For a more full-featured observability platform that includes traces, metrics, and logs in a single UI, see [SigNoz](../signoz/README.md).
Loading