Protocol-agnostic load testing library for Go

How it works

Goals

• Easy to reuse any custom client Go code
• Easy to grasp
• Have slim codebase (500-1k loc)
• No test harness or CLI, easy to integrate and run with plain go test
• Have predictable performance footprint when tested with protocol mocks
• Easy to create synthetic or user-based scenarios
• Scalable in k8s without complicated configuration or vendored UI interfaces
• Non-opinionated reporting, push any data to Loki

Setup

We are using nix for deps, see installation guide

nix develop

Run example tests with Grafana + Loki

make start

Insert GRAFANA_TOKEN created in previous command

export LOKI_URL=http://localhost:3030/loki/api/v1/push
export GRAFANA_URL=http://localhost:3000
export GRAFANA_TOKEN=...
export DATA_SOURCE_NAME=Loki
export DASHBOARD_FOLDER=LoadTests
export WASP_LOG_LEVEL=info
make dashboard

Run some tests:

make test_loki

Open your Grafana dashboard

Basic dashboard:

Remove environment:

make stop

Test Layout and examples

Check examples to understand what is the easiest way to structure your tests, run them both locally and remotely, at scale, inside k8s

How it works

Check this doc for more examples and project overview

Run pyroscope test

make pyro_start
make test_pyro_rps
make test_pyro_vu
make pyro_stop

Open pyroscope

You can also use trace.out in the root folder with Go default tracing UI

Loki debug

You can check all the messages the tool sends with env var WASP_LOG_LEVEL=trace

If Loki client fail to deliver a batch test will proceed, if you experience Loki issues, consider setting Timeout in LokiConfig or set MaxErrors: 10 to return an error after N Loki errors

MaxErrors: -1 can be used to ignore all the errors

Default Promtail settings are:

&LokiConfig{
    TenantID:                os.Getenv("LOKI_TENANT_ID"),
    URL:                     os.Getenv("LOKI_URL"),
    Token:                   os.Getenv("LOKI_TOKEN"),
    BasicAuth:               os.Getenv("LOKI_BASIC_AUTH"),
    MaxErrors:               10,
    BatchWait:               5 * time.Second,
    BatchSize:               500 * 1024,
    Timeout:                 20 * time.Second,
    DropRateLimitedBatches:  false,
    ExposePrometheusMetrics: false,
    MaxStreams:              600,
    MaxLineSize:             999999,
    MaxLineSizeTruncate:     false,
}

If you see errors like

ERR Malformed promtail log message, skipping Line=["level",{},"component","client","host","...","msg","batch add err","tenant","","error",{}]

Try to increase MaxStreams even more or check your Loki configuration