Testing

Testing

This page covers the SDK's own test suite and verification commands (for contributors), and the two patterns that make testing your app's integration deterministic: direct-data mode and a capturing log sink.

Verification commands

Run these before opening a PR. Each maps to a gate the Quality Checks workflow enforces.

Command	What it enforces
bundle exec rspec	The full RSpec suite on the current Ruby, with the coverage gates: an 85% global line floor, and ≥ 95% line + branch coverage on the critical algorithm group (Hashing, Bucketing, Rules). Coverage is single-sourced in spec/spec_helper.rb.
bundle exec rake	The default task: RSpec plus RuboCop.
bundle exec rbs -r net-http -r uri -r json -I sig validate	RBS signature validity (CRuby-only).
bundle exec steep check	Static type check of sig/ against lib/ (CRuby-only).
DISABLE_COVERAGE=1 bundle exec rspec spec/cross_sdk	The release-blocking Cross-SDK parity (MurmurHash3) gate — 100% of the vendored vectors must pass. DISABLE_COVERAGE=1 matches CI: this narrow subset opts out of the global line gate (coverage is enforced by the full-suite run).
bundle exec rspec spec/integration/full_chain_spec.rb	The release-blocking Full-chain release gate — the create→decide→track→flush loop, exact wire bytes, and zero raw-secret leakage at TRACE.
cd demo/rails && bundle exec ruby script/fork_smoke.rb	The release-blocking Puma-cluster fork smoke — boots a real Puma cluster (workers 2, preload_app!) and asserts events from ≥ 2 distinct forked worker PIDs reach the track endpoint. Runs offline against a local stub.

The suite runs on the CRuby 3.1–3.4 + JRuby matrix; coverage is enforced on CRuby only (the JRuby leg runs the suite without the gate). Do not change the coverage configuration — it is single-sourced in spec/spec_helper.rb.

Test layout

Directory	What lives there
spec/unit/	Per-class unit specs (mock collaborators, isolated logic).
spec/integration/	End-to-end specs: full_chain_spec.rb (the release-blocking create→decide→track→flush loop), runtime_recipes_spec.rb (the runtime-lifecycle recipes the quickstarts are transcribed from), fork_safety_spec.rb, factory_wiring_spec.rb.
spec/cross_sdk/	The cross-SDK MurmurHash3 parity vectors (the byte-identical bucketing proof).
spec/docs/	Docs-snippet smoke specs — run the documented code samples against the real gem so documentation never drifts.
spec/fixtures/	Vendored config fixtures (test-config.json).

Making decisions deterministic — direct-data mode

The biggest source of test flakiness is the asynchronous config fetch. Avoid it entirely by building the client in direct-data mode with a known config fixture, so decisions are available without the network:

require "convert_sdk"
require "json"

config = JSON.parse(File.read("spec/fixtures/test-config.json"))
sdk = ConvertSdk.create(data: config)

context = sdk.create_context("test-visitor")
variation = context.run_experience("homepage-test")
# assert on variation&.key

With data: set, the SDK installs its config without an HTTP call and fires ready synchronously. Use a stable explicit visitor id (e.g. "test-visitor") so bucketing is reproducible run to run — the same id always maps to the same variation for a given experience.

Keep tracking quiet in tests

Disable outbound tracking so tests emit no network events. Bucketing, rule evaluation, and sticky persistence still work with tracking off — only delivery is silenced:

sdk = ConvertSdk.create(data: config, tracking: false)

For a single call, use the per-call override: context.run_experience("key", { enable_tracking: false }). See Tracking Control.

Capturing log output

Any object that duck-types debug/info/warn/error is a valid log sink. A tiny in-memory capturing sink lets you assert on the lines the SDK emits — and is the pattern the SDK's own full-lifecycle TRACE gate uses to prove no raw secret reaches a sink:

class CapturingSink
  attr_reader :entries

  def initialize
    @entries = []
  end

  %i[debug info warn error].each do |level|
    define_method(level) { |message| @entries << [level, message.to_s] }
  end

  def messages = @entries.map(&:last)
end

sink = CapturingSink.new
sdk  = ConvertSdk.create(
  data:      config,
  log_level: ConvertSdk::LogLevel::TRACE,
  sink:      sink
)

# ... exercise the SDK ...
expect(sink.messages).to include(a_string_matching(/installed direct data config/))

Passing sink: at create time captures the construction-time lines too. See Initialization and the TRACE logging section.

Config-spec drift control

The SDK uses two orthogonal gates to detect drift between the backend serving OpenAPI spec and the SDK's field expectations. Neither gate subsumes the other; both are required.

Build-time gate (CRuby only — fires on spec changes)

When the backend serving spec changes, the backend workflow regenerates sig/convert_sdk/config/generated/types.rbs (compile-time-only RBS type aliases, erased at runtime — never required by the SDK) and opens a PR into ruby-sdk. The existing steep check CI job then type-checks steep/config_contract_probe.rb — a CI-only file that passes hash literals with every SDK-depended field to typed helper methods declared in sig/convert_sdk/config/probe_helpers.rbs.

When the regenerated RBS removes or renames a field the probe includes, Steep raises Ruby::ArgumentTypeMismatch (error level) on the exact probe line, and steep check exits non-zero — blocking the backend-opened PR pre-merge.

What the build-time gate catches: a field the SDK declares a dependency on is removed or renamed in the spec → RBS regeneration → PR → CI fails instantly.

What the build-time gate is blind to: (a) the spec being wrong vs. reality — if the spec lies, type-checking against it certifies green while reality differs; (b) reality drifting with no spec change — if the platform begins serving a different config shape without a spec PR, the gate is never triggered; (c) JRuby — the gate is CRuby-only (rbs C-ext cannot build on JRuby).

Staging suite (all engines — catches reality drift)

staging.yml runs daily at 06:17 UTC + on manual dispatch. It exercises the SDK against the live platform and is the only gate that catches:

• Spec-vs-reality drift — when the spec says one thing and the server sends another.
• Reality drifts with no spec change — the platform can change its config shape without anyone opening a spec PR; the only way to detect this is a scheduled run against the live platform.
• JRuby runtime behavior — the build-time gate is CRuby-only; staging is the only gate that runs the SDK end-to-end on JRuby against real config.

Do not remove the daily schedule from staging.yml — the schedule exists precisely because there is no spec-change event to hook on for reality-drift detection. A one-time or dispatch-only run would miss the class of drift where the spec is never updated.

The vendored golden

spec/fixtures/test-config.json is a static snapshot of a real config response. It is used by unit and integration specs to exercise the SDK's decision logic without a live fetch. It does NOT auto-update — it is intentionally frozen as a stable reference for deterministic bucketing tests.

Tips

• Use explicit visitor ids — never rely on a random id in tests; an explicit id makes bucketing assertions stable.
• Prefer fixtures over live keys — direct-data mode with a checked-in config fixture keeps tests hermetic and fast.
• Build a fresh client per test (or per group) so visitor and queue state do not leak between tests.

Related pages

• Initialization — direct-data mode and the sink: seam
• Tracking Control — disabling tracking
• Fork Safety & Runtime Recipes — TRACE logging and the fork smoke