Product Vision

Product Vision

Problem statement

1. Tests are report artifacts, not telemetry signals

JUnit XML / Allure results are a snapshot of one run. There is no native time-series, no trace context (test → step → network → browser action), and no correlation with application telemetry. Every tool re-parses XML and rebuilds a static HTML page. This is an architectural dead end.

2. Flaky tests destroy CI trust, but detection is primitive

Most teams "detect" flakes manually or via retries > 0. There is no stable test identity across refactors, no statistical model, and no auto-quarantine. A flaky test lives for months, quietly eroding the signal of a green build.

3. Root-cause analysis is manual archaeology

A test fails → an engineer opens the log, stack trace, screenshot, git blame, and searches "has this failed before?". 20–40 minutes per incident. No open-source tool does this automatically.

The thesis

If every test execution is treated as a span in a trace, then flaky detection, historical analytics, and AI root-cause become natural extensions of the telemetry — not separate hacks.

This is the load-bearing idea. Everything else follows from it.

Target users / personas

Persona	Pain	What Flakemetry gives them
SDET / QA Automation (core)	Flaky tests, no history, manual triage	Flaky score, auto-quarantine, history by test identity
Platform / DevEx Engineer	Slow, unreliable CI; no suite-health metrics	Suite health dashboard, slowest/most-failing, DORA-style test metrics
EM / Tech Lead	"Why is CI red again?", no trends	Trend dashboards, PR-level quality gates
OSS maintainer	Flaky CI scares off contributors	GitHub Action + PR comment "this failure is a known flake"

Wedge persona: an SDET in a 5–50-engineer team using Playwright + GitHub Actions.

Competitive landscape

Product	Model	Weakness we exploit
Allure	OSS reporter	Static report, no DB/history, no flaky model, no AI
ReportPortal	OSS, heavy (Java, ELK, RabbitMQ)	Painful deploy, dated UX, weak flaky ML
Currents.dev / Sorry-Cypress	SaaS/OSS	Tied to a runner's orchestration; not OTel
Trunk Flaky / BuildPulse	SaaS-only	Closed, pricey, no self-host, shallow trace depth
Datadog CI Visibility	Enterprise SaaS	Expensive, lock-in, overkill for teams < 200

Differentiators (in order of strength)

1. OTel-native ingestion — tests as traces/spans. Nobody does this well in OSS. Technical moat + brand narrative.
2. Stable Test Identity Engine — fingerprinting that survives refactors/moves/parameterization. See Test Identity Engine.
3. Explainable flaky scoring — a transparent Bayesian model with reason codes, not a black box. See Flaky Scoring.
4. AI RCA over traces + git diff — retrieval over historical failures + signature clustering, not "summarize this log". See AI RCA Architecture.
5. OSS-first, self-hostable, single docker compose up — anti-ReportPortal on DX.

Positioning statement

For SDETs and platform teams drowning in flaky tests and opaque CI failures, Flakemetry is an OpenTelemetry-native test intelligence platform that turns every test run into queryable telemetry — so flakiness is scored and explained, and failures are root-caused automatically. Unlike static reporters (Allure) or heavyweight servers (ReportPortal), Flakemetry is trace-first, explainable, and runs with one command.