FAQ

FAQ

How is this different from Allure?

Allure renders a static HTML report of one run. Flakemetry treats runs as telemetry: persistent history, a stable Test Identity Engine across refactors, an explainable Flaky Scoring, and AI RCA Architecture. Allure shows what happened; Flakemetry answers is this test trustworthy and why is it failing.

How is this different from ReportPortal?

ReportPortal is powerful but heavy (Java, ELK, RabbitMQ) with a dated UX and weak flaky modelling. Flakemetry is OTel-native, TypeScript end-to-end, runs with one docker compose up, and treats explainable flaky detection + AI RCA as first-class.

Why OpenTelemetry instead of a custom format?

Reusing OTel means a mature ecosystem (SDKs, collectors), and — crucially — correlation with application traces: a failing E2E can link to the backend spans it triggered. See OTel Test Conventions.

Does it only support Playwright?

Playwright first (M1). The instrumentation lives in a shared SDK, so Jest/Vitest reporters and a pytest/JUnit adapter follow in M4, all emitting the same OTel contract.

Will it slow down or break my CI?

No. Ingestion returns 202 instantly and the reporter fails open — if the endpoint is unreachable it buffers locally and never fails your test process. All heavy work is asynchronous. See Ingestion and Scaling.

Do I have to send my data / code to an AI provider?

No. AI RCA Architecture is provider-agnostic via LLMProvider. Run it on local Ollama for a fully air-gapped path; a PII/secret-scrubbing pass runs before anything is stored or sent regardless.

Is the flaky detection a black-box ML model?

No — deliberately. It's a transparent Beta-Binomial model with time-decay, and every score ships with reason codes explaining it. SDETs won't act on a score they can't understand. See Flaky Scoring.

What happens to my test history when I rename or move a test?

It's preserved. The Test Identity Engine matches across file moves (L2), renames (L3), and parameterization (L4), stitching prior fingerprints into aliases[].

Is it free? What's the business model?

The OSS core is MIT and fully self-hostable — nothing is crippled. A commercial layer covers hosting, managed AI at scale, RBAC/SSO, and long retention. See Monetization and OSS Model.

Can I extend it?

Yes — a plugin architecture (M4) exposes ingestion-source and analyzer hooks so you can add signals or sources without forking core. There's also a public REST API + webhooks.

Where do I follow progress / contribute?

The Roadmap board, Issues (filter by good first issue), and Discussions.