QA Myths

[sharovatov]

Why?

Problem. Teams routinely overwork and make poor decisions because they act on persistent myths (e.g., "quality can be measured directly", "QA slows us down", "more testing = higher quality", "testers own quality"). Similar myths exist in org psychology (e.g. "motivation comes from KPIs and extrinsic carrots/sticks").

Impact. By collecting these myths and documenting the logic of why they're myths and how they harm, we reduce waste, rework, and burnout; improve product outcomes and delivery flow; and give teams a shared vocabulary to push back on harmful practices.

Who benefits. Testers/QA, engineers, product managers, tech leads/architects, delivery managers, and executives.

Expected results

Outcomes

• An evergreen, curated catalog: Myth > Why it's a myth (logic) > Dangers > Better framing/countermeasures.
• A lightweight taxonomy (e.g., Metrics, Process/Flow, Testing, Organization/People) and context tags (startup/enterprise, safety-critical, etc.).
• Short "decision nudges" teams can apply in planning, reviews, and retros.

Questions

• Which myths are most common and in which contexts do they appear?
• What signals indicate a myth is driving decisions (e.g. "Goodharting" metrics, rising rework, fragile release flow)?
• What countermeasures work in practice, and what are their trade-offs?
• How do we communicate myth-busting effectively to leadership and stakeholders?
• What evidence (studies, case reports) supports or challenges each entry?

Collaboration

How to contribute:

• Post a myth as you see it.
• Add the concise logic chain (why it's a myth) and the concrete dangers.
• Link to any articles, studies, or case notes; summaries are welcome.

Target artifact

An ever-living Myth Catalog.

Background and additional information

Myth 1 — "Quality can be fully measured".

Why this is a myth

• Quality is fundamentally a match between what's desired and what's produced; both desire and perception are human and therefore subjective.
• External quality (customer satisfaction) and internal quality (modifiability/sustainability of the code) are both influenced by perceptions and context; neither is directly, fully measurable.
• Metrics like NPS/CSAT/CES and engineering metrics like coverage, complexity, defect counts, etc., are proxy metrics. These might correlate with aspects of quality but do not measure quality itself and can even conflict. The degree of correlation depedns on the context.
• Proxies are useful as signals for analysis, not as definitive measures or goals: when metrics with decent correlation are picked for the context, we can then use them to visualize trends, apply Shewhart control charts to separate noise from signal, investigate surges/drops. However, we need to regularly reassess whether a proxy metric still correlates with quality.

Dangers of believing the myth:

• Treating proxies as goals invites Goodhart's law: teams will game the number and harm real quality.
• Wasted effort (e.g., chasing high coverage while flaws in architecture, UX, or process persist).
• Blindness to customer perception and context; "good" metrics with unhappy users or fleeing employees.
• Neglect of internal quality signals (sustainability), leading to slower delivery and future failures.

more details

Myth 2 — "QA slows things down".

Why this is a myth

• Testing is part of production: you cannot call work "done" without verifying it meets expectations; testing no more "slows work" than stitching slows the making of a wallet.
• QA != testing. QA is the system of preventive, systematic measures that build quality into the process (training, process design, etc.), which reduces rework and speeds delivery.
• All the QA measures were designed to increase speed by improving quality. For instance, continuous training; reducing queues/wait time (e.g., ensemble/pairing, low WIP); shift‑left testing; removing KPIs that incentivize counterproductive behavior.
• The "QA is slow" perception usually comes from poorly designed processes (work waiting for reviews, handoffs, code freezes, deadlines/KPIs), not from QA itself.

Dangers of believing the myth:

• Cutting testing or QA creates rework loops, returns, outages, and reputation loss—slowing delivery far more.
• Managers add pressure/KPIs instead of fixing flow, increasing turnover and degrading collaboration and quality.
• Teams misdiagnose bottlenecks and leave systemic process waste untouched.

more details

Myth 3 — "More testing = higher quality".

Why this is a myth

• Testing is inspection, verification and learning; it reports defects but does not create quality. The right amount of testing depends on product complexity and risk.
• All testing exhibits diminishing returns; piling on more of the same testing yields little additional confidence.
• The appropriate strategy is context‑specific and diverse (see economics of testing research, not maximal: e.g., a high‑risk medical system warrants broad, multi‑level testing; a simple CMS — possibly not.
• Some practices have low marginal value beyond a point (e.g., mandating multiple sequential code‑review stages).
• Over‑investing in testing while neglecting QA measures (e.g., continuous refactoring, continuous learning) degrades internal quality and slows change.
• QA starts with understanding the client; even well‑built, low‑defect products can fail if they don't solve real user problems (e.g., limited customer engagement leading to poor adoption).

Dangers of believing the myth:

• Waste (unnecessary testing) and false confidence while root causes persist.
• Slower delivery because brittle systems require ever more testing to move safely.
• Masking declining internal quality; postponing refactoring and skills development.
• Building the wrong thing very well—great testing won't rescue a product that misses user needs.

more details

Myth 4 — "Testers are responsible for product quality".

Why this is a myth

• Quality is "what's desired" vs. "what's produced"; many determinants lie outside testers' control.
• Quality hinges on:
  • Choosing the right problem to solve (value definition).
  • Selecting an effective solution (design/approach quality).
  • Work processes (e.g., late testing under deadline pressure).
  • Architecture/infrastructure (testability, stable environments).
  • Developers' skills (producing stable, testable code).
  • Motivations/incentives (KPIs that fragment team focus).
• Testers rarely decide these; holding them solely accountable ignores the system that produces quality.

Dangers of believing the myth:

• Scapegoating testers while systemic issues go unaddressed.
• Misaligned incentives (e.g., bug‑count KPIs) that undermine collaboration and outcomes.
• Lower product quality from poor problem/solution choices, brittle architecture, weak processes, or insufficient skills—none fixed because attention is misdirected.
• Teamwide demoralization and lost opportunity to make everyone responsible for quality.

more details