Acceptance Testing Strategy

Acceptance Testing Strategy

Originally documented in Lab 9 Report. Extracted here so the strategy is discoverable from the sidebar.

1) How we structure our acceptance tests

• We structure our acceptance tests around specific user journeys (e.g., "Mentee registers → verifies email → logs in") and categorize them based on system roles (Mentee, Mentor, Admin).
• Following the structured test document format introduced in the lectures, each test includes specific fields such as a unique Test Case ID, Test Title, Pre-conditions, sequential Test Steps, Test Data, and Expected Results.
• To ensure complete traceability, every test case is directly linked to a specific requirement ID or user story.
• For every critical workflow, we design at least one "happy path" (normal operation) and 1–2 high-risk negative scenarios. We utilize extreme or invalid data (e.g., full mentor capacity, unauthorized access attempts, invalid tokens) to expose potential defects.
• We implement a seed or reset mechanism for test data isolation to ensure each test runs in a clean, predictable state.

2) Acceptance testing strategy

• Our core strategy relies on requirements-based black-box testing, focusing on validating the system's functionality and performance from the user's perspective without needing to know the internal code structure.
• We utilize Playwright for automated End-to-End (E2E) testing to verify realistic user flows within actual browser environments.
• System testing is executed in an integrated environment (frontend combined with backend, such as local Docker or a staging server) to ensure all components interact correctly and transfer the right data across interfaces.
• We prioritize critical workflows, business rules enforcement, and user-visible feedback mechanisms.
• Our coverage goal is 90–95% for critical paths. The primary success condition is that all critical flows pass seamlessly without yielding any significant UX degradation or regressions in previously working code.

3) Implementation conventions — keeping tests deterministic

These are the patterns the team has converged on after debugging flaky specs across Chromium / Firefox / WebKit. New acceptance tests should follow them by default; deviations should be justified in a comment.

3.1) Pre-authenticate via API, inject into storage

Tests whose goal is not to verify the login UI itself should skip /login entirely. The seed endpoints (POST /api/test/users, POST /api/test/admin) mint a server-side JWT we install directly into localStorage via Playwright's storageState (the official auth pattern).

This eliminates the framer-motion + React 18 hydration race that otherwise produces "POST never fires" symptoms when Playwright's fill() / click() lands before React commits the controlled-input state and binds the form's submit handler. Empirically, AT-02 went from 4 / 5 flake rate (across all three browsers) to 0 / 10 once the UI login was removed.

Helper: frontend/e2e/fixtures/session.js → newAuthenticatedContext(...).

3.2) Atomic Promise.all([waitForResponse, action]) for UI mutations

When the test fires a UI action that triggers a backend mutation, and a subsequent assertion depends on the backend state, wait for the response explicitly:

const [acceptResponse] = await Promise.all([
  mentorPage.waitForResponse(
    res => /\/api\/mentorship-requests\/\d+\/accept$/.test(res.url())
        && res.request().method() === 'PUT',
  ),
  inbox.acceptFirstRequest({ months: 3 }),
]);
expect(acceptResponse.ok()).toBe(true);

A bare await click() resolves as soon as the click event is dispatched — the mutation it triggers may still be in flight when the next assertion runs. This was the single most common race in our suite.

3.3) Isolate shared per-IP resources per test

The rate-limit middleware buckets are IP-keyed and shared across all browser projects. Tests that exercise rate-limit behaviour must:

• use a unique probe email scoped to that sub-test (ratelimit-probe-at07@example.invalid, headers-probe-at07@…), so the bucket pressure stays attributable
• call resetRateLimits(request) at both the start and end of the scenario so a stuck bucket can't leak into neighbouring tests
• avoid POST /api/auth/login as a probe outside the rate-limit test itself; use GET /api/users/me with the seed-minted session token to verify "API reachable" because it runs through a different rate-limit rule

3.4) Avoid waitForLoadState('networkidle')

Per the Playwright best-practices doc networkidle is "discouraged" — it's not a reliable readiness signal when the page makes background pings (telemetry, polling). Prefer locator-based waits: await expect(locator).toBeVisible({ timeout }).

3.5) Disable interactive controls until the React commit settles

If a spec must exercise the login UI (AT-01 + AT-03), the login form's submit button is disabled={!hydrated} until a useEffect flips hydrated to true on the first commit. Playwright's intrinsic auto-wait for Locator.click() then gates on hydration completion — this is the Playwright maintainers' own recommendation for the hydration race (microsoft/playwright#27759).

3.6) Stable data-testid selectors over text / class selectors

Tests select by data-testid exclusively. The constants module frontend/e2e/testids.js centralises every selector so a rename is a single-file grep. Mobile uses the parallel testID prop on React Native, applied across every screen AT-16 walks. Text matches (getByText) and CSS selectors (page.locator('.foo')) are forbidden in new specs — they break under styling refactors and copy edits.

3.7) Stability budget: zero flake locally over 10 consecutive runs

Before merging a new spec, the author runs the full suite ten times in sequence locally. A merge requires zero retries across all ten runs. Specs that produce intermittent retries even once are reworked. CI retains retries: 2 as a safety net for runner contention; the bar to add a spec is "0 retries needed when the runner is clean".

4) Acceptance criteria — building "the right thing"

• Core Goal Completion — Users must be able to successfully complete their primary objectives: registration, verification, login, mentor/mentee discovery, sending/answering requests, messaging, and receiving notifications.
• Business Rule Enforcement — The system must strictly apply and handle all defined business constraints, such as mentor capacity limits, role-based access restrictions, and accurate request status transitions.
• Clear User Feedback — The system must provide clear, understandable, and consistent user feedback (success or failure messages) for all inputs, particularly for borderline or incorrect input attempts.
• State Persistence and Integrity — System data must be persistent and accurate; the correct data must be displayed consistently across page reloads and different user sessions without data corruption.
• Zero Critical Regression — New updates or features must not break previously accepted and working workflows, which we verify continuously.
• Stakeholder Meetings — We conduct meetings with stakeholders to demonstrate the implemented features, gather direct feedback, and verify whether the delivered system truly aligns with their initial expectations and desired goals.

---

See also:

• Acceptance Tests — the seven concrete test cases (AT-01 through AT-07)
• Test Data Strategy — how we generate and validate the data the tests run against
• Test Plan & Coverage Wiki Page — MVP-era test plan and coverage targets