Skip to content

Define PR requirements for healthy repo ecosystem (definition of done) #106

@diberry

Description

@diberry

Scope

Define the canonical set of PR requirements (definition of done) for the Squad repository. This is the source of truth that reviewers check against, that CI gates enforce, and that the review process (#100) operationalizes.

Problem

PRs ship without documentation, changelog entries, export updates, or test coverage -- and reviewers have no canonical checklist to enforce. The recent bradygaster#640 (StorageProvider) ecosystem audit found 4 HIGH severity gaps (CHANGELOG missing, README section missing, docs page missing, /storage export missing) that should have been caught before review. Without a definition of done, authors don't know what to include and reviewers don't know what to check.

Related issues exist for process (#100) and automated enforcement (#104), but nothing defines the actual requirements those systems check against. This issue is that definition.


Definition of "User-Facing Change"

A PR contains a user-facing change if it introduces:

  • A new public API export (added to subpath exports in package.json)
  • A new CLI command or subcommand
  • A change to an existing public API surface (breaking or new parameters)

Internal refactors, infrastructure changes, and dependency updates are NOT user-facing.

Why this matters: The requirements categories below apply conditionally to user-facing changes. Knowing what counts as user-facing prevents half of all disputes about "does this need a CHANGELOG entry?"


PR Requirements by Category

Requirements are ordered by review flow (how a reviewer checks them), not by importance. All marked REQUIRED must be satisfied before merge. Some are automatically enforced (CI gates); others require human review.

a) Git Hygiene (REQUIRED -- automated)

  • Commits reference the issue number: "Closes #N" or "Part of #N" in commit message
  • No binary files or build artifacts committed (dist/, node_modules/, .DS_Store, etc.)
  • No unrelated file changes (bleed check: only files needed for the stated goal)
  • Clean commit history: logical commits or squashed to a single coherent commit
  • No force-pushes to shared branches (dev, main)

Enforced by: squad-ci.yml bleed check + git hooks


b) CI / Build (REQUIRED -- automated)

  • Build passes:
    pm run build completes successfully
  • All existing tests pass:
    pm test or
    pm run test
  • No new build warnings introduced without justification in PR description
  • Feature flags documented in code and CHANGELOG if gating new behavior

Enforced by: GitHub Actions CI pipeline


c) Code Quality (REQUIRED -- manual review for completeness, automated for linting)

  • All new public APIs have JSDoc/TSDoc comments
  • No lint errors: �slint passes
  • No type errors: sc --noEmit passes
  • Tests written for all new functionality
  • All tests pass: �itest exits 0

Enforced by: �slint in CI + manual code review


d) Documentation (REQUIRED for user-facing changes -- manual review)

  • CHANGELOG.md entry under [Unreleased] following Keep-a-Changelog format
    • Example: ### Added -- FeatureName (#N) -- short description
    • Required when: packages/squad-sdk/src/ or packages/squad-cli/src/ changes
  • README.md section updated if adding a new feature or module to the SDK
  • Docs feature page added under docs/src/content/docs/features/ if adding a user-facing capability
  • docs/src/navigation.ts updated if a new docs page is added
  • Docs build test updated if a new feature page is added (playwright test in CI)

Enforced by: Manual PR review (FIDO, Flight) + future automated checks in #104


e) Package / Exports (REQUIRED for new modules -- manual review)

Enforced by: Manual PR review + future export audit gate in #104


f) Samples (REQUIRED when API changes -- manual review)

  • Existing sample projects updated if the PR changes a public API they use
  • New user-facing features include at least one sample demonstrating usage
  • Sample README.md explains what the sample does and how to run it
  • Samples build and run:
    pm run build in sample directory succeeds

Enforced by: Manual PR review + #103 (Samples CI coverage)


Breaking Change Policy

Any PR that changes the public API in a backward-incompatible way (removal, signature change, behavior change) must:

  • Document the breaking change in CHANGELOG.md under a [BREAKING] section
  • Add a migration guide as a docs note in the affected feature page
  • Update all sample projects to use the new API
  • Include the migration path in the PR description (## Breaking Changes section)

Examples of breaking changes:

  • Removing an export that was previously public
  • Changing function signature (parameter name, order, type)
  • Changing CLI command name or subcommand structure
  • Changing configuration file format in backward-incompatible way

Examples NOT breaking:

  • Adding new optional parameters
  • Adding new exports
  • Adding new subcommands (existing ones unchanged)
  • Fixing documented behavior that was incorrect

Exception / Waiver Process

Recognized that emergencies (security fixes, critical bugs) require flexibility. Any requirement can be waived if documented and approved.

How to Request a Waiver

Add to PR description under a ## Waivers section:

`

Waivers

  • Item waived: (b) Documentation CHANGELOG entry
  • Reason: Security fix needs urgent release (CVE-2024-XXXXX)
  • Approval by: [Flight/FIDO to be stated in PR review comment]
    `

Waiver Approval Authority

  • Flight (architecture decisions): Internal refactors, infrastructure, module reorganization
  • FIDO (quality decisions): Bug fixes, test improvements, samples, CI changes
  • Both: External API changes, breaking changes

Waiver Approval Examples

Scenario Waivable Items Approver
Internal refactor (no API change) Documentation, Samples Flight
Security fix Documentation, Samples (with follow-up issue) FIDO
Test infrastructure change Documentation, Exports, Samples FIDO
CI/GitHub Actions update Documentation, Exports, Samples FIDO
Documentation-only change Code Quality (tests), Exports, Samples FIDO
Non-breaking SDK bugfix Samples FIDO

Documentation of Approved Waivers

Approver adds a comment to the PR:

`
Approved waiver:

  • Category: (b) Documentation
  • Item: CHANGELOG entry
  • Reason: [brief]
  • Follow-up: [if required for backfill, e.g., "Follow-up issue due by Friday"]
    `

This audit trail ensures:

  1. No silent waiving (hidden in PR title or description)
  2. Clear approval authority (author can't self-waive)
  3. Trackability for future compliance review

Self-Waiving Is Not Allowed

  • Do NOT use skip labels without explicit reviewer approval in PR comments
  • Do NOT remove requirements from checklist without documented waiver
  • Do NOT claim "documentation will follow" without a tracked follow-up issue

Edge Case Exemptions

Certain PR types are categorically exempt from specific requirements by nature. These are NOT waivers (no approval needed); they are structural exemptions.

PR Type Exempt From Reason
Internal refactor (no public API change) (d) Docs, (f) Samples No user-visible change
Test-only changes (d) Docs, (e) Exports, (f) Samples No production code changed
CI/GitHub Actions workflow changes (d) Docs, (e) Exports, (f) Samples Infrastructure only
Documentation-only changes (c) Code Quality (tests), (e) Exports, (f) Samples No code changes
Dependency bumps (routine maintenance) (d) Docs feature page, (f) Samples Routine dependency update

Important: Exemptions apply only when the PR genuinely affects NO code in the exempted categories. A PR touching both infrastructure AND public APIs is NOT exempt.


PR Size Guidelines

Size Files Changed Guidance
Small < 10 files Preferred. Fast to review, easy to revert.
Medium 10-20 files Acceptable. Justify scope in PR description.
Large > 20 files Must be split or justified with written explanation.
Red flag > 50 files STOP -- split the PR or get explicit written approval before proceeding.

The large-deletion-guard in squad-ci.yml already enforces the 50-file deletion threshold. The size guidelines above are the human-review equivalent. If splitting feels artificial (e.g., splitting a coherent 5-file feature into parts to meet size limits), that's a signal to request an exception rather than fragment the PR.


PR Description Template

Every PR description should answer these questions. This template will be published at .github/PULL_REQUEST_TEMPLATE.md (Phase 2 deliverable).

`

What

[One paragraph: what does this PR change?]

Why

[Problem being solved. Link the issue: Closes #N]

How

[Approach taken. Key design decisions and trade-offs.]

Testing

[What was tested and how. Manual steps or automated test names. "No changes" only if truly no user-facing change.]

Docs

[What documentation was updated. List the files. "N/A" only if truly no user-facing change.]

Breaking Changes

[Any backward-incompatible changes to public APIs. "None" if clean.]

Waivers

[If any requirements are waived (see Exception / Waiver Process section). Empty if none.]
`


Implementation Phases

Phase 1 -- Document requirements (THIS ISSUE)

Deliverable: Accept the requirements above as the canonical PR requirements for this repo.

Also in Phase 1: Create .github/PR_REQUIREMENTS.md as the versioned, committed source of truth. The GitHub issue becomes a discussion forum; the committed file is the enforced spec.

Why: The GitHub issue is mutable and unversioned. Anyone with write access can change it silently without PR review. A committed markdown file ensures:

  • Changes go through PR review (audit trail)
  • Requirements are versioned with the repo history
  • CI scripts can deterministically reference a specific version

Acceptance: Team agrees on requirements + .github/PR_REQUIREMENTS.md file committed to repo.

Phase 2 -- PR template

Deliverable: Create .github/PULL_REQUEST_TEMPLATE.md with the description template above (Section: PR Description Template).

Outcome: Authors see the structured checklist when opening a PR on GitHub.

Phase 3 -- Automated CI gates (via #104)

Deliverable: Implement CI checks that enforce the REQUIRED items in categories (a)-(f).

Key gates:

  • Git hygiene: bleed check, commit message validation
  • Build:
    pm run build,
    pm test
  • Code quality: �slint, sc --noEmit
  • Documentation: CHANGELOG entry check (detects user-facing API changes), docs page presence for new features
  • Exports: export map audit against src/ modules
  • Samples: updated if API changed

Outcome: PRs that skip requirements fail CI before review, reducing reviewer burden.

Phase 4 -- Reviewer checklist integration (via #100)

Deliverable: Add a structured reviewer checklist to the Flight/FIDO review process that maps to these categories.

Outcome: Human review has a structured rubric, not ad-hoc judgment.


Known Limitations and Scope Boundaries

This section acknowledges constraints and trade-offs reflected in the requirements above.

1. Enforceability Gap

9 of 18 requirements are manual-only with no CI gate:

  • README section updates (detection requires NLP)
  • Docs page relevance (subjective)
  • Sample completeness (subjective)
  • API migration guide quality (subjective)

Consequence: Enforcement depends on consistent reviewer judgment. Without clear precedent, reviewers will judge the same item differently across PRs.

Mitigation:

2. Theater Risk: "REQUIRED" vs "ENFORCED"

Calling something "REQUIRED" without automated enforcement creates an appearance of rigor without substance. Reviewers may:

  • Skip items when under time pressure
  • Apply inconsistently
  • Create undocumented workarounds

Mitigation:

  • Phase 3 prioritizes high-leverage automated gates (CHANGELOG, exports, build)
  • Manual-only items clearly labeled REQUIRED (manual review) in Phase 1
  • Waiver process provides escape hatch with audit trail

3. Over-Engineering for a 5-Person Team

Some requirements (samples, docs pages, feature announcement) are enterprise-grade. For a small team:

  • Timelines squeeze -> requirements waived
  • Samples first casualty under deadline
  • Docs quality suffers

Acknowledgment: This is a conscious trade-off. The requirements define "healthy ecosystem" not "sustainable sprint velocity." Individual PRs may waive items legitimately. The goal is that the sum of PRs ships with healthy ecosystem properties, not that every PR is perfect.

Mitigation:

  • Waiver process allows deliberate trade-offs
  • Follow-up issues ensure backfilling (e.g., "documentation follow-up due by Friday")
  • Quarterly review (Phase 1+ decision) to revisit proportionality

4. Silent Drift Risk

The requirements file in Phase 1 (.github/PR_REQUIREMENTS.md) is the source of truth. But:

Consequence: Over time, requirements diverge from enforcement.

Mitigation:

5. Ambiguities Not Yet Resolved

  • "Module" definition: Does refactoring an existing module count as "adding a module"? (Relevant for exports requirement)

    • Guidance: "Module" = new directory under packages/squad-sdk/src/ or new export namespace. Refactoring existing module does not trigger export requirement.
    • Follow-up decision: Document in .squad/decisions.md once finalized
  • "User-facing" detection for CHANGELOG: Currently manual. False positives on internal refactors, false negatives on config-driven behavior changes.


Relationship to Existing Issues

Issue Role
#100 (Review completeness) Defines the review process -- this issue defines what reviewers check against
#103 (Samples CI coverage) Implements the "samples" requirement category (f) above
#104 (PR completeness gates) Implements automated enforcement of REQUIRED items as CI gates

This issue is the source of truth. #100 and #104 derive their checklists from it.


Acceptance Criteria

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgo:needs-researchNeeds investigationsquadSquad triage inbox — Lead will assign to a membersquad:fidoAssigned to FIDO (Quality Owner)

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions