Automated documentation validation pipeline to detect drift from implementation

[oomokaro1]

What

Build a CI pipeline that automatically validates documentation against the actual codebase, detecting when docs describe features that don't exist or when code changes aren't reflected in docs.

Why

The documentation currently describes phantom endpoints, tables, and contract functions that were never implemented or have since been removed:

• openapi.yaml documents /merchants/me/anchor, /v1/escrow/claimable, /v1/escrow/claimable/:id/claim — none of these controllers exist in the backend
• architecture/backend.md references claimable_escrows and anchors tables — not in the Drizzle schema
• contract/spec.md documents create_claimable_escrow, claim_escrow, get_claimable_escrow — not in the Soroban contract
• architecture/frontend.md references useDexPrices hook and lib/dex — don't exist in the frontend

This creates a bad developer experience: contributors read the docs, try to use documented features, and find they don't work.

Scope

In scope

1. API route validator — parse NestJS controllers to extract route definitions (method, path, auth), diff against openapi.yaml, flag phantom endpoints and missing docs
2. Schema validator — parse src/db/schema.ts to extract table/column names, diff against architecture/backend.md, flag phantom tables/columns
3. Contract validator — parse orbitstream_contracts/src/lib.rs for #[contractimpl] functions, diff against contract/spec.md, flag phantom/missing functions
4. Link validator — parse all markdown files for internal links, verify targets exist, flag broken links
5. CI integration — run all validators in GitHub Actions, warn on PRs that add code without updating docs or vice versa
6. Drift report — npm run docs:drift-report generates a markdown summary of all phantom claims

Out of scope

• Auto-fixing docs (validators only report, don't modify)
• Content quality review (only checks existence, not accuracy of descriptions)
• External link checking (only internal file references)

Acceptance criteria

• npm run docs:validate:api detects endpoints in openapi.yaml not backed by actual controllers
• npm run docs:validate:schema detects tables/columns in docs not in schema.ts
• npm run docs:validate:contract detects functions in spec.md not in lib.rs
• npm run docs:validate:links detects broken internal markdown links
• npm run docs:drift-report generates a summary report
• GitHub Actions workflow runs validators on PRs touching orbitstream_docs/, OrbitStream_backend/src/, or orbitstream_contracts/src/
• All validators pass for the current codebase (after cleaning phantom references)
• Validators output machine-readable (JSON) for CI consumption

Technical context

• NestJS controllers are at OrbitStream_backend/src/*/ — route decorators are @Get(), @Post(), @Patch(), @Delete() with path strings
• Drizzle schema is at OrbitStream_backend/src/db/schema.ts — tables defined with pgTable(), columns with type helpers
• Soroban contract is at orbitstream_contracts/src/lib.rs — functions marked with #[contractimpl]
• OpenAPI spec is at orbitstream_docs/api/openapi.yaml
• Architecture docs are at orbitstream_docs/architecture/*.md
• Contract spec is at orbitstream_docs/contract/spec.md
• Consider ts-morph for TypeScript AST parsing, regex for Rust function extraction
• The orbitstream_docs repo has no package.json — the validators could live in the backend repo's scripts or as standalone Node scripts in the docs repo

Severity

medium — docs drift doesn't break functionality but misleads contributors and slows development.

[mayborn005]

Hello, I’ve gone through the requirements and I’m confident I can implement a proper solution that matches the acceptance criteria. Kindly assign this issue to me.

[adesuwa-tech]

Hi Maintainer,

I would like to work on Issue #3: Automated documentation validation pipeline to detect drift from implementation.

My approach:

• Review the current documentation workflow and identify where drift between implementation and docs can occur.
• Define validation checks that compare documented behavior, configuration, and API details against the actual codebase.
• Implement an automated pipeline step to run these checks in CI and surface mismatches early.
• Add or update tests to ensure the validation logic is reliable and catches documentation regressions.
• Update the documentation and contributor guidance to explain how the validation pipeline works and how to keep docs in sync.

I have experience with backend development, automation, documentation workflows, application security, and Git/GitHub workflows, and I am committed to delivering clean, reliable, and maintainable contributions.

If assigned, I will submit a clean, well-tested PR within 2 days.

Thank you for your consideration. I look forward to contributing to the project.

[glorious21-coder]

I understand the expected outcome and would love to work on this. I'll provide a solution that satisfies the acceptance criteria.

[Lspnjr1]

I would like to take on this task. I have solid experience building CI/CD pipelines with GitHub Actions and integrating automated quality checks into development workflows. I’ve worked on projects where maintaining consistency between documentation and the codebase was essential, implementing validation, linting, and automated testing to prevent outdated or inaccurate documentation from being merged.

For this feature, I understand the objective is to build a CI pipeline that automatically verifies documentation against the actual codebase, detecting when documented features are missing from the implementation or when code changes are not reflected in the documentation. My implementation will focus on reliability, maintainability, and seamless integration with the existing workflow. I will ensure the pipeline provides clear, actionable feedback, minimizes false positives, and is easy for contributors to understand and extend.

I prioritize clean architecture, comprehensive testing, and well-documented solutions. I’m confident I can deliver a production-ready pull request that meets the project’s requirements while improving documentation accuracy and developer experience.

[AnnieIj]

Hi! I'd love to work on this issue.

I plan to implement an automated documentation validation pipeline that detects documentation drift by comparing key documentation artifacts with the current implementation. I'll integrate the checks into the existing CI workflow, provide clear validation feedback for contributors, add the necessary tests, and document the process to help keep the documentation accurate and up to date. Could you please assign this issue to me?