AgentLighthouse

AgentLighthouse is Lighthouse for AI agents. It scans a repository and answers:

Can AI coding agents correctly understand, use, and verify this project?

It is local-first and deterministic by default. It does not require OpenAI, Anthropic, or any other model-provider key.

Why Agent-Readiness Matters

Coding agents work best when a project has clear setup commands, test commands, docs, API specs, MCP tool descriptions, examples, task workflows, and safety guidance. Many mature projects are excellent for humans but still hard for agents because the context is scattered or not machine-readable.

AgentLighthouse turns that context into a score, structured findings, reports, baselines, and PR deltas.

Try It In 30 Seconds

npx @agentlighthouse/cli@alpha scan .

That command runs locally, prints an agent-readiness score, and lists concrete findings. To keep reports:

npx @agentlighthouse/cli@alpha scan . --report-dir agentlighthouse-reports

This is an alpha release. Expect useful static analysis, JSON/Markdown/SARIF reports, baselines, and PR deltas. Do not expect hosted SaaS, AI-agent execution, token-backed PR comments, or perfect OpenAPI/MCP parsing yet.

What It Does Not Do

AgentLighthouse is not:

• a chatbot
• a replacement for Codex, Claude Code, Cursor, Copilot, OpenClaw, or other agents
• an AI IDE
• an agent execution platform
• a hosted governance suite
• a model-provider gateway

It prepares projects so existing and future agents can use them more reliably.

Quickstart

AgentLighthouse is available as a public alpha on npm. Use the explicit @alpha tag while the project is pre-1.0.

Alpha npm install:

npm install -g @agentlighthouse/cli@alpha
agentlighthouse scan .
agentlighthouse scan . --report-dir agentlighthouse-reports

Or run with npx:

npx @agentlighthouse/cli@alpha scan .

0.1.0-alpha.1 is the first usable npm alpha. The @agentlighthouse/cli@0.1.0-alpha.0 package was broken by a leaked workspace dependency and has been deprecated.

Current local development:

pnpm install
pnpm build
pnpm --filter @agentlighthouse/cli dev scan .
pnpm --filter @agentlighthouse/cli dev scan . --report-dir agentlighthouse-reports

In non-interactive automation, use CI=true pnpm install to avoid package-manager prompts.

The report bundle writes:

• scan.json
• scan.md
• scan.sarif
• pr-summary.md

Baselines

Create a baseline from the current accepted state:

pnpm --filter @agentlighthouse/cli dev baseline create . --output agentlighthouse-baseline.json
pnpm --filter @agentlighthouse/cli dev baseline validate agentlighthouse-baseline.json
pnpm --filter @agentlighthouse/cli dev baseline summary agentlighthouse-baseline.json

A baseline is a normal scan-result JSON file. Committing or updating it is an intentional project decision.

Compare Against A Baseline

pnpm --filter @agentlighthouse/cli dev scan . \
  --baseline agentlighthouse-baseline.json \
  --report-dir agentlighthouse-reports \
  --comparison-output agentlighthouse-delta.md \
  --comparison-format pr-summary

For PR-aware analysis:

git diff --name-status origin/main...HEAD > changed-files.txt
pnpm --filter @agentlighthouse/cli dev scan . \
  --baseline agentlighthouse-baseline.json \
  --changed-files changed-files.txt \
  --fail-on-pr-regression

CI Usage

Prefer the published alpha CLI for external repositories:

Simple scan gate:

npx @agentlighthouse/cli@alpha scan . --fail-under 75 --report-dir agentlighthouse-reports

Baseline gate:

npx @agentlighthouse/cli@alpha scan . \
  --baseline agentlighthouse-baseline.json \
  --report-dir agentlighthouse-reports \
  --fail-under 80 \
  --min-confidence medium \
  --fail-on-regression

PR-aware gate:

npx @agentlighthouse/cli@alpha scan . \
  --baseline agentlighthouse-baseline.json \
  --git-base origin/main \
  --git-head HEAD \
  --fail-on-pr-regression

Reports are written before gates fail.

GitHub Action

The GitHub Action is experimental for public alpha. It works from source and is useful for dogfooding, but direct npx @agentlighthouse/cli@alpha commands are the recommended CI path for external repositories until the action is hardened further.

- uses: actions/checkout@v4
  with:
    fetch-depth: 0
- uses: PainDeMie64/agentlighthouse@main
  with:
    path: "."
    baseline: agentlighthouse-baseline.json
    report-dir: agentlighthouse-reports
    git-base: origin/${{ github.base_ref }}
    git-head: HEAD
    fail-on-pr-regression: "true"

The current action is source-based: it installs and builds AgentLighthouse from the checked-out action path, then scans the caller workspace through GITHUB_WORKSPACE.

Example Output

AgentLighthouse Score: 85/100
Confidence: medium
Coverage: 67%

New high-severity findings:
1. Missing AGENTS.md
2. No test script in package.json

Recommended actions:
1. Create AGENTS.md with setup, tests, architecture, conventions, and safety rules.
2. Add a package.json test script or document the equivalent command clearly.

Interpreting Scores

AgentLighthouse measures agent-readiness, not general software quality.

• 90-100: strong agent context, still check confidence and coverage.
• 70-89: useful foundation with concrete gaps.
• 40-69: agents will likely need human help.
• 0-39: missing core agent-readable context or verifiability.

The score is shown with:

• confidence
• coverage
• human-readable project signals
• agent-specific context layer
• verifiability signals

Profiles

Supported profiles:

• default
• devtool
• api
• mcp
• docs
• library
• internal

Use a profile explicitly:

agentlighthouse scan . --profile api

Or set agentlighthouse.config.json.

Command Probes

Static analysis is the default. Command probes are opt-in:

agentlighthouse scan . --probe commands

AgentLighthouse never runs install commands or arbitrary commands copied from docs. Probes are intended for trusted local or CI environments.

OpenAPI And MCP Analysis

AgentLighthouse includes deterministic semantic checks for:

• OpenAPI operation IDs, descriptions, examples, auth, errors, pagination, rate limits, and destructive operations
• MCP tool names, descriptions, input schemas, examples, privacy, auth, and side effects

The goal is to explain what an AI coding agent is likely to misunderstand and how to fix it.

Local Development

pnpm install
pnpm build
pnpm test
pnpm typecheck
pnpm lint
pnpm validate:realworld
pnpm dev

The dashboard runs from apps/web.

Troubleshooting

• If a baseline fails validation, confirm it was created by agentlighthouse scan --format json or agentlighthouse baseline create, not by agentlighthouse compare.
• If git-based PR analysis fails, fetch enough history and retry with git diff --name-status <base>...<head>.
• If command probes fail, rerun the underlying package script locally; probes are opt-in and do not install dependencies.
• If a score feels low for a mature project, check confidence, coverage, and the agent-specific context layer before treating it as a software-quality judgment.

Repository Structure

apps/web                Next.js dashboard
packages/core           scanner, analyzers, scoring, schemas, generators, reporters
packages/cli            command-line interface
examples                sample projects and GitHub Action examples
docs                    product, architecture, scoring, CI, release, and validation docs
validation/reports      safe generated scan reports

Current Limitations

• Public npm packages are alpha-quality. Prefer explicit @alpha commands until a stable release exists.
• No hosted SaaS yet.
• No auth or billing.
• No token-backed PR comments.
• No GitHub Checks API integration.
• No AI agent execution.
• OpenAPI $ref and MCP static extraction are useful but not complete.
• The GitHub Action is source-based and experimental.

Release Readiness

Local release checks are available without publishing anything:

pnpm release:check
pnpm release:dry-run
pnpm release:fresh-clone
pnpm release:rehearsal

pnpm release:smoke packs the core and CLI packages, installs the tarballs in a temporary clean project, and runs the installed agentlighthouse binary. pnpm release:fresh-clone rehearses the README workflow and packed CLI install from a clean clone. No release script publishes packages, creates GitHub Releases, or creates git tags.

Roadmap

• Phase 3: docs crawler and deeper API/MCP reference resolution.
• Phase 4: deterministic benchmark runner.
• Phase 5: hosted dashboard and project history.

More Docs

• CI
• GitHub Action
• Baseline Comparison
• PR-Aware Analysis
• Scoring Model
• Rules
• Release Process
• First User Trial
• Next Issues
• Alpha Release Checklist
• Versioning
• Schema Stability