Aici

Tiny no-phone-home PR contract gates for AI output.

Status: v0.1.9 Last updated: 2026-05-19 License: MIT

Aici fails pull requests when LLM output contracts drift: JSON schema, text/regex, tool calls, latency, cost, and provider endpoints. It is a repo-local CLI and GitHub Action, not a broad eval platform.

The core product is free and open source. There is no Aici backend, no telemetry, and no hosted prompt storage.

Install In 5 Minutes

npx @mgicloud/aici init --config aici.yml
npx @mgicloud/aici run --config aici.yml

init creates a self-contained fixture test:

• aici.yml
• aici-example.output.json
• aici-example.schema.json

The starter test is deterministic and does not call a model provider. Reports are written to .aici/aici-report.md, .aici/aici-report.json, and .aici/aici-report.html.

What It Tests

• Text contracts: exact match, contains, regex.
• JSON contracts: parse checks and JSON Schema.
• Tool/function calls: expected tool name and argument schema.
• Runtime budgets: latency and known provider cost.
• Provider smoke tests: OpenAI, Anthropic, and OpenAI-compatible endpoints.

Example Config

version: 1
tests:
  - name: support-response-schema
    mockOutputFile: aici-example.output.json
    expect:
      json: true
      jsonSchema: aici-example.schema.json
      contains:
        - approved

GitHub Actions

For broad pull-request coverage, start fixture-only:

name: Aici PR Gate

on:
  pull_request:

jobs:
  aici:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
      issues: write
    steps:
      - uses: actions/checkout@v6
      - uses: MG-ge/aici@v0.1.9
        with:
          config: aici.yml
          pr-comment: true

Release tags use a bundled Action CLI and do not run npm ci or rebuild the project by default.

No Aici backend is involved. The Action appends a GitHub job summary by default; PR comments and report artifacts are opt-in after you review report contents.

Run live provider checks with secrets only from trusted branches, protected merge queues, scheduled jobs, or maintainer-approved workflows. Do not expose provider secrets to untrusted PR configs, prompts, schemas, fixtures, or tool definitions.

For trusted live jobs, enforce endpoint allowlists:

- uses: MG-ge/aici@v0.1.9
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  with:
    config: examples/openai/aici.yml
    allow-provider-secrets: true
    allowed-provider-endpoints: https://api.openai.com/v1/responses

Network Audit

Inspect network behavior before running live checks:

npx @mgicloud/aici audit --config aici.yml
npx @mgicloud/aici audit --config aici.yml --json
npx @mgicloud/aici run --config examples/openai/aici.yml \
  --allow-provider-endpoint https://api.openai.com/v1/responses

aici audit reports:

• configured provider endpoints
• empty judge endpoints in v0.1
• dependencies with license/source
• source file count and LOC
• telemetry: false
• aiciBackend: false

Aici enforces provider endpoint allowlists for its own live provider calls. It is not a process-level network sandbox for unrelated commands in the same CI job.

For fixture-only PR gates, you can run Aici inside Docker with --network none so the container has no outbound network path.

Live Providers

OpenAI:

OPENAI_API_KEY=... npx @mgicloud/aici run --config examples/openai/aici.yml

Anthropic:

ANTHROPIC_API_KEY=... npx @mgicloud/aici run --config examples/anthropic/aici.yml

OpenAI-compatible providers require type: openai-compatible and baseUrl. Remote compatible endpoints must use HTTPS. Plain HTTP is allowed only for localhost and loopback local-model endpoints.

Reports

Aici writes:

• Markdown for job summaries and PR comments.
• JSON for machine-readable artifacts.
• HTML for local review.

Reports may include model outputs and normalized tool-call arguments. Use redact for tenant ids, emails, customer ids, or known sensitive fixture values.

Docs

Document	Description
Quickstart	Install, first test, and GitHub Actions setup
Config Reference	aici.yml fields, providers, and checks
Examples Guide	Which example or template to start from
Recipes	Copy-paste support, extraction, tool-agent, live, and local-model setups
GitHub Action	Action inputs, PR comments, artifacts, and trusted live checks
Trusted Live CI	Safe workflow split for provider secrets and endpoint allowlists
Docker Strict Mode	Run fixture checks in a no-network container
Data Handling	What leaves your machine and what is stored
Network Audit	Endpoint audit, allowlists, and provider/judge boundaries
Security	Redaction, secrets, artifacts, and CI guidance
Troubleshooting	Common install, config, provider, and CI failures
Roadmap	Current scope, near-term work, and explicit non-goals
Release Checklist	Checks before publishing

Development

npm install
npm run verify
npm run security:audit

Useful local commands:

npm run sample
npm run sample:tool
npm run sample:fail
npm run validate:openai
npm run validate:anthropic

Feedback

Open an issue with the smallest sanitized example you can share:

• Product feedback
• Bug report

Do not open public issues for vulnerabilities or secret-handling bugs. Use the security policy instead.