Skip to content

v0.2.0

Choose a tag to compare

View all tags
@michaelegner michaelegner released this 31 Aug 10:30
· 44 commits to main since this release
v0.2.0
38dd033
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Project status: Experimental / pre-1.0. The Canonical Model, REST API surface, Graph Schema,
Adapter SPI, and configuration format are not yet guaranteed stable — expect breaking changes on a
future minor version bump. See ROADMAP.md.

License: Apache License, Version 2.0.

What v0.2 proves

AIP v0.2.0 adds a deterministic evaluation suite that verifies the architecture intelligence
introduced in v0.1 against independently authored ground truth.

v0.1 already builds an evidence-backed architecture knowledge graph from OpenAPI, AsyncAPI, and
OpenTelemetry, and classifies every relationship as CONFIRMED, OBSERVED_ONLY, or
NOT_OBSERVED_IN_WINDOW. v0.2 does not add another architecture-intelligence dimension — it
proves, reproducibly, that the intelligence v0.1 already produces is actually correct: known
declared input plus known observed input plus an observation context, run through real AIP
ingestion and runtime resolution, projected to canonical facts, and compared against
independently authored ground truth for a deterministic PASS/FAIL.

Highlights

  • A deterministic evaluation kernel (evaluation/) — real AIP ingestion and runtime resolution,
    never a second implementation of AIP's own semantics.
  • Ten core scenarios: REST and queue-based dependencies, CONFIRMED/OBSERVED_ONLY/
    NOT_OBSERVED_IN_WINDOW classification, topology/directionality, partial observation with
    qualitative coverage, evidence reconciliation, and a pure declared-only REST relation.
  • Exhaustive missing/unexpected/forbidden-fact detection — no in-scope actual fact escapes
    assertion.
  • Strict expected.yaml schema validation and deterministic comparison/report ordering.
  • Fully local reproducibility: no separately running Neo4j, no LLM API key required.

How to run the evaluator

git clone https://github.com/michaelegner/architecture-intelligence-platform.git
cd architecture-intelligence-platform
uv sync
uv run python -m evaluation run

Requires Docker (an ephemeral Testcontainers Neo4j) — no separately running Neo4j, no
OPENAI_API_KEY. See evaluation/README.md for scenario details,
the ground-truth format, and how to read a failure report.

Evaluation result

AIP Evaluation — I4

[PASS] 01-rest-confirmed
[PASS] 02-rest-observed-only
[PASS] 03-async-confirmed
[PASS] 04-orphan-messaging
[PASS] 05-mixed-rest-async
[PASS] 06-request-response-queue-pair
[PASS] 07-not-observed-in-window
[PASS] 08-evidence-reconciliation
[PASS] 09-partial-observation
[PASS] 10-declared-rest-relation

Scenarios:          10
Passed:             10
Failed:             0

Missing facts:      0
Unexpected facts:   0
Forbidden facts present: 0
Wrong statuses:     0
Evidence errors:    0

RESULT: PASS

Reproduced deterministically across repeated runs against independent fresh evaluation state.

What changed since v0.1.0

See CHANGELOG.md for the full entry. In summary: the
entire deterministic evaluation suite above is new; no v0.1 architecture-intelligence semantics
(canonical model, adapters, graph schema, runtime status classification, evidence model) changed.

Known limitations

  • Pre-1.0 surfaces (Canonical Model, REST API, Graph Schema, Adapter SPI, configuration format)
    are not yet guaranteed stable.
  • No Kubernetes/cloud discovery, no gRPC/protobuf adapter, no GraphRAG.
  • No broad validation against independent real-world reference systems yet (planned as a separate
    pre-1.0 qualification track, not part of this release).
  • Natural-language query requires an LLM provider key; deterministic evaluation does not.
  • DLQ directionality and cross-batch HTTP correlation remain optional, not-yet-added evaluation
    scenarios (both already covered by other means: DLQ mapping is captured declaratively, and
    cross-batch correlation is exercised in the runtime demo).

Quick Start

git clone https://github.com/michaelegner/architecture-intelligence-platform.git
cd architecture-intelligence-platform
cp .env.example .env
docker compose up

Then open http://localhost:8000. See the root README.md for the full Quick
Start, runtime demo, and feature walkthrough.

Documentation

  • README.md — project overview, Quick Start, runtime demo
  • evaluation/README.md — the deterministic evaluation suite
  • docs/ — architecture, canonical model, graph/evidence model, ingestion, analyses,
    OpenTelemetry, configuration, security model, development, adapter development
  • docs/specifications/0.2.0/ — the v0.2.0 design/implementation
    history (I1-I5)
  • ROADMAP.md / CHANGELOG.md — where this is headed,
    and what's shipped so far

Security

Please report security vulnerabilities privately — see SECURITY.md. Never
report a vulnerability as a public issue.

Assets 2
Loading