Skip to content

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 05 Nov 15:06
· 44 commits to main since this release
f217ae6

ontos serve now keeps its Article-12 audit trail in Postgres, and
ontos query and ontos ingest write audit records too. Every tool
failure is audited, and nothing runs when the audit store is unavailable.
Store writes can no longer rewrite fact history, and supersession is
atomic. Read paths hide facts and exported entities that would reveal
restricted entities. Neo4j reads are deterministic and no longer
enumerate every path. ask can run under ontos serve with a local
Ollama planner, and ontology suggestions can be LLM-assisted from a local
model (#66 M2b). Read the upgrade notes: this release has breaking
changes.

Upgrade notes

  • Breaking: ONTOS_ENV=prod requires ONTOS_AUDIT_DB_URL. Without
    it, ontos serve refuses to boot. The Helm chart already sets it from
    the ontos-audit-db Secret.
  • Breaking: audit is fail-closed.
    • An MCP tool call whose audit record can't be written fails with
      audit unavailable. A transport rejection (401/403) becomes a 500.
    • ontos query and ontos ingest exit with code 1 before running if
      the audit store can't be opened.
  • Async database URLs only. Audit DB URLs must use an async driver:
    postgresql+asyncpg:// (install the new postgres extra; the image
    includes it) or sqlite+aiosqlite://. .env.example previously listed
    sync URLs that were always rejected.
  • Bounded arguments. MCP search k must be 1–100 and traverse
    depth 1–10. Values outside those ranges are rejected and audited.
  • Breaking for custom GraphStore implementations.
    • add_fact must reject an existing id (FactAlreadyExistsError).
    • close_fact must refuse an already-closed fact (FactNotOpenError).
    • Stores must implement the new atomic supersede(new_fact, closes),
      which ingest now calls.
  • Fewer results by design. Facts whose endpoint entities the caller
    can't see are no longer returned (#91). Ordinary exports omit untyped
    entities that no exported fact names (#100); --include-acl-mask
    exports are unchanged.
  • ask errors. ask is disabled until ONTOS_PLANNER_BACKEND=ollama
    and ONTOS_ONTOLOGY_PATH are set. Planning failures return a fixed
    message instead of the planner's error text.
  • Ranking changes. Search ordering is now deterministic. Traversals
    return each fact once, and executor scores use true hop distances.
    Expect rankings to differ from 0.7.0.
  • New ontos query --acting-on-behalf-of SUBJECT. Without it, ontos query still sees public facts only.

Performance — Neo4j traverse no longer enumerates every path (#104)

  • The problem. traverse matched (start)-[:RELATES*1..depth]-(end) and
    deduplicated relationships afterwards. Variable-length matching enumerates
    every path, which grows exponentially with depth on dense graphs.
  • The fix. It now expands one hop per query, breadth-first like NetworkX,
    inside a single read transaction. Each relationship is visited once per
    level, and the ACL, validity and relation filters apply at every hop.
  • Same results. A fact is reachable through a visible path within depth
    hops exactly when one of its endpoints is reached within depth - 1, so the
    result set is unchanged.
  • Benchmark. On a 40-node clique at depth 4 the traversal takes 2.6s.
    Before this change the run didn't finish within the 4-minute limit.

Fixed — executor reports each fact's real hop distance (#103)

  • Wrong distances. DeterministicExecutor stamped every fact a traversal
    step returned with one hop. With TraversalStep(depth=3), a fact three
    hops from the seed scored like a direct neighbour, and hops_from_seed
    was wrong.
  • The fix. Each fact's distance from the step's start is now computed
    over the step's results, following the step's direction, and adds up across
    steps. Distances are computed before migration translation, so a fact the
    migration drops can't make the facts beyond it look unreachable.

Fixed — storage reads are deterministic and match across backends (#102)

  • Neo4j search applied LIMIT k to an unordered match, so the top-k set
    could change between calls. It now scores matches exactly like NetworkX
    (exact field match 1.0, otherwise by match length). Ties go to the oldest
    ingested_at, then fact_id. A raw node's placeholder name no longer
    affects the score.
  • NetworkX traverse(direction="both") returned an edge once from each
    end. It now returns each fact once.
  • Ordering. Both backends return traversals in the same ingest order.

Fixed — ask works under ontos serve and hides planner errors (#101)

  • ask can now be enabled. Nothing ever built a planner for the server,
    so ask always answered "no planner configured". Set
    ONTOS_PLANNER_BACKEND=ollama with ONTOS_PLANNER_MODEL,
    ONTOS_PLANNER_URL, ONTOS_PLANNER_TIMEOUT and ONTOS_ONTOLOGY_PATH.
    Only the local Ollama backend is offered, so planning stays inside the
    customer's VPC. An unknown backend, or a missing ontology, stops the server
    at boot.
  • Planner errors are no longer echoed. A failed plan used to return
    str(exc) to the caller and write it into the audit record; that text comes
    from the model or provider. Callers now get "the question could not be
    planned". The audit record and logs carry only the error type.

Security — exports no longer reveal entities named only by forbidden facts (#100)

  • The leak. add_fact creates an untyped endpoint node for every id it
    references. scan(), which every ontos export format uses, treated those
    ACL-less nodes as public. An export therefore dropped a restricted fact but
    still listed its endpoint, which proves the hidden fact exists. This was the
    same leak #82 and #91 fixed for search, traversal and entity search.
  • The fix, in NetworkX and Neo4j. An untyped node is now exported only
    when a fact or merge record in the same scan names it. Typed entities follow
    their own ACL as before. The privileged --include-acl-mask export is
    unchanged.

Fixed — ontos query and ontos ingest are audited (#94)

  • One Article-12 record per invocation. Both commands touched the graph
    without writing any record. They now write one each, success or failure, to
    ONTOS_AUDIT_DB_URL (the default is the local SQLite chain, as with
    ontos export). The record is written before results are printed.
    • ontos query records the plan, the returned fact ids and the
      authorization decisions.
    • ontos ingest records the ids of the facts it wrote, plus counts,
      extractor and ontology versions. Warnings and errors are left out because
      they can quote document text.
  • Fail closed. If the audit store is unavailable, neither command plans,
    extracts or reads anything.
  • ontos query --acting-on-behalf-of SUBJECT. Returns only the facts
    SUBJECT may view, resolved through the configured authz backend. The subject
    is asserted by the local operator, recorded as operator_asserted, and is
    not authenticated. Without the flag, queries still see public facts only.
  • The dev signing-key warning goes to stderr, so the JSON on stdout stays
    parseable.

Fixed — store writes never rewrite history (#95)

  • close_fact no longer rewrites a closed fact. Closing an already-closed
    fact used to move its t_invalid and replace superseded_by in both
    backends. It now raises FactNotOpenError. Closing a missing fact is still
    a no-op.
  • A duplicate add_fact is rejected in both backends with
    FactAlreadyExistsError. Before this, NetworkX replaced the fact in place
    and Neo4j created a second edge with the same fact_id.
  • Supersession is atomic. The new GraphStore.supersede(new_fact, closes)
    adds the new fact and closes the old ones together; ingest now uses it.
    Neo4j runs it in one transaction. NetworkX validates everything before
    writing. A failed supersede leaves the store unchanged, where a failed
    close used to leave two active facts in one scope.
  • Shared tests. The NetworkX compliance tests and the Neo4j integration
    tests run the same checks.

Fixed — ontos serve persists its audit trail (#93)

  • Article-12 records no longer vanish on restart. ontos serve (live and
    --snapshot) always used the in-memory audit sink and never read
    ONTOS_AUDIT_DB_URL, though the Helm chart sets it. With the variable set,
    the server now writes to the persistent hash-chained sink
    (PostgresAuditEmitter) and creates its table at startup. Without it,
    records stay in memory (dev only), and ONTOS_ENV=prod refuses to boot.
  • Tool calls, transport rejections (401/403) and the audit tool all use
    the persistent sink.
    The request path now awaits the async sink.
  • Fail closed. If an audit record cannot be written, the call fails with
    audit unavailable and returns no data. A transport rejection becomes a
    500. Sink error text is never returned or logged.
  • Records carry the real version. model_versions.runtime is now the
    package version, replacing the hard-coded 0.0.1.
  • New postgres extra (asyncpg), included in the container image.
    Before this the image could not connect to a Postgres audit DB.
    .env.example now uses async drivers (sqlite+aiosqlite://,
    postgresql+asyncpg://); the sync URLs it listed were always rejected.

Fixed — every MCP tool failure is audited

  • Some tool failures left no Article-12 record. After authentication, a
    malformed as_of, a malformed fact_id/query_id, an authz backend outage
    or a store error escaped the tool with no audit record, against the
    audit-first rule. Every tool failure now writes one record with the decision
    invalid_request, authorization_error or error.
  • Callers get a safe message. Errors return a generic ToolError and never
    echo backend text; only the error type is logged (mcp-tool-failed).
  • search k is bounded to 1–100 and traverse depth to 1–10.
    Previously NetworkX had no depth cap, a huge k returned everything, and a
    negative k silently dropped results. Out-of-range values are rejected and
    audited.
  • Gate-blocking test. It reads the tool list from the server, so any new
    tool must either be covered or be explicitly listed as having no validated
    arguments.

Fixed — security: facts exposing restricted endpoint entities

  • Read paths checked only the fact's own ACL. search, traverse,
    explain (facts_for_entity), provenance (get_visible_fact) and
    search_entities ignored the ACL of the fact's endpoint entities.
  • Effect: a public fact pointing at an entity that a restricted document
    later claimed (the #48 None → X upgrade) was returned to callers without
    access, revealing the restricted entity's id. scan() and exports already
    hid such facts; this happened on both backends, and through the executor
    and ask.
  • Now: a fact is visible only if the caller may see the fact and both
    endpoint entities. Neo4j traversal also requires every node on the path to
    be visible.
  • Compatibility: callers without access to an endpoint entity no longer
    receive facts touching it. That is the intended behaviour.

Added — sovereign ontology suggestions (#66 M2b, #89)

  • ontos ontology suggestions list --llm-model MODEL enables
    LLM-assisted ontology suggestions using a local Ollama model.
    Omitting the flag preserves deterministic M2a behavior.
  • M2a add_pattern suggestions run first. Only unresolved
    unknown relationship types reach Ollama. The result combines
    deterministic and validated LLM suggestions.
  • M2b supports map_to_existing and new_relation_type.
    New relationship names must match the observed predicate.
  • Model requests are separated by source ACL and split into
    batches of at most 100 distinct patterns per request.
  • The model receives only entity-type labels, safe predicates,
    permitted mappings and request-local candidate indexes.
    Candidate identities, source IDs and ACL values are excluded.
  • Suggestions are validated against the active ontology, with
    supporting evidence and HMAC references computed server-side.
  • --ollama-timeout SECONDS sets the request timeout, defaulting
    to 180 seconds. The value must be positive and finite.
  • M2b accepts only the approved local Ollama adapter through
    an HTTP loopback endpoint. Remote Ollama endpoints, injected
    HTTP clients, redirects, proxies and bridge LLMs are refused.
  • All requests remain authorized and audited, with generation
    mode, approved generator identity/version and invalid proposal
    counts in audit metadata. Suggestions are not persisted or
    written into the trusted graph.

Compatibility notes — ontology suggestion consumers

  • The serialized OntologySuggestion shape now includes
    mapped_relation_type, proposed_relation_type,
    observed_predicate and extractor_versions. Existing M2a
    add_pattern results include null values for the first three
    fields and an empty list for extractor_versions.
  • M2a suggestion audit arguments now include invalid_count: 0.
  • M2b audit records identify LLM-assisted generation separately
    from deterministic M2a generation. Approved generator ID and
    version are recorded in M2b outcome audit metadata.
  • Consumers using a strict JSON schema or exact audit-field
    matching must account for these additional fields.
  • --llm-model does not replace deterministic suggestions;
    it adds eligible LLM-generated suggestions.
  • M2b uses the existing trusted-local-operator identity boundary.
    No remotely callable ontology suggestion tool is introduced.
  • Approval, ontology migration and document re-ingestion remain
    outside M2b and are planned for M3.