docs: reconcile the commit-scope list with the codebase - #8
Merged
Conversation
Add the real areas that were missing (model, bundle, config) and the process scope actually in use (sdd), and state the two rules the list now follows: scope is the subsystem or domain, not the command (an init change is scoped cli), and the list grows as code lands, like packages. The unbuilt pipeline scopes stay as known roadmap.
This was referenced Jul 23, 2026
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
* feat(lifecycle): fail-safe deprecated-concept predicate (status-aware-retrieval PR1) New src/openkos/lifecycle.py leaf: deprecated_concept_ids(bundle_dir) returns concepts deprecated for retrieval — own status:deprecated OR target of any non-self supersedes edge (fail-safe: every member of any supersedes cycle, including mutual 2-cycles, is hidden as unresolved). Adds a generic filter_hits helper. Query-time filtering foundation for MVP-3 gap #8 S1; no consumers wired yet. Imports only openkos.model.okf to stay cycle-free. Includes the status-aware-retrieval SDD change artifacts (proposal/spec/design/tasks). * feat(retrieval): status-aware query filtering in answer() (status-aware-retrieval PR2) Wire the lifecycle predicate into answer(): with include_deprecated=False (default), filter FTS and vector hits before the initial fuse (so deprecated concepts never become PPR seeds) and graph hits after PPR. include_deprecated=True skips the predicate walk entirely. All four AnswerResult counters and the retrieval: stderr line report post-filter values (R3). Narrows lifecycle's _HasConceptId.concept_id to a read-only property so frozen hit dataclasses satisfy it under mypy --strict. Adds per-channel leak-safety tests plus a non-vacuous regression that a deprecated concept never becomes a graph seed. * feat(resolution): status-aware candidate filtering in contradictions and duplicates (status-aware-retrieval PR3) find_contradictions and find_candidates now exclude deprecated/superseded concepts by default via the shared lifecycle predicate, threading include_deprecated (True skips the predicate walk). In contradiction.py the filter is applied inside _candidate_pairs BEFORE the _MAX_PAIRS cap slice, so deprecated-touching pairs can never consume cap slots ahead of live pairs sorting past the cap; total_pair_count reflects the live-only count. The duplicates CLI inherits find_candidates' new default-exclude with no code change. All-live bundles are byte-identical. Adds starvation and both-sides-of-pair regressions. * feat(cli): --include-deprecated flag for query, contradictions, adjudicate, duplicates (status-aware-retrieval PR4) Expose the library-level include_deprecated on the CLI: each of the four candidate/retrieval commands gains a --include-deprecated flag (default False, so deprecated and superseded concepts are excluded by default) threaded into its answer()/find_contradictions()/find_candidates() call. Adds bidirectional CLI tests per command plus an end-to-end assertion that query's retrieval: stderr line reports post-filter counts. Documents the flag on query and duplicates in docs/cli.md. * docs(sdd): correct spec cycle scenario to fail-safe (status-aware-retrieval) Effective Status Resolution and its cycle scenario said mutual-cycle concepts must be treated as live; the shipped predicate fails safe (any-length supersedes cycle → deprecated) after the PR1 review closed a real false-negative leak. Amend spec.md to match the shipped and tested behavior so the archived spec is accurate. Documentation only; no code change.
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
… spec Merge the status-aware-retrieval delta into a canonical capability spec (openspec/specs/status-aware-retrieval/spec.md) and move the completed change folder to openspec/changes/archive/2026-07-22-status-aware-retrieval/ with its archive and verify reports. Closes MVP-3 gap #8 S1. Capabilities 24 -> 25.
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
* feat(cli): reference-aware forget with tombstones (gap #8 S2a) Make `forget` reference-aware and give it a tombstone. Phase A now scans the whole bundle once (new src/openkos/bundle/references.py detect-only helper that reuses merge's link/relation scanners) to enumerate every inbound markdown link and typed relation targeting the concept, plus outbound supersedes edges for a resurrection disclosure; the forget refuses when inbound references exist unless --force (orthogonal to the confirm gate), and the log line is upgraded from a plain marker to a **Tombstone** entry. Fail-closed on unverifiable referrers: a file whose frontmatter fails to parse but whose text mentions the target id is surfaced as an "unverifiable" reference and blocks the forget unless --force, closing a fail-open where a malformed referrer could be silently deleted-around. Extends the forget-command capability; scope is self-only (descendant cascade deferred to S2b). * style: apply ruff format to forget and its tests (gap #8 S2a) * test(forget): patch typer/references modules directly for mypy whole-tree clean (gap #8 S2a)
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
…(gap #8 S2a) Merge the reference-aware-forget delta into the canonical forget-command spec (tombstone log entry, inbound-reference detection, unverifiable fail-closed detection, refuse-unless-force, force/confirm-gate orthogonality, resurrection disclosure; remove the stale Known Limitation and the contradicted Non-Goal) and move the completed change folder to openspec/changes/archive/. Closes MVP-3 gap #8 S2a; S2b (cascade/scope-depth) deferred.
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
* feat(bundle): provenance-descendant closure helper (forget-cascade PR1)
New src/openkos/bundle/provenance.py: find_provenance_descendants computes the
orphan-after-delete cascade set for forget --scope source. A concept joins the
purge set iff its provenance is NON-EMPTY and a subset of the current set,
iterated to fixpoint (sorted, deterministic). The non-empty guard is the
over-deletion barrier — a concept with empty/absent provenance is never swallowed
into the set. Canonical-layer, no openkos.graph import. No consumers wired yet
(the forget cascade wiring is PR2). Includes the change's SDD artifacts.
* feat(cli): forget --scope source cascade over provenance descendants (forget-cascade PR2)
Wire --scope {self,source} into forget. `self` is byte-identical to S2a via a
unified size-1 purge-set data path; `source` cascades over the orphan-closure
purge set from find_provenance_descendants. Over the set: per-member inbound-ref
detection with a set-difference drop (intra-set backlinks never block; external
refs and unverifiable referrers over every member id still refuse unless --force),
a full-set preview with a Total count line, a count-stating confirm prompt
orthogonal to --force, N tombstone lines in ascending id order, and N unlinks LAST
in sorted order (catalog-before-file, non-transactional, git-recoverable). On a
partial-cascade failure the error reports how many of N were removed. Path-safety
runs on the root first; descendant ids are disk-discovered, never user input.
* docs(sdd): check off phase 4-6 tasks, spec the K-of-N partial-failure report (forget-cascade)
Mark the reference-aware-forget-cascade Phase 4-6 tasks complete on disk (they
were only recorded in Engram), and add the K-of-N partial-cascade error report
to the Catalog-Before-File Write Ordering requirement so the shipped behavior is
in the delta spec before archive.
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
…nd spec (gap #8 S2b) Merge the S2b cascade delta into the canonical forget-command spec (add Scope Selection, Provenance Descendant Resolution, Full-Set Preview + Count Confirmation; revise inbound detection, unverifiable detection, refuse gate, log entry, resurrection disclosure, and catalog-before-file ordering to operate over the purge set with a K-of-N partial-failure report) and move the completed change folder to openspec/changes/archive/. Closes MVP-3 gap #8 S2b.
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
* docs(sdd): spec/design/tasks for sensitivity-fail-closed-filter (gap #8 S3) * feat: sensitivity fail-closed predicate + S1-pattern seams (gap #8 · S3a) Add src/openkos/sensitivity.py: sensitive_concept_ids(bundle_dir, threshold) computes the confidential-blocked concept id set in one okf._iter_docs walk, failing closed on absent/blank sensitivity, read/parse errors, and unknown values (never delegating absent/blank to okf._rank, which resolves those to private). Wire the predicate into the four S1-pattern seams that reuse lifecycle.filter_hits/member-drop shape: query (retrieval/answer.py), contradictions (resolution/contradiction.py), adjudicate (resolution/adjudication.py), and suggest-relations (resolution/edge_typing.py). Add --include-confidential to query, contradictions, adjudicate, and suggest-relations, mirroring --include-deprecated's zero-cost escape. * feat: sensitivity divergent seams + extract floor gate (gap #8 · S3b) Thread LintDoc.sensitivity through lint.py's collect_docs and filter blocked docs from resolution/volatility_typing.py before per-type sampling; add --include-confidential to suggest-volatility. Gate extract's llm.chat call on the workspace default_sensitivity floor in cli/main.py::_stage_derived_objects (extract has no per-doc sensitivity value, unlike the other five llm.chat seams); add --include-confidential to ingest to bypass the floor. Add a defense-in-depth guarded re-read skip in retrieval/answer.py::_assemble_context so a confidential concept id is excluded even if it slipped past the upstream hit-seam filter. * fix: close extract floor-gate fail-open on blank default_sensitivity (gap #8 · S3) _stage_derived_objects gated extraction by calling okf._rank directly on cfg.default_sensitivity, but okf._rank(None)/_rank("") resolve to "private" for the unrelated combine_sensitivity merge-floor use case -- so a blank or whitespace default_sensitivity in config never tripped the confidential floor gate and let raw source text reach llm.chat unfiltered. Introduce sensitivity.blocks_llm_send as the one shared fail-closed authority (absent/blank always blocks, present values delegate to okf._rank) and delegate both the per-doc predicate and the extract floor gate to it, removing the duplicated inline fail-closed logic. Proven RED-first: a default_sensitivity: "" fixture previously called llm.chat; it now short-circuits with the existing Source-only degrade message. * fix: close sensitivity walk-bypass leak in query context assembly (gap #8 · S3) sensitive_concept_ids discovers confidential docs via one okf._iter_docs walk, which can silently drop a subtree it cannot list. _assemble_context re-reads each hit doc by direct path, which succeeds even when the doc was invisible to that walk -- a confidential concept known only through the persisted FTS/vector/graph index could reach llm.chat despite the pre-filter. _assemble_context now independently re-checks each doc's own freshly re-read frontmatter against sensitivity.blocks_llm_send at the actual send point, walk-independent of the precomputed blocked set. include_confidential skips this re-check too, preserving byte-identical opt-in behavior. This is query-specific: every other llm.chat seam derives its candidates from the same walk-based predicate, so a walk-invisible doc is never a candidate there and needs no change (recorded as a follow-up in design.md). Proven RED-first via a monkeypatched sensitive_concept_ids returning an empty frozenset (simulating a walk that missed the doc entirely). * fix: remove dead LintDoc.sensitivity field (gap #8 · S3) LintDoc.sensitivity was populated in collect_docs but never read anywhere: suggest_volatility filters via the shared sensitivity.sensitive_concept_ids predicate, not doc.sensitivity. An unread field carrying a security-relevant raw value is an attractive nuisance -- a future maintainer could inline okf._rank(doc.sensitivity) and reintroduce a fail-open bypass of the shared predicate. Removed the field, its population, and its two isolated tests; suggest suggest_volatility continues to use the shared fail-closed predicate. * docs: document --include-confidential across sensitivity-aware CLI verbs Added --include-confidential to query's and ingest's existing flag tables (both already documented --include-deprecated/--auto). contradictions, adjudicate, suggest-relations, and suggest-volatility had no docs/cli.md section at all (a pre-existing MVP-2/3 documentation gap, not introduced by this feature) -- added minimal (MVP 3)-tagged sections for each, with a flag table covering every flag they accept. * docs(sdd): record correction batch and follow-ups for sensitivity-fail-closed-filter (gap #8 S3) Append design.md's "Known follow-ups (harden before cloud/export slice)" note (shared-walk perf, walk-bypass observability) and tasks.md's "Correction batch (post-4R-review)" section documenting the 4 fixes applied on top of Phase 1+2, with their RED/GREEN/REFACTOR evidence and the whole- tree CI verification.
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
…#117) * fix: wrap TypeError from yaml.safe_load as ValueError in read_config (gap #8 · S3c) except yaml.YAMLError alone leaves a raw TypeError uncaught if a PyYAML constructor code path raises it directly (e.g. some versions do this for a mapping with an unhashable complex key) instead of a YAMLError subclass. Callers only guard (OSError, ValueError), so it could escape as an uncaught traceback. Widen the except tuple to (yaml.YAMLError, TypeError). RED test forces the scenario via monkeypatching yaml.safe_load, since the pinned PyYAML 6.0.3 pure-Python SafeLoader already wraps every unhashable-complex-key shape tried as ConstructorError (a YAMLError subclass), so it does not reproduce via real YAML content in this environment today. * refactor: extract shared llm JSON parsing helpers, delete 5 clones (gap #8 · S3c) adjudication.py, edge_typing.py, volatility_typing.py, and contradiction.py each carried a byte-identical module-local _extract_json_object trio (_strip_code_fence/_first_brace_block/ _extract_json_object), and extraction/concept.py carried a list-variant _extract_json_items. All five existed only because of a 'no cross-import of _-prefixed symbols' design note. Add src/openkos/llm/parsing.py, a stdlib-only leaf (mirrors llm/base.py) exposing PUBLIC extract_json_object/extract_json_items -- public names resolve that note's intent without violating it. Migrate all 5 call sites to import and call it, deleting every local clone. Pure refactor, zero behavior change, verified byte-for-byte via each call site's existing test suite staying green plus a new focused tests/unit/llm/test_parsing.py for the shared module. contradiction.py's tests called the removed private clone directly; updated those 3 assertions to call openkos.llm.parsing.extract_json_object instead. * docs(sdd): mark Phase 3 (S3c hygiene) tasks complete (gap #8 · S3)
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
…are-llm capability spec (gap #8 S3)
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
…low-up) (#118) * docs(sdd): spec/design/tasks for directory-walk-observability (S3 follow-up) * feat: warn on incomplete directory walk, close 4 sensitivity leak paths (gap #8 · S3 follow-up) Adds cli/observability.warn_if_walk_incomplete: a shared STDERR signal (mode="warn", exit 0) wired into query/adjudicate/suggest-relations/ suggest-volatility/contradictions when okf._walk_errors shows the bundle scan was incomplete, so the confidential-content filter's coverage gap is never silent. --include-confidential suppresses it since the filter is already off. mode="refuse" is a stable, unimplemented seam for a future cloud-egress mode. Also ports query's existing send-time re-check (sensitivity.blocks_llm_send on freshly re-read frontmatter) into the four remaining load paths that still trusted the precomputed blocked-id set: contradiction._load_doc, edge_typing._load_doc, adjudication._load_members, and a new per-doc re-read guard in volatility_typing (LintDoc carries no sensitivity field). A confidential document the walk missed can no longer reach an llm.chat payload through any of the five verbs. sensitivity.py and retrieval/answer.py are unchanged. * refactor: centralize fail-closed sensitivity predicate into should_block The identical inline check `not include_confidential and sensitivity.blocks_llm_send(metadata.get("sensitivity"))` was duplicated across 5 send-time re-check sites (query, contradictions, adjudicate, suggest-relations, suggest-volatility). A future edit to the semantic could update fewer than all 5, silently reintroducing a partial fail-open. Add a pure `sensitivity.should_block(metadata, *, include_confidential)` predicate and adopt it at all 5 sites. Each site keeps its own action on block (skip/degrade/continue/filter); only the boolean decision is now centralized. Behavior-preserving -- all existing leak-closure and query tests stay green. sensitivity.py remains a pure no-I/O leaf. * fix: thread include_confidential into volatility guard, scope reread to sampled docs _reread_sensitivity_blocked(doc) omitted include_confidential, unlike its 3 siblings that take the flag directly -- correctness depended entirely on the call site wrapping every invocation in `if not include_confidential:`. A maintainer calling the guard directly (by analogy with the other 3) could easily reintroduce a fail-open bypass. The guard now takes include_confidential itself and short-circuits to False before any read, preserving the original zero-I/O bypass cost. Also stop re-reading every bundle doc's frontmatter before sampling only N_SAMPLE_CONCEPTS=5 per type -- this verb's ids come from a live lint.collect_docs walk (not graph.db), so the walk-miss leak this guard defends against cannot fire here; a full-bundle re-read was needless I/O for the weakest-leverage guard of the four. _sample_bodies_by_type is renamed _sample_docs_by_type and now returns LintDoc objects so the walk-independent recheck applies only to the small sampled-per-type set, after sampling. The upstream sensitive_concept_ids walk-based blocked filter still applies to the full bundle unchanged (cheap id-membership check, no I/O). New tests confirm: the guard is call-count-bounded to the sampled subset, a confidential doc that would be sampled is still excluded (fail-closed preserved), and a private sampled doc is still sent. * docs(sdd): record correction batch, defer query hot-path walk sharing Add a "Correction batch (post-4R-review)" section to tasks.md documenting the 3 fixes (centralized should_block, volatility guard signature symmetry, sampled-only reread scoping) plus a spec-check confirming no requirement changed. Correct the "27/27 tasks" status wording, which conflated the 25-task-item count with the 27-new-test count. Add a "Known Follow-ups" section to design.md documenting the deferred query hot-path multi-walk cost (warn_if_walk_incomplete + deprecated_concept_ids + sensitive_concept_ids each walk the bundle independently) -- the real fix is sharing one walk (design option 3c), reserved for a later slice, not implemented here.
jasonssdev
added a commit
that referenced
this pull request
Jul 23, 2026
…e 2, MVP-2 complete) (#121) * docs(sdd): spec/design/tasks for privacy-purge-history-scrub (Slice 2, complete RTBF) * feat: scrub index.md/log.md history in the same purge rewrite (gap #8 · S2) Completes right-to-be-forgotten: purge now content-scrubs every historical bundle/index.md and bundle/log.md blob (catalog bullets, log entries, and forget tombstones for the purge set) in the SAME git-filter-repo pass as Slice 1's whole-file expunge, via a static --file-info-callback snippet with zero subject-data interpolation. Also cleans live log.md tombstones (_purge_clean_live_log) and removes the now-obsolete residual-leak warning, since no residual is left to warn about. * test: cover history-scrub parity, collision-safety, and live cleanup (gap #8 · S2) Adds a multi-commit fixture (residual in an earlier commit, rewritten later) plus: a parity test proving the snippet's bytes identity-matcher and bundle.index._link_identity agree on every case; collision-safety tests proving a surviving sibling's bullet, a prose-only id mention, and an unrelated concept's body all round-trip byte-identical while the purge target's bullet/tombstone are gone from every historical commit; and CLI/ unit coverage for remove_log_entry, live log.md tombstone cleanup, and the removed residual warning. * docs(sdd): mark privacy-purge-history-scrub Slice 2 tasks complete All 30 tasks (sections 0-4) implemented and verified; records the apply-time LOC actuals (~1125) against the 800-line budget forecast for the orchestrator/reviewer to resolve. * fix: prevent purge temp-file leak and map setup OSError to GitError Each expunge_paths temp file (paths/snippet/sidecar) now assigns its Path to the tracking variable immediately on creation, before its write loop runs, so the finally block unlinks it even if that write raises mid-loop (e.g. ENOSPC/EIO/quota) -- most load-bearing for the sidecar file, which holds the plaintext purge-set identities being erased. An OSError during temp-file setup is now mapped to GitError (pre-subprocess, so history is guaranteed NOT rewritten), preserving the CLI's GitError/GitFinalizeError distinction and messaging. * fix: reconcile bytes/Python link-identity divergence on multi-leading-slash targets Both openkos.bundle.index._link_identity and the vcs/git.py bytes _FILE_INFO_CALLBACK_SNIPPET twin now strip ALL leading slashes before further normalization, so a target like //concepts/foo.md resolves to the same identity (concepts/foo) on both sides -- previously the Python side kept a spurious leading // for multi-slash targets while the bytes side normalized correctly, an under/over-scrub risk for the irreversible purge history-scrub. Added a lockstep NOTE cross-reference from _link_identity to its bytes twin and the parity test. * docs: update purge section for complete right-to-be-forgotten Removes the Slice-1-era mandatory residual-leak warning and its 'not complete right-to-be-forgotten' language, and documents that purge now content-scrubs index.md/log.md across all history and cleans up the live log.md tombstone in the same rewrite pass. * docs(sdd): record correction batch and size:exception acceptance in tasks.md Documents the 4R correction batch (temp-file leak, OSError mapping, index.md anchor asymmetry, bytes/Python identity divergence, cross-ref comment, stale docs, tightened collision-safety test) and records the orchestrator's size:exception acceptance for the test-dense safety coverage.
jasonssdev
added a commit
that referenced
this pull request
Aug 9, 2026
#389) (#501) Refs #389, carrying its help-surface bullets. The remaining four stay open. Typer publishes each command's raw `__doc__` unless the command sets `help=`, and none did, so an end user read "Wires the pure `bundle.provenance.resolve_backfill_raises` sweep core (design D4/D5) into Typer's confirm-gate", plus `decision #717`, `MVP-3 gap #8 S2a`, `Slice 1` and `ADR-0003`. There was never a choice to make: `help=` decouples the published text from the docstring. All 26 commands now publish a user-facing line while their docstrings keep every reference maintainers rely on. This matters past `--help` -- MCP tool descriptions are usually derived from the same source. The commands were also listed flat in declaration order, which put `purge` -- irreversible and rare -- fourth while `query`, the value moment, sat near the bottom. They are now grouped into five panels by what the reader is trying to do, with the rule written down beside the app. Grouping alone made one thing worse, visible only on rendering the real output: Rich prints a panel when it first meets a command belonging to it, so declaration order still decided panel order and "Remove" landed SECOND. `PANEL_ORDER` plus a stable sort of the registry makes reading order explicit. Three review rounds, each catching what the last missed. Round 2 found two published lines that were clean but WRONG -- `adjudicate` read as though invoking it performs merges when its plain invocation is read-only -- and an internal token still published on `forget --scope`. Round 3 fixed an anchor in one of the patterns round 2 added, which matched a bare slice number but missed the lettered forms this codebase mostly uses. Separately, the sweep's first draft dropped a bounded-scope honesty statement from `set-sensitivity`'s published help, and the existing suite caught it. Reviews: `review-6e884bb4642ac6fb`, `review-f3a75ff7d21f93f8` and `review-5c4e2c90128c01d4`, all approved with receipts and zero blockers. The `pre-pr` gate denied on receipt scope, not review coverage: each receipt binds its own increment, so none covers the branch union. Merged under explicit maintainer action, disclosed in the pull request body.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AGENTS.md's Conventional-Commits scope list with what the repo actually has and uses: addsmodel,bundle,config(real packages that were missing) andsdd(the process scope already used ~10× foropenspec/artifacts).init/ingest/querychange is scopedcli), and the list grows as code lands (per the "create a package when its code arrives" principle). Unbuilt pipeline scopes stay as known roadmap.Type of change
docs— documentation only (no code)Changes
AGENTS.mdmodel, bundle, config, sdd; adds the subsystem-not-command and grows-with-code rulesHow was this tested?
Doc-only; no code, tests unaffected. Verified against the actual
src/openkos/packages (model,bundle,cli,config) and the scopes present in git history (cli,sdd,okf).Checklist