Local-first compute policy: maximize safe local-model offload

[sergstack]

Goal

Define the AI-OS policy for using local models wherever they are safe, useful, measurable, and cheaper than frontier reasoning, without weakening evidence, authority, routing, or acceptance semantics.

This issue owns policy and promotion rules only. Execution mechanics belong to sergstack/local-developer-worker (LDW).

Related LDW implementation issue: sergstack/local-developer-worker#56.

Architectural boundary

AI-OS owns:
- task-class policy
- offload eligibility
- risk / quality floor
- evidence semantics
- promotion gates
- fallback requirements
- owner / authority boundaries

LDW owns:
- bounded context execution
- local Ollama calls
- schema validation
- routing mechanics
- telemetry
- fallback / escalation execution

Local-model output is never silently promoted from candidate inference to observed fact or owner authority.

Core decision rule

Before using expensive frontier reasoning, ask:

Can this step be safely offloaded to a local model and verified cheaply enough that the expected error is detectable before it affects acceptance or an external side effect?

If yes: local-first candidate.
If no: frontier / deterministic / human path as appropriate.

Initial task classes

Local-first candidates

Candidate classes to evaluate for promotion:

• high-volume log classification / clustering;
• bulk extraction from bounded text;
• summarization / compaction of non-authoritative context;
• repo/file triage and candidate ranking;
• normalization / restructuring into a fixed schema;
• duplicate / similarity candidate detection;
• initial hypothesis generation;
• low-risk terminal triage;
• bounded document/text classification;
• first-pass anomaly candidate generation.

Local-candidate with mandatory stronger review

• code-review first pass;
• debugging hypotheses;
• test-failure explanation;
• architectural alternatives generation;
• analytical narrative drafts;
• risk candidate generation.

These outputs remain candidate-only until the owning project or stronger model verifies the material conclusion.

Frontier/human floor

Do not promote to local-first without separate evidence:

• material architecture decisions;
• governance changes;
• acceptance / Closure Review decisions;
• production / merge / deploy authorization;
• residual-risk acceptance;
• material financial conclusions where the local model output itself would determine the result;
• owner-only business decisions.

Progressive Disclosure policy

AI-OS should require minimum-sufficient context for material Codex work:

goal / constraints
-> bounded context pack
-> inspect result
-> expand only on evidenced missing context

Broad repository reading is not the default for unfamiliar multi-file work when LDW context-pack capability is available.

Policy requirements:

• disclose context incrementally;
• preserve explicit inclusion/exclusion reasons;
• require a missing-context reason or deterministic trigger for expansion;
• do not infer token savings unless measured;
• exact requirements, IDs, authority, acceptance criteria, open defects and evidence refs must not be lost through compaction.

Summarization / compaction policy

Summarization is permitted only with a loss-aware preservation contract.

Preserve exactly or source-reference durably

• original goal / task boundary;
• constraints;
• requirement IDs and current states;
• defect IDs and open defects;
• current iteration / continuation state when applicable;
• acceptance criteria;
• authority status;
• source / evidence references;
• unresolved unknowns / blockers;
• actions already attempted when repetition would be harmful.

May be compressed

• repetitive discussion;
• explored low-value alternatives;
• verbose logs already represented by deterministic evidence;
• intermediate narrative not carrying unique constraints or evidence.

A local summary is a candidate representation. It must not replace authoritative source evidence when exact source content is required for a material decision.

Offload promotion contract

A task class may move into the local-first allowlist only after a matched evaluation records:

• representative task corpus;
• deterministic or stronger-model acceptance check;
• quality / error comparison against the current control;
• latency;
• frontier-provider token use where observable;
• bytes/context presented to frontier model where observable;
• local compute burden where observable;
• failure modes;
• false-accept / false-reject behavior where applicable;
• fallback rate;
• privacy / data-boundary assessment;
• rollback / disable path.

Synthetic evidence may prove technical shape but cannot by itself justify production promotion of a real task class.

Evidence semantics

Required distinctions:

• deterministic observation != model inference;
• local-model candidate != verified fact;
• verified execution != semantic-quality acceptance;
• summary != source evidence;
• model confidence != authority;
• frontier review != owner approval.

If a local result materially affects a decision, the consuming owner must know its provenance and verification status.

Tool-selection policy

For equivalent outcomes prefer, in order where applicable:

1. deterministic local operation;
2. bounded retrieval / search;
3. local-model semantic operation with verification;
4. frontier-model reasoning;
5. human material decision.

Do not use semantic inference for a result that an existing deterministic tool can establish authoritatively.

Examples:

• test status -> test parser, not model interpretation;
• Git state -> Git facts, not model inference;
• file discovery -> deterministic inventory/search before broad semantic reading;
• large semantic candidate set -> local triage before frontier inspection when verified safe.

KPI

Primary KPI:

Increase verified useful work per unit of frontier context, frontier tokens, latency and manual attention without increasing undetected semantic error or authority risk.

Measure per promoted class where available:

• local-offload rate;
• frontier-token reduction;
• bytes/context reduction;
• matched latency delta;
• verification failure rate;
• fallback/escalation rate;
• false acceptance rate;
• repeated context expansion;
• manual interventions;
• local compute time / resource use.

Do not optimize token reduction at the expense of correctness or evidence integrity.

Governance / ownership

AI-OS owns:

• canonical task-class registry / policy;
• risk floor;
• allow / candidate / blocked status;
• evidence requirements;
• promotion / rollback decision;
• cross-project semantics.

LDW must not autonomously promote a task class based only on telemetry or successful local execution.

Initial phases

P0 — capability and gap audit

Map current AI-OS + LDW capabilities for:

• progressive disclosure;
• summarization / context compaction;
• local offload;
• adaptive model routing;
• deterministic verification;
• tool/context telemetry.

Classify each as:

• already sufficient;
• partial gap;
• material gap;
• not needed.

P1 — policy registry + first real allowlist pilots

Use existing LDW capabilities first. Prioritize classes with high volume, low downside and cheap verification.

Do not build a new worker if LDW already supplies the required execution primitive.

P2 — expand local-first classes by measured evidence

Promote only after matched real-task studies.

P3 — automated policy-assisted routing

Only after several promoted classes show stable quality and measurable benefit. Automation must remain bounded by explicit policy and preserve fallback/escalation.

Hard constraints

1. No local model receives owner authority.
2. No automatic acceptance / merge / deploy / production authorization.
3. No task-class promotion from synthetic evidence alone.
4. No raw secret/provider payload persistence solely for optimization.
5. No weakening deterministic validation to improve local-model success rate.
6. No silent fallback that changes the evidence class.
7. No duplicated local-offload policy inside each project; AI-OS owns the shared policy.
8. LDW remains execution/evidence machinery, not semantic owner.
9. Local inference endpoint/security constraints remain fail-closed.
10. Every promoted class has an explicit disable/rollback path.

Acceptance criteria

• AI-OS/LDW ownership boundary is explicit and non-overlapping;
• task classes can be represented as local-first / candidate-review / frontier-floor / blocked;
• promotion requires measured representative evidence;
• progressive disclosure and compaction preservation rules are explicit;
• deterministic tools remain authoritative where applicable;
• local output provenance survives into material downstream decisions;
• rollback removes a promoted local route without changing canonical AI-OS semantics;
• no change grants local models authority over acceptance or side effects.

Revisit triggers

• new local model / hardware materially changes capability;
• a promoted class shows quality regression;
• frontier cost/latency changes materially;
• LDW execution semantics change;
• a new deterministic verifier makes another class safe to offload;
• context/tool telemetry shows a new high-volume opportunity.

Initial handoff

Next owner for execution design: sergstack/local-developer-worker.

AI-OS should first finalize policy/classification requirements; LDW should then audit which mechanics already exist and implement only the missing bounded primitives.

Production promotion is not authorized by this issue alone.

[sergstack]

Linked execution issue: sergstack/local-developer-worker#56 — LDW owns the bounded mechanics for context compaction, local-model offload, verification, telemetry, fallback/escalation, and matched real-task evaluation. AI-OS #345 remains the policy/promotion owner.

[sergstack]

Implementation is ready in PR #347.

Delivered: canonical policy, schema-validated task-class registry, P0 capability/gap audit, regression tests, generated AI-OS Knowledge bundles, and provenance refresh. The related LDW mechanics work remains owned by sergstack/local-developer-worker#56; no duplicate issue was created.

The production local_first allowlist remains intentionally empty until representative matched evidence and owner promotion. No local model receives acceptance, side-effect, merge, deploy, or production authority.

PR: #347