Skip to content

Commit 42cd22f

Browse files
committed
feat: ship multi-stage guardrails — ingest-router + read-router + adaptive memory-router + composition
Ships three new agentos primitives + a self-calibrating extension to memory-router. Together with the existing memory-router and query-router, they form the agentos multi-stage guardrails pattern: every pipeline boundary has an LLM-as-judge orchestrator that classifies its input and picks a strategy. @framers/agentos/ingest-router (NEW): - Input-stage orchestrator. Classifies content (short/long-conversation, long-article, code, structured-data, multimodal) and picks a storage strategy (raw-chunks, summarized, observational, fact-graph, hybrid, skip). - Four shipping presets: raw-chunks (default), summarized, observational, hybrid. - Provider-agnostic LLMIngestClassifier with base + few-shot prompts. - FunctionIngestDispatcher with registry-of-executors pattern. - Budget-aware dispatch with three modes (hard / soft / cheapest-fallback). - 38 contract tests across routing tables, pure selectIngestStrategy, classifier parsing, dispatcher routing, top-level decide+decideAndDispatch. @framers/agentos/read-router (NEW): - Read-stage orchestrator. Classifies query+evidence into one of five intents (precise-fact, multi-source-synthesis, time-interval, preference-recommendation, abstention-candidate) and picks a reader strategy (single-call, two-call-extract-answer, commit-vs-abstain, verbatim-citation, scratchpad-then-answer). - Three shipping presets: precise-fact (default), synthesis, temporal. - Provider-agnostic LLMReadIntentClassifier. - FunctionReadDispatcher. - Budget-aware dispatch. - 26 contract tests. @framers/agentos/memory-router/adaptive (NEW, extends existing): - AdaptiveMemoryRouter: self-calibrating router that derives routing tables from a workload-specific calibration dataset. Consumers run Phase A on their workload and get a tailored router instead of using LongMemEval-S Phase B presets. - aggregateCalibration: rolls (category, backend) samples into mean cost + mean accuracy + n cells. - selectByPreset: picks a backend per category given aggregated calibration + a preset rule (minimize-cost / balanced / maximize- accuracy). - buildAdaptiveRoutingTable: builds a complete frozen RoutingTable with calibrated picks where data is available; preset fallback elsewhere. - 13 contract tests. @framers/agentos/multi-stage-guardrails (NEW): - MultiStageGuardrails class composes IngestStage + RecallStage + ReadStage into one pipeline. - IngestStage / RecallStage / ReadStage are minimal pluggable interfaces — agentos routers satisfy them via thin adapters (ingestRouterAsStage / memoryRouterAsStage / readRouterAsStage), custom implementations slot in too. - ingest / recall / read methods for independent stages; recallAndRead for end-to-end. - 10 contract tests. Total: 87 new tests (163 total across all four memory-router family modules). All four modules build clean and load at runtime via their respective package.json subpaths (./ingest-router, ./read-router, ./memory-router, ./multi-stage-guardrails). Each module ships a README documenting public types, usage patterns (decide-only, decide+dispatch, manual override, budget-aware, custom implementations), and the relationship to sibling primitives. Same single-OpenAI-key reproducibility as memory-router: classifiers talk to provider-agnostic LLM adapter interfaces; no SDK imports inside any module.
1 parent bb14d70 commit 42cd22f

25 files changed

Lines changed: 4017 additions & 0 deletions

package.json

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -263,6 +263,21 @@
263263
"default": "./dist/memory-router/index.js",
264264
"types": "./dist/memory-router/index.d.ts"
265265
},
266+
"./ingest-router": {
267+
"import": "./dist/ingest-router/index.js",
268+
"default": "./dist/ingest-router/index.js",
269+
"types": "./dist/ingest-router/index.d.ts"
270+
},
271+
"./read-router": {
272+
"import": "./dist/read-router/index.js",
273+
"default": "./dist/read-router/index.js",
274+
"types": "./dist/read-router/index.d.ts"
275+
},
276+
"./multi-stage-guardrails": {
277+
"import": "./dist/multi-stage-guardrails/index.js",
278+
"default": "./dist/multi-stage-guardrails/index.js",
279+
"types": "./dist/multi-stage-guardrails/index.d.ts"
280+
},
266281
"./memory/index": {
267282
"import": "./dist/memory/index.js",
268283
"default": "./dist/memory/index.js",

src/ingest-router/IngestRouter.ts

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
/**
2+
* @file IngestRouter.ts
3+
* @description Top-level input-stage orchestrator that composes the
4+
* ingest classifier and the pure {@link selectIngestStrategy} into a
5+
* single per-content routing call.
6+
*
7+
* Same shape as MemoryRouter (recall-stage) so the multi-stage guardrails
8+
* orchestrator can compose them uniformly. Decide-only and decide+dispatch
9+
* flows are both supported.
10+
*
11+
* @module @framers/agentos/ingest-router/IngestRouter
12+
*/
13+
14+
import type {
15+
IIngestClassifier,
16+
IngestClassifierResult,
17+
} from './classifier.js';
18+
import {
19+
DEFAULT_INGEST_COSTS,
20+
type IngestStrategyCostPoint,
21+
} from './costs.js';
22+
import {
23+
PRESET_INGEST_TABLES,
24+
type IngestContentKind,
25+
type IngestRouterPreset,
26+
type IngestRoutingTable,
27+
type IngestStrategyId,
28+
} from './routing-tables.js';
29+
import {
30+
selectIngestStrategy,
31+
type IngestBudgetMode,
32+
type IngestRoutingDecision,
33+
} from './select-strategy.js';
34+
import type {
35+
IIngestDispatcher,
36+
IngestDispatchResult,
37+
} from './dispatcher.js';
38+
39+
export interface IngestBudgetPolicy {
40+
readonly perIngestUsd?: number;
41+
readonly mode?: IngestBudgetMode;
42+
}
43+
44+
export interface IngestRouterOptions {
45+
readonly classifier: IIngestClassifier;
46+
readonly preset?: IngestRouterPreset;
47+
readonly routingTable?: IngestRoutingTable;
48+
readonly mapping?: Partial<Record<IngestContentKind, IngestStrategyId>>;
49+
readonly budget?: IngestBudgetPolicy;
50+
readonly strategyCosts?: Readonly<
51+
Record<IngestStrategyId, IngestStrategyCostPoint>
52+
>;
53+
readonly useFewShotPrompt?: boolean;
54+
readonly dispatcher?: IIngestDispatcher<unknown, unknown>;
55+
}
56+
57+
export interface IngestRouterDecideOptions {
58+
/**
59+
* Optional manual override of the classifier. When set, the classifier
60+
* is NOT invoked and the routing table is consulted with this kind
61+
* directly. Useful when the caller already knows the content kind
62+
* (e.g., file extension determines code vs structured-data).
63+
*/
64+
readonly manualKind?: IngestContentKind;
65+
readonly groundTruthKind?: IngestContentKind | null;
66+
readonly useFewShotPrompt?: boolean;
67+
}
68+
69+
export interface IngestRouterDecision {
70+
readonly classifier: IngestClassifierResult;
71+
readonly routing: IngestRoutingDecision;
72+
}
73+
74+
export interface IngestRouterDispatchedResult<TOutcome> {
75+
readonly decision: IngestRouterDecision;
76+
readonly outcome: TOutcome;
77+
readonly strategy: IngestStrategyId;
78+
}
79+
80+
export class IngestRouterDispatcherMissingError extends Error {
81+
constructor() {
82+
super(
83+
'IngestRouter.decideAndDispatch requires a dispatcher. ' +
84+
'Either pass a dispatcher in options or call `decide` and dispatch yourself.',
85+
);
86+
this.name = 'IngestRouterDispatcherMissingError';
87+
}
88+
}
89+
90+
/**
91+
* Public input-stage orchestrator. One instance per ingest endpoint;
92+
* reuse across content events.
93+
*/
94+
export class IngestRouter {
95+
private readonly classifier: IIngestClassifier;
96+
private readonly preset: IngestRouterPreset;
97+
private readonly routingTable: IngestRoutingTable;
98+
private readonly budgetPerIngestUsd: number | null;
99+
private readonly budgetMode: IngestBudgetMode;
100+
private readonly strategyCosts: Readonly<
101+
Record<IngestStrategyId, IngestStrategyCostPoint>
102+
>;
103+
private readonly defaultUseFewShotPrompt: boolean;
104+
private readonly dispatcher: IIngestDispatcher<unknown, unknown> | null;
105+
106+
constructor(options: IngestRouterOptions) {
107+
this.classifier = options.classifier;
108+
this.preset = options.preset ?? 'raw-chunks';
109+
this.dispatcher = options.dispatcher ?? null;
110+
111+
const baseTable = options.routingTable ?? PRESET_INGEST_TABLES[this.preset];
112+
if (options.mapping) {
113+
const patched: Record<IngestContentKind, IngestStrategyId> = {
114+
...baseTable.defaultMapping,
115+
};
116+
for (const key of Object.keys(options.mapping) as IngestContentKind[]) {
117+
const ov = options.mapping[key];
118+
if (ov) patched[key] = ov;
119+
}
120+
this.routingTable = Object.freeze({
121+
preset: baseTable.preset,
122+
defaultMapping: Object.freeze(patched),
123+
});
124+
} else {
125+
this.routingTable = baseTable;
126+
}
127+
128+
this.budgetPerIngestUsd = options.budget?.perIngestUsd ?? null;
129+
this.budgetMode = options.budget?.mode ?? 'cheapest-fallback';
130+
this.strategyCosts = options.strategyCosts ?? DEFAULT_INGEST_COSTS;
131+
this.defaultUseFewShotPrompt = options.useFewShotPrompt ?? false;
132+
}
133+
134+
async decide(
135+
content: string,
136+
options?: IngestRouterDecideOptions,
137+
): Promise<IngestRouterDecision> {
138+
let classifier: IngestClassifierResult;
139+
if (options?.manualKind) {
140+
classifier = {
141+
kind: options.manualKind,
142+
tokensIn: 0,
143+
tokensOut: 0,
144+
model: 'manual',
145+
};
146+
} else {
147+
const useFewShot =
148+
options?.useFewShotPrompt ?? this.defaultUseFewShotPrompt;
149+
classifier = await this.classifier.classify(
150+
content,
151+
useFewShot ? { useFewShotPrompt: true } : undefined,
152+
);
153+
}
154+
155+
const routing = selectIngestStrategy({
156+
predictedKind: classifier.kind,
157+
groundTruthKind: options?.groundTruthKind ?? null,
158+
config: {
159+
table: this.routingTable,
160+
budgetPerIngestUsd: this.budgetPerIngestUsd,
161+
budgetMode: this.budgetMode,
162+
strategyCosts: this.strategyCosts,
163+
},
164+
});
165+
166+
return { classifier, routing };
167+
}
168+
169+
async decideAndDispatch<TOutcome, TPayload = undefined>(
170+
content: string,
171+
dispatchPayload?: TPayload,
172+
options?: IngestRouterDecideOptions,
173+
): Promise<IngestRouterDispatchedResult<TOutcome>> {
174+
if (!this.dispatcher) {
175+
throw new IngestRouterDispatcherMissingError();
176+
}
177+
178+
const decision = await this.decide(content, options);
179+
const dispatched = (await this.dispatcher.dispatch({
180+
strategy: decision.routing.chosenStrategy,
181+
content,
182+
payload: dispatchPayload as unknown,
183+
})) as IngestDispatchResult<TOutcome>;
184+
185+
return {
186+
decision,
187+
outcome: dispatched.outcome,
188+
strategy: dispatched.strategy,
189+
};
190+
}
191+
}

src/ingest-router/README.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# @framers/agentos/ingest-router
2+
3+
Input-stage LLM-as-judge orchestrator. Classifies incoming content and picks an ingest strategy: `raw-chunks`, `summarized`, `observational`, `fact-graph`, `hybrid`, or `skip`. Sibling of `@framers/agentos/memory-router` (recall-stage) and `@framers/agentos/read-router` (read-stage).
4+
5+
## Why route at ingest
6+
7+
Different content benefits from different storage. A 3-turn chat snippet doesn't justify the LLM cost of observation extraction; a 50-turn customer-support thread does. A long-form article benefits from session-summarized contextual retrieval; structured CSV data doesn't. The router takes the content (plus an optional manual override), classifies it, and picks the storage strategy that the downstream MemoryRouter can recall against effectively.
8+
9+
## Six content kinds
10+
11+
| Kind | Examples |
12+
|---|---|
13+
| `short-conversation` | 1-3 turn chats, brief Q&A |
14+
| `long-conversation` | extended chat sessions, support threads |
15+
| `long-article` | blog posts, paper sections, long emails |
16+
| `code` | source files, configs, schemas |
17+
| `structured-data` | CSV, JSON record lists, table dumps |
18+
| `multimodal` | content with images, video frames, audio |
19+
20+
## Six ingest strategies
21+
22+
| Strategy | What it writes | Cost |
23+
|---|---|---|
24+
| `raw-chunks` | turn/chunk traces with embeddings | $0.0001/ingest |
25+
| `summarized` | session/document summary prefixed to every chunk | $0.005/ingest |
26+
| `observational` | structured observation log replacing raw turns | $0.020/ingest |
27+
| `fact-graph` | extracted fact triples + entity-relation graph | $0.015/ingest |
28+
| `hybrid` | parallel raw + summarized + observational | $0.030/ingest |
29+
| `skip` | content discarded; nothing written | $0 |
30+
31+
## Four shipping presets
32+
33+
| Preset | Strategy | When to use |
34+
|---|---|---|
35+
| `raw-chunks` (default) | every kind → raw-chunks | high-volume / cost-sensitive workloads where retrieval does the work |
36+
| `summarized` | long-* and code → summarized; short stays raw | documents/conversations with global context that aids recall |
37+
| `observational` | long-conversation → observational; long-article → summarized | conversational workloads with multi-session synthesis questions |
38+
| `hybrid` | long-* → hybrid; short stays raw | cost-tolerant workloads with heterogeneous retrieval needs |
39+
40+
## Installation
41+
42+
Already part of `@framers/agentos`. Import from the subpath:
43+
44+
```ts
45+
import {
46+
LLMIngestClassifier,
47+
IngestRouter,
48+
FunctionIngestDispatcher,
49+
} from '@framers/agentos/ingest-router';
50+
```
51+
52+
## Usage
53+
54+
### Decide-only
55+
56+
```ts
57+
const router = new IngestRouter({
58+
classifier: new LLMIngestClassifier({ llm: openaiAdapter }),
59+
preset: 'summarized',
60+
});
61+
62+
const { classifier, routing } = await router.decide(content);
63+
console.log(classifier.kind); // 'long-conversation'
64+
console.log(routing.chosenStrategy); // 'observational'
65+
console.log(routing.estimatedCostUsd); // 0.020
66+
```
67+
68+
### Decide + dispatch
69+
70+
```ts
71+
type Outcome = { writtenTraces: number };
72+
73+
const router = new IngestRouter({
74+
classifier: new LLMIngestClassifier({ llm: openaiAdapter }),
75+
preset: 'observational',
76+
dispatcher: new FunctionIngestDispatcher<Outcome>({
77+
'raw-chunks': async (content) => ({ writtenTraces: await rawIngest(content) }),
78+
summarized: async (content) => ({ writtenTraces: await summarizedIngest(content) }),
79+
observational: async (content) => ({ writtenTraces: await omIngest(content) }),
80+
}),
81+
});
82+
83+
const { decision, outcome } = await router.decideAndDispatch(content);
84+
```
85+
86+
### Manual kind override
87+
88+
When the caller already knows the content kind (e.g., file extension determines code), skip the LLM classifier:
89+
90+
```ts
91+
const decision = await router.decide(content, {
92+
manualKind: 'code',
93+
});
94+
// classifier is not invoked; routing table consulted with 'code' directly.
95+
```
96+
97+
### Budget-aware
98+
99+
```ts
100+
const router = new IngestRouter({
101+
classifier,
102+
preset: 'observational',
103+
budget: {
104+
perIngestUsd: 0.005, // tight per-ingest ceiling
105+
mode: 'cheapest-fallback', // silently downgrade to a fitting strategy
106+
},
107+
});
108+
```
109+
110+
## API surface
111+
112+
- `IngestContentKind`, `IngestStrategyId`, `IngestRouterPreset`, `IngestRoutingTable`
113+
- `INGEST_CONTENT_KINDS` — the six-kind tuple
114+
- `RAW_CHUNKS_TABLE`, `SUMMARIZED_TABLE`, `OBSERVATIONAL_TABLE`, `HYBRID_TABLE`, `PRESET_INGEST_TABLES`
115+
- `IngestStrategyCostPoint`, `DEFAULT_INGEST_COSTS`
116+
- `selectIngestStrategy` (pure function)
117+
- `IngestRoutingDecision`, `IngestRouterConfig`, `IngestBudgetMode`
118+
- `IIngestClassifier`, `IIngestClassifierLLM`, `LLMIngestClassifier`
119+
- `INGEST_CLASSIFIER_SYSTEM_PROMPT`, `INGEST_CLASSIFIER_SYSTEM_PROMPT_FEWSHOT`
120+
- `IIngestDispatcher`, `FunctionIngestDispatcher`
121+
- `IngestRouter`, `IngestRouterOptions`, `IngestRouterDecideOptions`, `IngestRouterDispatchedResult`
122+
- Errors: `IngestRouterUnknownKindError`, `IngestRouterBudgetExceededError`, `UnsupportedIngestStrategyError`, `IngestRouterDispatcherMissingError`
123+
124+
## Related
125+
126+
- `@framers/agentos/memory-router` — recall-stage sibling
127+
- `@framers/agentos/read-router` — read-stage sibling
128+
- `@framers/agentos/multi-stage-guardrails` — composition primitive that wires the three stages together
129+
- `@framers/agentos/core/guardrails` + `agentos-ext-grounding-guard` — output-stage validation

0 commit comments

Comments
 (0)