Skip to content

Commit 831aefa

Browse files
neo-opus-adatobiu
andauthored
fix(ai): dimension-gatherer degrade-not-throw on Chroma connection failure (#14130) (#14258)
createLiveDimensionConsistencyGatherer awaited storageRouter.ready() + getMemoryCollection()/getSummaryCollection() with no guard, so a Chroma connection failure threw. In Orchestrator.dataIntegrityEvidenceGatherer (coverage gathered first, then await dimensionGatherer()), that throw discarded the already-gathered coverage diagnosis and blanked the entire hourly data-integrity sweep — violating #14130 AC3 (connection failure -> empty samples, never a thrown gather or a suppressed coverage signal). Wrap the live-binding resolution in try-catch -> return null on failure (degrade-not-throw; null is filtered by the evidence-gather's .filter(Boolean), so coverage survives). The audit primitive already degraded on a probe failure; this extends the same contract to the connection boundary. 7 specs pass (+ a new AC3 test: ready()-throws and mid-resolution getCollection()-throws both resolve to null). node --check clean. Co-authored-by: tobiu <tobiasuhlig78@gmail.com>
1 parent 42e925d commit 831aefa

2 files changed

Lines changed: 35 additions & 6 deletions

File tree

ai/daemons/orchestrator/services/dimensionConsistencyGatherer.mjs

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -68,11 +68,21 @@ export async function gatherDimensionConsistencyDiagnosis({
6868
*/
6969
export function createLiveDimensionConsistencyGatherer({storageRouter, expectedDimension, serviceId, auditFn} = {}) {
7070
return async observedAt => {
71-
await storageRouter.ready();
72-
const collections = [
73-
{collection: await storageRouter.getMemoryCollection(), collectionName: 'neo-agent-memory'},
74-
{collection: await storageRouter.getSummaryCollection(), collectionName: 'neo-agent-sessions'}
75-
];
76-
return gatherDimensionConsistencyDiagnosis({collections, expectedDimension, serviceId, observedAt, auditFn});
71+
// Degrade-not-throw at the live-binding boundary, not only inside the audit primitive (#14130 AC3).
72+
// A Chroma *connection* failure (storageRouter.ready / getCollection) must skip the dimension signal
73+
// for this cycle and return null — never throw. The shared `dataIntegrityEvidenceGatherer` gathers
74+
// coverage first then awaits this; an unguarded throw here would discard the already-gathered coverage
75+
// diagnosis and blank the whole hourly sweep. A down-Chroma is surfaced separately (coverage gatherer
76+
// + container-health), so a silent dimension-skip is the safe degrade; null is filtered downstream.
77+
try {
78+
await storageRouter.ready();
79+
const collections = [
80+
{collection: await storageRouter.getMemoryCollection(), collectionName: 'neo-agent-memory'},
81+
{collection: await storageRouter.getSummaryCollection(), collectionName: 'neo-agent-sessions'}
82+
];
83+
return await gatherDimensionConsistencyDiagnosis({collections, expectedDimension, serviceId, observedAt, auditFn});
84+
} catch (error) {
85+
return null;
86+
}
7787
};
7888
}

test/playwright/unit/ai/daemons/orchestrator/services/dimensionConsistencyGatherer.spec.mjs

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,5 +74,24 @@ test.describe('createLiveDimensionConsistencyGatherer', () => {
7474
auditFn = async ({collectionName, expectedDimension}) => ({collection: collectionName, expectedDimension, mismatchedVectorCount: 0, sampledCount: 100}),
7575
gather = createLiveDimensionConsistencyGatherer({storageRouter: router, expectedDimension: 1024, serviceId: 'mc-server', auditFn});
7676
expect(await gather(1000)).toBeNull()
77+
});
78+
79+
test('degrades to null (never throws) on a Chroma connection failure — coverage is not suppressed (#14130 AC3)', async () => {
80+
// ready() throws (Chroma down): an unguarded throw here would discard the already-gathered coverage
81+
// diagnosis in the shared evidence-gather and blank the whole sweep. Must resolve to null, not reject.
82+
const downRouter = {
83+
ready : async () => { throw new Error('chroma connection refused'); },
84+
getMemoryCollection : async () => ({id: 'mem'}),
85+
getSummaryCollection: async () => ({id: 'sum'})
86+
};
87+
await expect(createLiveDimensionConsistencyGatherer({storageRouter: downRouter, expectedDimension: 1024, serviceId: 'mc-server'})(1000)).resolves.toBeNull();
88+
89+
// mid-resolution failure (getCollection throws after a successful ready()) — same degrade.
90+
const midFailRouter = {
91+
ready : async () => {},
92+
getMemoryCollection : async () => { throw new Error('chroma get failed'); },
93+
getSummaryCollection: async () => ({id: 'sum'})
94+
};
95+
await expect(createLiveDimensionConsistencyGatherer({storageRouter: midFailRouter, expectedDimension: 1024, serviceId: 'mc-server'})(1000)).resolves.toBeNull()
7796
})
7897
});

0 commit comments

Comments
 (0)