Skip to content

Commit 740bb1e

Browse files
neo-opus-adatobiu
andauthored
feat(ai): wire dimension-consistency gatherer into the data-integrity runner evidence seam (#14225) (#14226)
* feat(ai): wire dimension-consistency gatherer into the data-integrity runner evidence seam (#14225) #14130 slice-2 (slice-1 = #14216, merged). Wires gatherDimensionConsistencyDiagnosis into Orchestrator.dataIntegrityEvidenceGatherer as a 2nd producer alongside coverage. - New dataIntegrityDimensionGatherer getter: runs the gatherer over the live Memory Core collections (StorageRouter.getMemoryCollection/getSummaryCollection) with AiConfig.vectorDimension (root-level leaf) + dataIntegrityServiceId. - dataIntegrityEvidenceGatherer folds the dimension diagnosis into diagnoses alongside coverage ([coverageDiagnosis, dimensionDiagnosis].filter(Boolean)), one shared observedAt. - evidenceFacts (vector-dimension-mismatch) compose with the assembler's existing per-collection fold, no glue (per @neo-opus-vega, seam owner). Degrade-not-throw preserved (down-Chroma → no false diagnosis; coverage = the offline baseline). actionClass:escalate drop out of scope (#14138). 79 Orchestrator specs + node --check clean; gatherer #14216-tested + assembler tested. @neo-opus-vega primary-reviews. Resolves #14225. * refactor(ai): extract live dimension-gatherer resolution into a factory (#14226) Addresses @neo-gpt's #14226 placement RC: the live Memory Core collection resolution + producer-config binding no longer live in a durable Orchestrator getter. Moved into createLiveDimensionConsistencyGatherer() in the gatherer service module; the orchestrator reads AiConfig.vectorDimension at its use-site (per the ADR-0019 SSOT discipline) and injects the resolved deps (storageRouter, expectedDimension, serviceId). The durable get dataIntegrityDimensionGatherer() is removed; dataIntegrityEvidenceGatherer calls the factory inline. auditFn threads through for test isolation (matching the sibling gatherer's seam). 6 specs pass (4 existing + 2 new factory tests covering collection-resolution + config-binding); node --check clean on both modules. --------- Co-authored-by: tobiu <tobiasuhlig78@gmail.com>
1 parent 82255a3 commit 740bb1e

3 files changed

Lines changed: 81 additions & 6 deletions

File tree

ai/daemons/orchestrator/Orchestrator.mjs

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ import {appendHealEvent, healEventsToRecentRuns, queryHealLedger, readHealLedger
4141
import {Memory_StorageRouter as StorageRouter, Memory_TextEmbeddingService as TextEmbeddingService} from '../../services.mjs';
4242
import {buildDataIntegrityCoverageDiagnosis} from './services/dataIntegrityCoverageDiagnosis.mjs';
4343
import {assembleDataIntegrityEvidence} from './services/dataIntegrityEvidenceAssembler.mjs';
44+
import {createLiveDimensionConsistencyGatherer} from './services/dimensionConsistencyGatherer.mjs';
4445
import DreamService from './services/DreamService.mjs';
4546
import SwarmHeartbeatService from './services/SwarmHeartbeatService.mjs';
4647
import GoldenPathSynthesizer from '../../services/graph/GoldenPathSynthesizer.mjs';
@@ -580,12 +581,19 @@ export class Orchestrator extends Base {
580581
* @returns {Function} `async () => Promise<Object[]>` — the assembled classifier-input rows.
581582
*/
582583
get dataIntegrityEvidenceGatherer() {
583-
const coverageGatherer = this.dataIntegrityCoverageGatherer,
584-
serviceId = this.dataIntegrityServiceId;
584+
const coverageGatherer = this.dataIntegrityCoverageGatherer,
585+
dimensionGatherer = createLiveDimensionConsistencyGatherer({
586+
storageRouter : StorageRouter,
587+
expectedDimension: AiConfig.vectorDimension,
588+
serviceId : this.dataIntegrityServiceId
589+
}),
590+
serviceId = this.dataIntegrityServiceId;
585591
return async () => {
586-
const coverageResult = await coverageGatherer(),
587-
coverageDiagnosis = buildDataIntegrityCoverageDiagnosis({coverageResult, observedAt: Date.now(), serviceId}),
588-
diagnoses = coverageDiagnosis ? [coverageDiagnosis] : [];
592+
const observedAt = Date.now(),
593+
coverageResult = await coverageGatherer(),
594+
coverageDiagnosis = buildDataIntegrityCoverageDiagnosis({coverageResult, observedAt, serviceId}),
595+
dimensionDiagnosis = await dimensionGatherer(observedAt),
596+
diagnoses = [coverageDiagnosis, dimensionDiagnosis].filter(Boolean);
589597
return assembleDataIntegrityEvidence({diagnoses, serviceId});
590598
};
591599
}

ai/daemons/orchestrator/services/dimensionConsistencyGatherer.mjs

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,33 @@ export async function gatherDimensionConsistencyDiagnosis({
4646

4747
return buildDimensionConsistencyDiagnosis({samples, observedAt, serviceId});
4848
}
49+
50+
/**
51+
* @summary Builds the live-Chroma dimension-consistency gatherer the data-integrity sweep schedules: it
52+
* resolves the Memory Core collections from the injected storage router and binds the producer config, then
53+
* returns a closure that runs {@link gatherDimensionConsistencyDiagnosis} over the resolved live collections.
54+
*
55+
* The live-collection resolution + config binding live here with the gatherer service (not in the
56+
* scheduling/orchestrator class): the orchestrator reads the AiConfig leaf at its use-site per the SSOT
57+
* discipline and injects the resolved `expectedDimension` + the `storageRouter`, keeping this module free of
58+
* config/Neo imports while owning the gather wiring. Injected `storageRouter` makes the factory testable in
59+
* isolation (no live Chroma).
60+
*
61+
* @param {Object} options
62+
* @param {Object} options.storageRouter The Memory Core storage router — supplies `ready()` +
63+
* `getMemoryCollection()` / `getSummaryCollection()`.
64+
* @param {Number} options.expectedDimension The configured embedding dimension (read at the orchestrator use-site).
65+
* @param {String} options.serviceId The Memory Core compose-service identifier.
66+
* @param {Function} [options.auditFn] The dimension-audit primitive — passed through to the gatherer; injectable for tests.
67+
* @returns {Function} `async (observedAt) => Promise<Object|null>` — a dimension-mismatch recovery-diagnosis or null.
68+
*/
69+
export function createLiveDimensionConsistencyGatherer({storageRouter, expectedDimension, serviceId, auditFn} = {}) {
70+
return async observedAt => {
71+
await storageRouter.ready();
72+
const collections = [
73+
{collection: await storageRouter.getMemoryCollection(), collectionName: 'neo-agent-memory'},
74+
{collection: await storageRouter.getSummaryCollection(), collectionName: 'neo-agent-sessions'}
75+
];
76+
return gatherDimensionConsistencyDiagnosis({collections, expectedDimension, serviceId, observedAt, auditFn});
77+
};
78+
}

test/playwright/unit/ai/daemons/orchestrator/services/dimensionConsistencyGatherer.spec.mjs

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import {test, expect} from '@playwright/test';
22
import Neo from '../../../../../../../src/Neo.mjs';
33
import * as core from '../../../../../../../src/core/_export.mjs';
4-
import {gatherDimensionConsistencyDiagnosis} from '../../../../../../../ai/daemons/orchestrator/services/dimensionConsistencyGatherer.mjs';
4+
import {createLiveDimensionConsistencyGatherer, gatherDimensionConsistencyDiagnosis} from '../../../../../../../ai/daemons/orchestrator/services/dimensionConsistencyGatherer.mjs';
55

66
// The live-Chroma audit+scheduling half of the dimension-consistency detect signal: it samples each
77
// collection via the (injected) dimension-audit primitive and feeds the samples to the pure producer.
@@ -39,3 +39,40 @@ test.describe('dimensionConsistencyGatherer', () => {
3939
expect(await gatherDimensionConsistencyDiagnosis({collections, expectedDimension: 1024, serviceId: 'svc', observedAt: 1000, auditFn})).not.toBeNull()
4040
})
4141
});
42+
43+
// The live-binding factory: owns the Memory Core collection resolution + config binding (moved out of the
44+
// Orchestrator getter per #14226's placement RC), so the orchestrator just reads the AiConfig leaf at its
45+
// use-site and injects the resolved deps. Injected storageRouter + auditFn keep it testable without live Chroma.
46+
test.describe('createLiveDimensionConsistencyGatherer', () => {
47+
const makeRouter = () => {
48+
const calls = {ready: 0};
49+
return {
50+
calls,
51+
ready : async () => { calls.ready++; },
52+
getMemoryCollection : async () => ({id: 'mem'}),
53+
getSummaryCollection: async () => ({id: 'sum'})
54+
};
55+
};
56+
57+
test('resolves the memory + summary collections from the router, binds config, and builds a diagnosis on a mismatch', async () => {
58+
const router = makeRouter(),
59+
audited = [],
60+
auditFn = async ({collectionName, expectedDimension}) => {
61+
audited.push({collectionName, expectedDimension});
62+
return {collection: collectionName, expectedDimension, mismatchedVectorCount: collectionName === 'neo-agent-memory' ? 4 : 0, sampledCount: 100};
63+
},
64+
gather = createLiveDimensionConsistencyGatherer({storageRouter: router, expectedDimension: 1024, serviceId: 'mc-server', auditFn}),
65+
diagnosis = await gather(1000);
66+
expect(router.calls.ready).toBe(1);
67+
expect(audited.map(a => a.collectionName)).toEqual(['neo-agent-memory', 'neo-agent-sessions']);
68+
expect(audited.every(a => a.expectedDimension === 1024)).toBe(true);
69+
expect(diagnosis).not.toBeNull()
70+
});
71+
72+
test('returns null when both live collections match the dimension', async () => {
73+
const router = makeRouter(),
74+
auditFn = async ({collectionName, expectedDimension}) => ({collection: collectionName, expectedDimension, mismatchedVectorCount: 0, sampledCount: 100}),
75+
gather = createLiveDimensionConsistencyGatherer({storageRouter: router, expectedDimension: 1024, serviceId: 'mc-server', auditFn});
76+
expect(await gather(1000)).toBeNull()
77+
})
78+
});

0 commit comments

Comments
 (0)