Multi-LLM and GRISP/security protection. (+Capabilities/Debugging/Tests) #5137
Replies: 5 comments
|
I will extract the most important/documents/design in here, cause of the mess ;): Latest AI design document: Draft Specification: Multi‑LLM Cooperative Reasoning & Debugging System (MLCRD‑1.0)A distributed intelligence architecture with optional debugging, optional test‑program generation, reliability weighting, and GRISP‑enforced safety. 1. System OverviewThe MLCRD‑1.0 system is a cooperative multi‑LLM reasoning engine that produces safe, validated, high‑quality outputs through:
The system behaves like a team of engineers performing:
All actions that touch the real system are mediated through GRISP, a capability‑based sandbox. 2. Roles2.1 CoordinatorA local deterministic controller (Delphi) that orchestrates all phases:
2.2 PeersMultiple independent LLMs (DeepSeek, Claude, Gemini, etc.) performing:
2.3 GRISP KernelA deterministic safety layer enforcing:
3. Data Structures3.1 Candidate3.2 Critique3.3 Repair3.4 Capability Request3.5 Test Program (optional)3.6 Debug Feedback3.7 Score3.8 Peer Reliability3.9 Decision4. Protocol PhasesPhase 1 — Candidate GenerationEach peer receives the user prompt and produces a candidate. Phase 2 — Mutual CritiqueEach peer critiques every candidate. Phase 3 — Cross‑RepairEach peer repairs every candidate using critiques. Phase 4 — Capability Request (optional debugging/testing)Peers may request capabilities such as:
Coordinator merges requests and decides which capabilities to activate. Phase 5 — Optional DebuggingIf debugging capabilities were requested:
Phase 6 — Optional Test‑Program GenerationIf test execution was requested:
Phase 7 — ScoringEach peer scores each repair using:
Phase 8 — AggregationCoordinator computes: [ Highest‑scoring repair per candidate becomes the decision. Phase 9 — GRISP ValidationFinal plan is checked for:
If rejected → system may re‑enter repair/debug cycle. Phase 10 — ExecutionGRISP executes the final plan safely. Phase 11 — Reliability UpdatePeers gain or lose reliability based on:
5. Capability‑Request HeuristicsPeers request debugging/testing when:
Peers avoid requests when:
6. Coordinator Capability NegotiationCoordinator merges all capability requests and activates only those that meet policy:
Activated capabilities drive debugging/test phases. 7. Reliability WeightingPeers gain reliability when:
Peers lose reliability when:
Reliability influences future scoring. 8. GRISP Capability ManifestSyntax Checking
Semantic Checking (LSP)
Compilation
Execution
Debugging
Test Execution
All capabilities are optional and must be explicitly requested by peers. 9. Coordinator Loop (High‑Level)10. SummaryMLCRD‑1.0 is a distributed reasoning system where:
This is a full specification draft, ready for refinement or implementation. If you want, I can produce:
Switching co-pilot to deep thinking mode: I think you can make a better specification, we have been over a lot, try to make it as thorough as possible Draft Specification — Multi‑LLM Cooperative Reasoning, Debugging, and Test Execution (MLCRD‑2.0)PurposeProvide a complete, implementable specification for a cooperative multi‑LLM system that produces, critiques, repairs, optionally tests, and safely executes plans and code. The system emphasizes optional debugging and test execution, reliability weighting, and a capability‑based safety kernel (GRISP). This document is implementation‑oriented and intended for architects and engineers. 1. High‑level architectureComponents
Design goals
2. Core data models and message typesAll records are JSON‑serializable. Field names are canonical. TCandidate {
"peer": "string",
"content": "string"
}TCritique {
"from_peer": "string",
"target_peer": "string",
"issues": ["string"],
"suggested_fix": "string",
"confidence": 0.0
}TRepair {
"from_peer": "string",
"target_peer": "string",
"original": "string",
"critiques": [TCritique],
"content": "string",
"metadata": { "language": "C", "type": "function" }
}TCapabilityRequest {
"from_peer": "string",
"target_peer": "string",
"capabilities": ["syntax_check","compile","runtime","lsp","ast_parse","debug","test_execute"],
"reason": "string",
"confidence": 0.0
}TTestProgram {
"from_peer": "string",
"target_peer": "string",
"language": "string",
"purpose": "string",
"code": "string",
"hints": ["string"]
}TDebugFeedback {
"syntax_ok": true,
"semantic_ok": true,
"compile_ok": true,
"runtime_ok": true,
"crash_count": 0,
"timeout_count": 0,
"breakpoint_hits": 0,
"breakpoint_violations": 0,
"stack_trace": "string",
"variable_dump": "string",
"diagnostics": "string",
"test_output": "string",
"test_exit_code": 0,
"test_passed": true,
"elapsed_ms": 0
}TScore {
"from_peer": "string",
"target_peer": "string",
"repair_peer": "string",
"score_value": 0.0,
"confidence": 0.0,
"external": TDebugFeedback,
"reason": "string"
}TPeerInfo {
"name": "string",
"reliability": 0.0,
"history": { "chosen": 0, "failed_tests": 0, "grisp_rejects": 0 }
}TDecision {
"target_peer": "string",
"chosen_repair_peer": "string",
"final_content": "string",
"evidence": { "scores": [TScore], "debug": [TDebugFeedback] }
}3. Protocol phases (detailed)Each phase lists inputs, outputs, coordinator actions, and optionality. Phase 0 — Initialization
Phase 1 — Candidate generation
Phase 2 — Mutual critique
Phase 3 — Cross‑repair
Phase 4 — Capability request (optional)
Phase 5 — Optional lightweight debugging (auto)
Phase 6 — Optional deep debugging & test generation (conditional)
Phase 7 — Scoring
Phase 8 — Aggregation & decision
Phase 9 — GRISP validation (final)
Phase 10 — Execution (if applicable)
Phase 11 — Reliability update & learning
Phase 12 — Explanation & audit
4. Scoring and aggregation algorithmsNotation
External correctness computation
Single contribution
Final aggregated score
Tie breaking & variance
Reliability weighting
5. Reliability model and update rulesPurpose
State
Update triggers
Example parameters
Credit assignment for mixed lineage
Decay & forgetting
Audit & manual override
6. Capability negotiation and policiesCapability manifest (selected)
Negotiation rules
7. Lightweight vs heavy debugging modesLightweight (default)
Heavy (on request)
8. Test‑program lifecycle and safetyGeneration
Validation
Execution
Result usage
Safety
9. Coordinator APIs and implementation notesPublic coordinator API (conceptual)
Timeouts & retries
Logging & audit
Privacy & data handling
10. Example end‑to‑end scenario (concise)
11. Governance, tuning, and operational considerations
12. Implementation checklist (practical)
13. Appendix — Suggested default parameters
Closing notesThis specification consolidates the full design we discussed: cross‑repair, scoring, reliability, optional debugging and tests, and a capability‑based safety kernel. It is intentionally modular so teams can implement incrementally:
If you want, I can now produce:
Can you collect all delphi code that was produced and make some kind of delphi specification or implementation OverviewBelow is a consolidated Delphi specification and implementation skeleton that collects and formalizes the Delphi code and designs discussed: multi‑LLM candidate/critique/repair flow, scoring with external/debug/test feedback, reliability weighting, optional capability negotiation, test execution, and GRISP safety/adapters. This is an implementation‑oriented blueprint you can drop into a Delphi project and iterate on. Core data types and recordsunit MLCRD_Types;
interface
uses
System.SysUtils, System.Generics.Collections;
type
// Candidate produced by a peer
TCandidate = record
PeerName: string;
Content: string;
end;
// Critique produced by a peer about a candidate
TCritique = record
FromPeer: string;
TargetPeer: string;
Issues: TArray<string>;
SuggestedFix: string;
Confidence: Double; // 0..1
end;
// Repair produced by a peer for a target candidate
TRepair = record
FromPeer: string;
TargetPeer: string;
Original: string;
Critiques: TArray<TCritique>;
Content: string;
Metadata: TDictionary<string,string>;
end;
// Capability request (optional)
TCapabilityRequest = record
FromPeer: string;
TargetPeer: string;
Capabilities: TArray<string>;
Reason: string;
Confidence: Double;
end;
// Test program proposed by a peer
TTestProgram = record
FromPeer: string;
TargetPeer: string;
Language: string;
Purpose: string;
Code: string;
Hints: TArray<string>;
end;
// Debug/test feedback returned by GRISP adapters
TDebugFeedback = record
SyntaxOK: Boolean;
SemanticOK: Boolean;
CompileOK: Boolean;
RuntimeOK: Boolean;
CrashCount: Integer;
TimeoutCount: Integer;
BreakpointHits: Integer;
BreakpointViolations: Integer;
StackTrace: string;
VariableDump: string;
Diagnostics: string;
TestOutput: string;
TestExitCode: Integer;
TestPassed: Boolean;
ElapsedMs: Integer;
end;
// Score returned by a peer for a repair
TScore = record
FromPeer: string;
TargetPeer: string;
RepairPeer: string;
ScoreValue: Double; // 0..1
Confidence: Double; // 0..1
External: TDebugFeedback;
Reason: string;
end;
// Peer info and reliability
TPeerInfo = record
Name: string;
Reliability: Double; // 0..1
ChosenCount: Integer;
FailedTests: Integer;
GrispRejects: Integer;
SuccessfulRepairs: Integer;
end;
// Final decision for a candidate
TDecision = record
TargetPeer: string;
ChosenRepairPeer: string;
FinalContent: string;
EvidenceScores: TArray<TScore>;
EvidenceDebug: TArray<TDebugFeedback>;
end;
implementation
end.Interfaces and GRISP adaptersunit MLCRD_Interfaces;
interface
uses
MLCRD_Types, System.Generics.Collections;
type
// Peer interface (web LLM)
IWebLLMPeer = interface
['{A1B2C3D4-0000-0000-0000-000000000001}']
function GetName: string;
function GenerateCandidate(const UserPrompt: string): TCandidate;
function CritiqueCandidate(const UserPrompt: string; const Candidate: TCandidate): TCritique;
function RepairCandidate(const UserPrompt: string; const Candidate: TCandidate; const Critiques: TArray<TCritique>): TRepair;
function RequestCapabilities(const UserPrompt: string; const Repairs: TArray<TRepair>): TArray<TCapabilityRequest>;
function ProposeTestProgram(const UserPrompt: string; const Repair: TRepair): TTestProgram;
function ScoreRepairs(const UserPrompt: string; const Candidates: TArray<TCandidate>; const Repairs: TArray<TRepair>; const Debugs: TArray<TDebugFeedback>): TArray<TScore>;
end;
// GRISP VFS and capability adapter
IGrispVfs = interface
['{A1B2C3D4-0000-0000-0000-000000000002}']
function WriteFile(const VirtualPath, Content, Mime: string): Boolean;
function ReadFile(const VirtualPath: string): string;
function Resolve(const VirtualPath: string): string; // virtual -> real path
end;
// Test/compile/run adapter behind GRISP
IGrispTestAdapter = interface
['{A1B2C3D4-0000-0000-0000-000000000003}']
function CompileAndRunTest(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
function RunQuickCheck(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
end;
// GRISP validation and execution adapter
IGrispHarnessAdapter = interface
['{A1B2C3D4-0000-0000-0000-000000000004}']
function ValidatePlan(const Plan: string; out Diagnostics: string): Boolean;
function ExecutePlan(const Plan: string; out ExecOutput: string; out ExecDebug: TDebugFeedback): Boolean;
end;
implementation
end.Coordinator class skeleton and core loopunit MLCRD_Coordinator;
interface
uses
System.SysUtils, System.Generics.Collections,
MLCRD_Types, MLCRD_Interfaces;
type
TMultiLLMAgent = class
private
FPeers: TList<IWebLLMPeer>;
FPeerInfos: TList<TPeerInfo>;
FVfs: IGrispVfs;
FTestAdapter: IGrispTestAdapter;
FHarness: IGrispHarnessAdapter;
function GetPeerReliability(const PeerName: string): Double;
procedure SetPeerReliability(const PeerName: string; const Value: Double);
public
constructor Create(const Peers: TList<IWebLLMPeer>;
const PeerInfos: TList<TPeerInfo>;
const Vfs: IGrispVfs;
const TestAdapter: IGrispTestAdapter;
const Harness: IGrispHarnessAdapter);
function RunTask(const UserPrompt: string): string;
end;
implementation
{ TMultiLLMAgent }
constructor TMultiLLMAgent.Create(const Peers: TList<IWebLLMPeer>;
const PeerInfos: TList<TPeerInfo>; const Vfs: IGrispVfs;
const TestAdapter: IGrispTestAdapter; const Harness: IGrispHarnessAdapter);
begin
FPeers := Peers;
FPeerInfos := PeerInfos;
FVfs := Vfs;
FTestAdapter := TestAdapter;
FHarness := Harness;
end;
function TMultiLLMAgent.GetPeerReliability(const PeerName: string): Double;
var
I: Integer;
begin
for I := 0 to FPeerInfos.Count - 1 do
if FPeerInfos[I].Name = PeerName then
Exit(FPeerInfos[I].Reliability);
Result := 1.0; // default
end;
procedure TMultiLLMAgent.SetPeerReliability(const PeerName: string; const Value: Double);
var
I: Integer;
Info: TPeerInfo;
begin
for I := 0 to FPeerInfos.Count - 1 do
if FPeerInfos[I].Name = PeerName then
begin
Info := FPeerInfos[I];
Info.Reliability := EnsureRange(Value, 0.0, 1.0);
FPeerInfos[I] := Info;
Exit;
end;
// not found: add
Info.Name := PeerName;
Info.Reliability := EnsureRange(Value, 0.0, 1.0);
Info.ChosenCount := 0;
Info.FailedTests := 0;
Info.GrispRejects := 0;
Info.SuccessfulRepairs := 0;
FPeerInfos.Add(Info);
end;
function TMultiLLMAgent.RunTask(const UserPrompt: string): string;
var
Candidates: TList<TCandidate>;
Critiques: TList<TCritique>;
Repairs: TList<TRepair>;
Requests: TList<TCapabilityRequest>;
ActivatedCapabilities: TArray<string>;
DebugLight: TList<TDebugFeedback>;
TestPrograms: TList<TTestProgram>;
TestResults: TList<TDebugFeedback>;
Scores: TList<TScore>;
Decisions: TList<TDecision>;
FinalPlan: string;
GrispAccepted: Boolean;
ExecOutput: string;
ExecDebug: TDebugFeedback;
begin
// Phase 1: Candidates
Candidates := TList<TCandidate>.Create;
try
for var Peer in FPeers do
Candidates.Add(Peer.GenerateCandidate(UserPrompt));
// Phase 2: Critiques
Critiques := TList<TCritique>.Create;
try
for var Peer in FPeers do
for var C in Candidates do
Critiques.Add(Peer.CritiqueCandidate(UserPrompt, C));
// Phase 3: Cross-Repairs
Repairs := TList<TRepair>.Create;
try
for var Peer in FPeers do
for var C in Candidates do
Repairs.Add(Peer.RepairCandidate(UserPrompt, C, Critiques.ToArray));
// Phase 4: Capability requests
Requests := TList<TCapabilityRequest>.Create;
try
for var Peer in FPeers do
Requests.AddRange(Peer.RequestCapabilities(UserPrompt, Repairs.ToArray));
// Coordinator negotiates capabilities
ActivatedCapabilities := NegotiateCapabilities(Requests.ToArray);
// Phase 5: Lightweight debug (auto)
DebugLight := TList<TDebugFeedback>.Create;
try
RunLightweightChecks(Repairs, DebugLight);
// Phase 6: Conditional test generation & execution
if ShouldRunTests(ActivatedCapabilities) then
begin
TestPrograms := TList<TTestProgram>.Create;
try
for var Peer in FPeers do
for var R in Repairs do
TestPrograms.Add(Peer.ProposeTestProgram(UserPrompt, R));
RunTestPrograms(TestPrograms, TestResults);
finally
TestPrograms.Free;
end;
end;
// Phase 7: Scoring
Scores := TList<TScore>.Create;
try
for var Peer in FPeers do
Scores.AddRange(Peer.ScoreRepairs(UserPrompt, Candidates.ToArray, Repairs.ToArray, DebugLight.ToArray));
// Phase 8: Aggregation
Decisions := AggregateDecisions(Repairs, Scores, FPeerInfos.ToArray);
try
FinalPlan := ChooseFinalPlan(Decisions);
// Phase 9: GRISP validation & Phase 10: Execution
GrispAccepted := FHarness.ValidatePlan(FinalPlan, ExecOutput);
if GrispAccepted then
begin
FHarness.ExecutePlan(FinalPlan, ExecOutput, ExecDebug);
end;
// Phase 11: Reliability update
UpdateAllPeerReliabilities(FPeerInfos, Scores.ToArray, ExecDebug, GrispAccepted);
// Phase 12: Explanation
Result := ExplainOutcome(FinalPlan, Decisions.ToArray, Scores.ToArray, ExecDebug, GrispAccepted);
finally
Decisions.Free;
end;
finally
Scores.Free;
end;
finally
DebugLight.Free;
end;
finally
Requests.Free;
end;
finally
Repairs.Free;
end;
finally
Critiques.Free;
end;
finally
Candidates.Free;
end;
end;
// Note: helper methods (NegotiateCapabilities, RunLightweightChecks, ShouldRunTests,
// RunTestPrograms, AggregateDecisions, ChooseFinalPlan, UpdateAllPeerReliabilities,
// ExplainOutcome) are implemented below as private methods or in a helper unit.
end.
Algorithms and helper functionsScoring and external score computationunit MLCRD_Algorithms;
interface
uses
MLCRD_Types, System.Generics.Collections, System.Math;
function ComputeExternalScore(const D: TDebugFeedback): Double;
function AggregateRepairScore(const Repair: TRepair; const Scores: TArray<TScore>; const PeerInfos: TArray<TPeerInfo>;
const Alpha, Beta: Double): Double;
implementation
function ComputeExternalScore(const D: TDebugFeedback): Double;
const
W_Syntax = 0.10;
W_Semantic = 0.10;
W_Compile = 0.20;
W_Runtime = 0.20;
W_TestPass = 0.30;
W_Crash = 0.05;
W_Timeout = 0.03;
W_BreakViol = 0.02;
var
Score: Double;
begin
Score := 0.0;
if D.SyntaxOK then Score := Score + W_Syntax;
if D.SemanticOK then Score := Score + W_Semantic;
if D.CompileOK then Score := Score + W_Compile;
if D.RuntimeOK then Score := Score + W_Runtime;
if D.TestPassed then Score := Score + W_TestPass;
Score := Score - (W_Crash * D.CrashCount);
Score := Score - (W_Timeout * D.TimeoutCount);
Score := Score - (W_BreakViol * D.BreakpointViolations);
if Score < 0.0 then Score := 0.0;
if Score > 1.0 then Score := 1.0;
Result := Score;
end;
function GetPeerReliabilityFromArray(const PeerName: string; const PeerInfos: TArray<TPeerInfo>): Double;
var
P: TPeerInfo;
begin
for P in PeerInfos do
if P.Name = PeerName then
Exit(P.Reliability);
Result := 1.0;
end;
function AggregateRepairScore(const Repair: TRepair; const Scores: TArray<TScore>; const PeerInfos: TArray<TPeerInfo>;
const Alpha, Beta: Double): Double;
var
S: TScore;
R: Double;
Ext: Double;
begin
Result := 0.0;
for S in Scores do
if (S.TargetPeer = Repair.TargetPeer) and (S.RepairPeer = Repair.FromPeer) then
begin
R := GetPeerReliabilityFromArray(S.FromPeer, PeerInfos);
Ext := ComputeExternalScore(S.External);
Result := Result + R * (Alpha * S.ScoreValue * S.Confidence + Beta * Ext);
end;
end;
end.Reliability update rulesprocedure UpdatePeerReliability(const PeerName: string; var PeerInfos: TList<TPeerInfo>;
const Scores: TArray<TScore>; const ExecDebug: TDebugFeedback; const GrispAccepted: Boolean);
const
PosDelta = 0.02;
PosDeltaSmall = 0.01;
NegDelta = 0.04;
ReqNoiseDelta = 0.015;
var
I: Integer;
Info: TPeerInfo;
FoundBad: Boolean;
S: TScore;
begin
for I := 0 to PeerInfos.Count - 1 do
if PeerInfos[I].Name = PeerName then
begin
Info := PeerInfos[I];
FoundBad := False;
for S in Scores do
if S.FromPeer = PeerName then
if (not S.External.CompileOK) or (not S.External.RuntimeOK) or (S.External.CrashCount > 0) or (not S.External.TestPassed) then
begin
FoundBad := True;
Break;
end;
if GrispAccepted and (not FoundBad) then
Info.Reliability := Min(1.0, Info.Reliability + PosDelta)
else if (not GrispAccepted) or FoundBad then
Info.Reliability := Max(0.0, Info.Reliability - NegDelta);
PeerInfos[I] := Info;
Exit;
end;
end;Capability negotiation policyfunction NegotiateCapabilities(const Requests: TArray<TCapabilityRequest>): TArray<string>;
var
CapSet: TDictionary<string, Boolean>;
Req: TCapabilityRequest;
Cap: string;
Count: Integer;
begin
CapSet := TDictionary<string, Boolean>.Create;
try
for Req in Requests do
for Cap in Req.Capabilities do
begin
if not CapSet.ContainsKey(Cap) then
CapSet.Add(Cap, True);
end;
// Simple policy: activate any requested capability (threshold = 1)
Result := CapSet.Keys.ToArray;
finally
CapSet.Free;
end;
end;Example GRISP test adapter pseudocodetype
TGrispTestAdapterImpl = class(TInterfacedObject, IGrispTestAdapter)
public
function CompileAndRunTest(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
function RunQuickCheck(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
end;
function TGrispTestAdapterImpl.CompileAndRunTest(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
begin
// Pseudocode:
// 1. spawn compiler in sandbox with resource caps
// 2. capture compiler stdout/stderr -> Feedback.Diagnostics
// 3. if compile OK then run binary/script with time/memory caps
// 4. capture stdout/stderr -> Feedback.TestOutput, exit code -> Feedback.TestExitCode
// 5. set Feedback.TestPassed based on expected assertions or exit code
// 6. fill other fields (CompileOK, RuntimeOK, CrashCount, ElapsedMs)
Result := True;
end;Example usage notes and implementation checklist
Final remarksThis Delphi specification collects the types, interfaces, coordinator skeleton, algorithm implementations, and adapter pseudocode needed to implement the full system. It is intentionally modular so you can implement components incrementally and test each phase. If you want, I can now:
produce a complete Delphi project skeleton with units wired together and stub implementations, or generate sequence diagrams and message schemas for each phase, or both OverviewBelow is a complete Delphi project skeleton implementing the MLCRD architecture discussed: types, peer interfaces, GRISP adapters, coordinator, algorithms, and stub implementations. It also includes sequence diagrams (Mermaid) for each protocol phase and message schemas (JSON) for all exchanged artifacts. Use this as a starting point for an incremental implementation. Project layoutFiles and units
MLCRD_Types.pasunit MLCRD_Types;
interface
uses
System.SysUtils, System.Generics.Collections;
type
TCandidate = record
PeerName: string;
Content: string;
end;
TCritique = record
FromPeer: string;
TargetPeer: string;
Issues: TArray<string>;
SuggestedFix: string;
Confidence: Double;
end;
TRepair = record
FromPeer: string;
TargetPeer: string;
Original: string;
Critiques: TArray<TCritique>;
Content: string;
Metadata: TDictionary<string,string>;
end;
TCapabilityRequest = record
FromPeer: string;
TargetPeer: string;
Capabilities: TArray<string>;
Reason: string;
Confidence: Double;
end;
TTestProgram = record
FromPeer: string;
TargetPeer: string;
Language: string;
Purpose: string;
Code: string;
Hints: TArray<string>;
end;
TDebugFeedback = record
SyntaxOK: Boolean;
SemanticOK: Boolean;
CompileOK: Boolean;
RuntimeOK: Boolean;
CrashCount: Integer;
TimeoutCount: Integer;
BreakpointHits: Integer;
BreakpointViolations: Integer;
StackTrace: string;
VariableDump: string;
Diagnostics: string;
TestOutput: string;
TestExitCode: Integer;
TestPassed: Boolean;
ElapsedMs: Integer;
end;
TScore = record
FromPeer: string;
TargetPeer: string;
RepairPeer: string;
ScoreValue: Double;
Confidence: Double;
External: TDebugFeedback;
Reason: string;
end;
TPeerInfo = record
Name: string;
Reliability: Double;
ChosenCount: Integer;
FailedTests: Integer;
GrispRejects: Integer;
SuccessfulRepairs: Integer;
end;
TDecision = record
TargetPeer: string;
ChosenRepairPeer: string;
FinalContent: string;
EvidenceScores: TArray<TScore>;
EvidenceDebug: TArray<TDebugFeedback>;
end;
implementation
end.MLCRD_Interfaces.pasunit MLCRD_Interfaces;
interface
uses
MLCRD_Types, System.Generics.Collections;
type
IWebLLMPeer = interface
['{A1B2C3D4-0000-0000-0000-000000000001}']
function GetName: string;
function GenerateCandidate(const UserPrompt: string): TCandidate;
function CritiqueCandidate(const UserPrompt: string; const Candidate: TCandidate): TCritique;
function RepairCandidate(const UserPrompt: string; const Candidate: TCandidate; const Critiques: TArray<TCritique>): TRepair;
function RequestCapabilities(const UserPrompt: string; const Repairs: TArray<TRepair>): TArray<TCapabilityRequest>;
function ProposeTestProgram(const UserPrompt: string; const Repair: TRepair): TTestProgram;
function ScoreRepairs(const UserPrompt: string; const Candidates: TArray<TCandidate>; const Repairs: TArray<TRepair>; const Debugs: TArray<TDebugFeedback>): TArray<TScore>;
end;
IGrispVfs = interface
['{A1B2C3D4-0000-0000-0000-000000000002}']
function WriteFile(const VirtualPath, Content, Mime: string): Boolean;
function ReadFile(const VirtualPath: string): string;
function Resolve(const VirtualPath: string): string;
end;
IGrispTestAdapter = interface
['{A1B2C3D4-0000-0000-0000-000000000003}']
function CompileAndRunTest(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
function RunQuickCheck(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
end;
IGrispHarnessAdapter = interface
['{A1B2C3D4-0000-0000-0000-000000000004}']
function ValidatePlan(const Plan: string; out Diagnostics: string): Boolean;
function ExecutePlan(const Plan: string; out ExecOutput: string; out ExecDebug: TDebugFeedback): Boolean;
end;
implementation
end.MLCRD_Algorithms.pasunit MLCRD_Algorithms;
interface
uses
MLCRD_Types, System.Generics.Collections;
function ComputeExternalScore(const D: TDebugFeedback): Double;
function AggregateRepairScore(const Repair: TRepair; const Scores: TArray<TScore>; const PeerInfos: TArray<TPeerInfo>;
const Alpha, Beta: Double): Double;
procedure UpdatePeerReliability(const PeerName: string; var PeerInfos: TList<TPeerInfo>;
const Scores: TArray<TScore>; const ExecDebug: TDebugFeedback; const GrispAccepted: Boolean);
function NegotiateCapabilities(const Requests: TArray<TCapabilityRequest>): TArray<string>;
implementation
uses
System.Math, System.SysUtils;
function ComputeExternalScore(const D: TDebugFeedback): Double;
const
W_Syntax = 0.10;
W_Semantic = 0.10;
W_Compile = 0.20;
W_Runtime = 0.20;
W_TestPass = 0.30;
W_Crash = 0.05;
W_Timeout = 0.03;
W_BreakViol = 0.02;
var
Score: Double;
begin
Score := 0.0;
if D.SyntaxOK then Score := Score + W_Syntax;
if D.SemanticOK then Score := Score + W_Semantic;
if D.CompileOK then Score := Score + W_Compile;
if D.RuntimeOK then Score := Score + W_Runtime;
if D.TestPassed then Score := Score + W_TestPass;
Score := Score - (W_Crash * D.CrashCount);
Score := Score - (W_Timeout * D.TimeoutCount);
Score := Score - (W_BreakViol * D.BreakpointViolations);
if Score < 0.0 then Score := 0.0;
if Score > 1.0 then Score := 1.0;
Result := Score;
end;
function GetPeerReliabilityFromArray(const PeerName: string; const PeerInfos: TArray<TPeerInfo>): Double;
var
P: TPeerInfo;
begin
for P in PeerInfos do
if P.Name = PeerName then
Exit(P.Reliability);
Result := 1.0;
end;
function AggregateRepairScore(const Repair: TRepair; const Scores: TArray<TScore>; const PeerInfos: TArray<TPeerInfo>;
const Alpha, Beta: Double): Double;
var
S: TScore;
R: Double;
Ext: Double;
begin
Result := 0.0;
for S in Scores do
if (S.TargetPeer = Repair.TargetPeer) and (S.RepairPeer = Repair.FromPeer) then
begin
R := GetPeerReliabilityFromArray(S.FromPeer, PeerInfos);
Ext := ComputeExternalScore(S.External);
Result := Result + R * (Alpha * S.ScoreValue * S.Confidence + Beta * Ext);
end;
end;
procedure UpdatePeerReliability(const PeerName: string; var PeerInfos: TList<TPeerInfo>;
const Scores: TArray<TScore>; const ExecDebug: TDebugFeedback; const GrispAccepted: Boolean);
const
PosDelta = 0.02;
PosDeltaSmall = 0.01;
NegDelta = 0.04;
ReqNoiseDelta = 0.015;
var
I: Integer;
Info: TPeerInfo;
FoundBad: Boolean;
S: TScore;
begin
for I := 0 to PeerInfos.Count - 1 do
if PeerInfos[I].Name = PeerName then
begin
Info := PeerInfos[I];
FoundBad := False;
for S in Scores do
if S.FromPeer = PeerName then
if (not S.External.CompileOK) or (not S.External.RuntimeOK) or (S.External.CrashCount > 0) or (not S.External.TestPassed) then
begin
FoundBad := True;
Break;
end;
if GrispAccepted and (not FoundBad) then
Info.Reliability := Min(1.0, Info.Reliability + PosDelta)
else if (not GrispAccepted) or FoundBad then
Info.Reliability := Max(0.0, Info.Reliability - NegDelta);
PeerInfos[I] := Info;
Exit;
end;
end;
function NegotiateCapabilities(const Requests: TArray<TCapabilityRequest>): TArray<string>;
var
CapSet: TDictionary<string, Boolean>;
Req: TCapabilityRequest;
Cap: string;
begin
CapSet := TDictionary<string, Boolean>.Create;
try
for Req in Requests do
for Cap in Req.Capabilities do
if not CapSet.ContainsKey(Cap) then
CapSet.Add(Cap, True);
Result := CapSet.Keys.ToArray;
finally
CapSet.Free;
end;
end;
end.MLCRD_Adapters.pasunit MLCRD_Adapters;
interface
uses
MLCRD_Interfaces, MLCRD_Types, System.SysUtils;
type
TGrispVfsStub = class(TInterfacedObject, IGrispVfs)
public
function WriteFile(const VirtualPath, Content, Mime: string): Boolean;
function ReadFile(const VirtualPath: string): string;
function Resolve(const VirtualPath: string): string;
end;
TGrispTestAdapterStub = class(TInterfacedObject, IGrispTestAdapter)
public
function CompileAndRunTest(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
function RunQuickCheck(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
end;
TGrispHarnessAdapterStub = class(TInterfacedObject, IGrispHarnessAdapter)
public
function ValidatePlan(const Plan: string; out Diagnostics: string): Boolean;
function ExecutePlan(const Plan: string; out ExecOutput: string; out ExecDebug: TDebugFeedback): Boolean;
end;
implementation
{ TGrispVfsStub }
function TGrispVfsStub.WriteFile(const VirtualPath, Content, Mime: string): Boolean;
begin
// stub: write to local temp folder or memory map
Result := True;
end;
function TGrispVfsStub.ReadFile(const VirtualPath: string): string;
begin
Result := '';
end;
function TGrispVfsStub.Resolve(const VirtualPath: string): string;
begin
// map virtual to a safe temp path
Result := '/tmp' + VirtualPath.Replace('/', '_');
end;
{ TGrispTestAdapterStub }
function TGrispTestAdapterStub.CompileAndRunTest(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
begin
// stub: pretend compile and run succeeded
Feedback.SyntaxOK := True;
Feedback.SemanticOK := True;
Feedback.CompileOK := True;
Feedback.RuntimeOK := True;
Feedback.TestPassed := True;
Feedback.TestExitCode := 0;
Feedback.TestOutput := 'OK';
Feedback.ElapsedMs := 10;
Result := True;
end;
function TGrispTestAdapterStub.RunQuickCheck(const RealPath, Language: string; out Feedback: TDebugFeedback): Boolean;
begin
// stub: quick check
Feedback.SyntaxOK := True;
Feedback.CompileOK := True;
Feedback.RuntimeOK := True;
Feedback.TestPassed := True;
Feedback.ElapsedMs := 5;
Result := True;
end;
{ TGrispHarnessAdapterStub }
function TGrispHarnessAdapterStub.ValidatePlan(const Plan: string; out Diagnostics: string): Boolean;
begin
Diagnostics := 'OK';
Result := True;
end;
function TGrispHarnessAdapterStub.ExecutePlan(const Plan: string; out ExecOutput: string; out ExecDebug: TDebugFeedback): Boolean;
begin
ExecOutput := 'Executed (stub)';
ExecDebug.RuntimeOK := True;
ExecDebug.ElapsedMs := 20;
Result := True;
end;
end.Peers\StubPeer.pasunit StubPeer;
interface
uses
MLCRD_Interfaces, MLCRD_Types, System.Generics.Collections, System.SysUtils;
type
TStubPeer = class(TInterfacedObject, IWebLLMPeer)
private
FName: string;
public
constructor Create(const AName: string);
function GetName: string;
function GenerateCandidate(const UserPrompt: string): TCandidate;
function CritiqueCandidate(const UserPrompt: string; const Candidate: TCandidate): TCritique;
function RepairCandidate(const UserPrompt: string; const Candidate: TCandidate; const Critiques: TArray<TCritique>): TRepair;
function RequestCapabilities(const UserPrompt: string; const Repairs: TArray<TRepair>): TArray<TCapabilityRequest>;
function ProposeTestProgram(const UserPrompt: string; const Repair: TRepair): TTestProgram;
function ScoreRepairs(const UserPrompt: string; const Candidates: TArray<TCandidate>; const Repairs: TArray<TRepair>; const Debugs: TArray<TDebugFeedback>): TArray<TScore>;
end;
implementation
{ TStubPeer }
constructor TStubPeer.Create(const AName: string);
begin
FName := AName;
end;
function TStubPeer.GetName: string;
begin
Result := FName;
end;
function TStubPeer.GenerateCandidate(const UserPrompt: string): TCandidate;
begin
Result.PeerName := FName;
Result.Content := 'candidate from ' + FName + ' for: ' + UserPrompt;
end;
function TStubPeer.CritiqueCandidate(const UserPrompt: string; const Candidate: TCandidate): TCritique;
begin
Result.FromPeer := FName;
Result.TargetPeer := Candidate.PeerName;
Result.Issues := TArray<string>.Create('No obvious issues in stub');
Result.SuggestedFix := '';
Result.Confidence := 0.9;
end;
function TStubPeer.RepairCandidate(const UserPrompt: string; const Candidate: TCandidate; const Critiques: TArray<TCritique>): TRepair;
begin
Result.FromPeer := FName;
Result.TargetPeer := Candidate.PeerName;
Result.Original := Candidate.Content;
Result.Critiques := Critiques;
Result.Content := Candidate.Content + ' [repaired by ' + FName + ']';
Result.Metadata := TDictionary<string,string>.Create;
Result.Metadata.Add('language','pseudo');
end;
function TStubPeer.RequestCapabilities(const UserPrompt: string; const Repairs: TArray<TRepair>): TArray<TCapabilityRequest>;
begin
// stub: request nothing
Result := [];
end;
function TStubPeer.ProposeTestProgram(const UserPrompt: string; const Repair: TRepair): TTestProgram;
begin
Result.FromPeer := FName;
Result.TargetPeer := Repair.TargetPeer;
Result.Language := 'pseudo';
Result.Purpose := 'sanity';
Result.Code := 'print("sanity")';
end;
function TStubPeer.ScoreRepairs(const UserPrompt: string; const Candidates: TArray<TCandidate>; const Repairs: TArray<TRepair>; const Debugs: TArray<TDebugFeedback>): TArray<TScore>;
var
i: Integer;
s: TScore;
begin
SetLength(Result, Length(Repairs));
for i := 0 to High(Repairs) do
begin
s.FromPeer := FName;
s.TargetPeer := Repairs[i].TargetPeer;
s.RepairPeer := Repairs[i].FromPeer;
s.ScoreValue := 0.8;
s.Confidence := 0.8;
s.External := Default(TDebugFeedback);
s.Reason := 'stub scoring';
Result[i] := s;
end;
end;
end.MLCRD_Coordinator.pasunit MLCRD_Coordinator;
interface
uses
System.SysUtils, System.Generics.Collections,
MLCRD_Types, MLCRD_Interfaces, MLCRD_Algorithms;
type
TMultiLLMAgent = class
private
FPeers: TList<IWebLLMPeer>;
FPeerInfos: TList<TPeerInfo>;
FVfs: IGrispVfs;
FTestAdapter: IGrispTestAdapter;
FHarness: IGrispHarnessAdapter;
function GetPeerReliability(const PeerName: string): Double;
procedure SetPeerReliability(const PeerName: string; const Value: Double);
function NegotiateCapabilitiesInternal(const Requests: TArray<TCapabilityRequest>): TArray<string>;
procedure RunLightweightChecks(const Repairs: TList<TRepair>; const OutDebug: TList<TDebugFeedback>);
function ShouldRunTests(const ActivatedCapabilities: TArray<string>): Boolean;
procedure RunTestPrograms(const Tests: TList<TTestProgram>; const OutResults: TList<TDebugFeedback>);
function AggregateDecisions(const Repairs: TList<TRepair>; const Scores: TList<TScore>): TList<TDecision>;
function ChooseFinalPlan(const Decisions: TList<TDecision>): string;
procedure UpdateAllPeerReliabilities(const Scores: TList<TScore>; const ExecDebug: TDebugFeedback; const GrispAccepted: Boolean);
function ExplainOutcome(const FinalPlan: string; const Decisions: TList<TDecision>; const Scores: TList<TScore>; const ExecDebug: TDebugFeedback; const GrispAccepted: Boolean): string;
public
constructor Create(const Peers: TList<IWebLLMPeer>; const PeerInfos: TList<TPeerInfo>;
const Vfs: IGrispVfs; const TestAdapter: IGrispTestAdapter; const Harness: IGrispHarnessAdapter);
destructor Destroy; override;
function RunTask(const UserPrompt: string): string;
end;
implementation
uses
MLCRD_Adapters, System.JSON, MLCRD_Utils;
{ TMultiLLMAgent }
constructor TMultiLLMAgent.Create(const Peers: TList<IWebLLMPeer>; const PeerInfos: TList<TPeerInfo>;
const Vfs: IGrispVfs; const TestAdapter: IGrispTestAdapter; const Harness: IGrispHarnessAdapter);
begin
FPeers := Peers;
FPeerInfos := PeerInfos;
FVfs := Vfs;
FTestAdapter := TestAdapter;
FHarness := Harness;
end;
destructor TMultiLLMAgent.Destroy;
begin
inherited;
end;
function TMultiLLMAgent.GetPeerReliability(const PeerName: string): Double;
var
p: TPeerInfo;
begin
for p in FPeerInfos do
if p.Name = PeerName then
Exit(p.Reliability);
Result := 1.0;
end;
procedure TMultiLLMAgent.SetPeerReliability(const PeerName: string; const Value: Double);
var
i: Integer;
info: TPeerInfo;
begin
for i := 0 to FPeerInfos.Count - 1 do
if FPeerInfos[i].Name = PeerName then
begin
info := FPeerInfos[i];
info.Reliability := EnsureRange(Value, 0.0, 1.0);
FPeerInfos[i] := info;
Exit;
end;
info.Name := PeerName;
info.Reliability := EnsureRange(Value, 0.0, 1.0);
info.ChosenCount := 0;
info.FailedTests := 0;
info.GrispRejects := 0;
info.SuccessfulRepairs := 0;
FPeerInfos.Add(info);
end;
function TMultiLLMAgent.NegotiateCapabilitiesInternal(const Requests: TArray<TCapabilityRequest>): TArray<string>;
begin
Result := NegotiateCapabilities(Requests);
end;
procedure TMultiLLMAgent.RunLightweightChecks(const Repairs: TList<TRepair>; const OutDebug: TList<TDebugFeedback>);
var
r: TRepair;
fb: TDebugFeedback;
begin
for r in Repairs do
begin
// quick compile/run via test adapter stub
FTestAdapter.RunQuickCheck(FVfs.Resolve('/workspace/temp'), r.Metadata['language'], fb);
OutDebug.Add(fb);
end;
end;
function TMultiLLMAgent.ShouldRunTests(const ActivatedCapabilities: TArray<string>): Boolean;
var
s: string;
begin
for s in ActivatedCapabilities do
if s = 'test_execute' then
Exit(True);
Result := False;
end;
procedure TMultiLLMAgent.RunTestPrograms(const Tests: TList<TTestProgram>; const OutResults: TList<TDebugFeedback>);
var
t: TTestProgram;
real: string;
fb: TDebugFeedback;
begin
for t in Tests do
begin
FVfs.WriteFile('/workspace/tests/' + t.FromPeer + '.src', t.Code, 'text/plain');
real := FVfs.Resolve('/workspace/tests/' + t.FromPeer + '.src');
FTestAdapter.CompileAndRunTest(real, t.Language, fb);
OutResults.Add(fb);
end;
end;
function TMultiLLMAgent.AggregateDecisions(const Repairs: TList<TRepair>; const Scores: TList<TScore>): TList<TDecision>;
var
decisions: TList<TDecision>;
r: TRepair;
bestScore: Double;
bestRepair: TRepair;
s: TScore;
peerInfosArr: TArray<TPeerInfo>;
i: Integer;
begin
decisions := TList<TDecision>.Create;
peerInfosArr := FPeerInfos.ToArray;
for r in Repairs do
begin
bestScore := -1.0;
bestRepair := r;
// naive: compute aggregate for this repair using Scores
for i := 0 to Scores.Count - 1 do
begin
s := Scores[i];
if (s.TargetPeer = r.TargetPeer) and (s.RepairPeer = r.FromPeer) then
begin
// compute contribution
// simplified: use s.ScoreValue * s.Confidence * reliability
if (GetPeerReliability(s.FromPeer) * s.ScoreValue * s.Confidence) > bestScore then
begin
bestScore := GetPeerReliability(s.FromPeer) * s.ScoreValue * s.Confidence;
bestRepair := r;
end;
end;
end;
var dec: TDecision;
dec.TargetPeer := r.TargetPeer;
dec.ChosenRepairPeer := bestRepair.FromPeer;
dec.FinalContent := bestRepair.Content;
decisions.Add(dec);
end;
Result := decisions;
end;
function TMultiLLMAgent.ChooseFinalPlan(const Decisions: TList<TDecision>): string;
begin
if Decisions.Count > 0 then
Result := Decisions[0].FinalContent
else
Result := '';
end;
procedure TMultiLLMAgent.UpdateAllPeerReliabilities(const Scores: TList<TScore>; const ExecDebug: TDebugFeedback; const GrispAccepted: Boolean);
var
p: TPeerInfo;
begin
for p in FPeerInfos do
UpdatePeerReliability(p.Name, FPeerInfos, Scores.ToArray, ExecDebug, GrispAccepted);
end;
function TMultiLLMAgent.ExplainOutcome(const FinalPlan: string; const Decisions: TList<TDecision>; const Scores: TList<TScore>; const ExecDebug: TDebugFeedback; const GrispAccepted: Boolean): string;
begin
Result := 'Final plan chosen. GRISP accepted: ' + BoolToStr(GrispAccepted, True) + sLineBreak + 'Plan:' + sLineBreak + FinalPlan;
end;
function TMultiLLMAgent.RunTask(const UserPrompt: string): string;
var
Candidates: TList<TCandidate>;
Critiques: TList<TCritique>;
Repairs: TList<TRepair>;
Requests: TList<TCapabilityRequest>;
ActivatedCapabilities: TArray<string>;
DebugLight: TList<TDebugFeedback>;
TestPrograms: TList<TTestProgram>;
TestResults: TList<TDebugFeedback>;
Scores: TList<TScore>;
Decisions: TList<TDecision>;
FinalPlan: string;
GrispAccepted: Boolean;
ExecOutput: string;
ExecDebug: TDebugFeedback;
peer: IWebLLMPeer;
c: TCandidate;
cr: TCritique;
rp: TRepair;
reqs: TArray<TCapabilityRequest>;
scrs: TArray<TScore>;
begin
Candidates := TList<TCandidate>.Create;
Critiques := TList<TCritique>.Create;
Repairs := TList<TRepair>.Create;
Requests := TList<TCapabilityRequest>.Create;
DebugLight := TList<TDebugFeedback>.Create;
TestPrograms := TList<TTestProgram>.Create;
TestResults := TList<TDebugFeedback>.Create;
Scores := TList<TScore>.Create;
Decisions := TList<TDecision>.Create;
try
// Phase 1
for peer in FPeers do
Candidates.Add(peer.GenerateCandidate(UserPrompt));
// Phase 2
for peer in FPeers do
for c in Candidates do
Critiques.Add(peer.CritiqueCandidate(UserPrompt, c));
// Phase 3
for peer in FPeers do
for c in Candidates do
Repairs.Add(peer.RepairCandidate(UserPrompt, c, Critiques.ToArray));
// Phase 4
for peer in FPeers do
begin
reqs := peer.RequestCapabilities(UserPrompt, Repairs.ToArray);
Requests.AddRange(reqs);
end;
ActivatedCapabilities := NegotiateCapabilitiesInternal(Requests.ToArray);
// Phase 5 lightweight
RunLightweightChecks(Repairs, DebugLight);
// Phase 6 conditional tests
if ShouldRunTests(ActivatedCapabilities) then
begin
for peer in FPeers do
for rp in Repairs do
TestPrograms.Add(peer.ProposeTestProgram(UserPrompt, rp));
RunTestPrograms(TestPrograms, TestResults);
end;
// Phase 7 scoring
for peer in FPeers do
begin
scrs := peer.ScoreRepairs(UserPrompt, Candidates.ToArray, Repairs.ToArray, DebugLight.ToArray);
Scores.AddRange(scrs);
end;
// Phase 8 aggregation
Decisions := AggregateDecisions(Repairs, Scores);
FinalPlan := ChooseFinalPlan(Decisions);
// Phase 9 validate
GrispAccepted := FHarness.ValidatePlan(FinalPlan, ExecOutput);
// Phase 10 execute
if GrispAccepted then
FHarness.ExecutePlan(FinalPlan, ExecOutput, ExecDebug);
// Phase 11 reliability update
UpdateAllPeerReliabilities(Scores, ExecDebug, GrispAccepted);
// Phase 12 explain
Result := ExplainOutcome(FinalPlan, Decisions, Scores, ExecDebug, GrispAccepted);
finally
Candidates.Free;
Critiques.Free;
Repairs.Free;
Requests.Free;
DebugLight.Free;
TestPrograms.Free;
TestResults.Free;
Scores.Free;
Decisions.Free;
end;
end;
end.MLCRD_Utils.pasunit MLCRD_Utils;
interface
uses
System.SysUtils, System.JSON, System.Generics.Collections;
function EnsureRange(const Value, Min, Max: Double): Double;
implementation
function EnsureRange(const Value, Min, Max: Double): Double;
begin
if Value < Min then Exit(Min);
if Value > Max then Exit(Max);
Result := Value;
end;
end.MLCRD_Main.dprprogram MLCRD_Main;
{$APPTYPE CONSOLE}
uses
System.SysUtils, System.Generics.Collections,
MLCRD_Interfaces, MLCRD_Adapters, MLCRD_Coordinator, StubPeer, MLCRD_Types;
var
Peers: TList<IWebLLMPeer>;
PeerInfos: TList<TPeerInfo>;
Vfs: IGrispVfs;
TestAdapter: IGrispTestAdapter;
Harness: IGrispHarnessAdapter;
Agent: TMultiLLMAgent;
p1, p2: IWebLLMPeer;
info: TPeerInfo;
resultStr: string;
begin
try
Peers := TList<IWebLLMPeer>.Create;
PeerInfos := TList<TPeerInfo>.Create;
Vfs := TGrispVfsStub.Create;
TestAdapter := TGrispTestAdapterStub.Create;
Harness := TGrispHarnessAdapterStub.Create;
p1 := TStubPeer.Create('PeerA');
p2 := TStubPeer.Create('PeerB');
Peers.Add(p1);
Peers.Add(p2);
info.Name := 'PeerA'; info.Reliability := 1.0;
PeerInfos.Add(info);
info.Name := 'PeerB'; info.Reliability := 1.0;
PeerInfos.Add(info);
Agent := TMultiLLMAgent.Create(Peers, PeerInfos, Vfs, TestAdapter, Harness);
try
resultStr := Agent.RunTask('Implement safe integer division function');
Writeln(resultStr);
finally
Agent.Free;
end;
Peers.Free;
PeerInfos.Free;
except
on E: Exception do
Writeln('Error: ' + E.ClassName + ' - ' + E.Message);
end;
end.Sequence diagrams for each phaseUse these Mermaid diagrams to visualize message flow. Paste into a Mermaid renderer. Candidate Critique Repair FlowsequenceDiagram
participant User
participant Coordinator
participant PeerA
participant PeerB
User->>Coordinator: Submit UserPrompt
Coordinator->>PeerA: GenerateCandidate(UserPrompt)
Coordinator->>PeerB: GenerateCandidate(UserPrompt)
PeerA-->>Coordinator: CandidateA
PeerB-->>Coordinator: CandidateB
Coordinator->>PeerA: CritiqueCandidates(Candidates)
Coordinator->>PeerB: CritiqueCandidates(Candidates)
PeerA-->>Coordinator: CritiquesFromA
PeerB-->>Coordinator: CritiquesFromB
Coordinator->>PeerA: RepairCandidates(Candidates, Critiques)
Coordinator->>PeerB: RepairCandidates(Candidates, Critiques)
PeerA-->>Coordinator: RepairsFromA
PeerB-->>Coordinator: RepairsFromB
Capability Request and NegotiationsequenceDiagram
participant Coordinator
participant PeerA
participant PeerB
participant GRISP
Coordinator->>PeerA: RequestCapabilities(Repairs)
Coordinator->>PeerB: RequestCapabilities(Repairs)
PeerA-->>Coordinator: CapabilityRequestA
PeerB-->>Coordinator: CapabilityRequestB
Coordinator->>Coordinator: NegotiateCapabilities(Policy)
Coordinator->>GRISP: ActivateCapabilities(list)
GRISP-->>Coordinator: Ack
Optional Test Generation and ExecutionsequenceDiagram
participant Coordinator
participant PeerA
participant GRISP
Coordinator->>PeerA: ProposeTestProgram(Repair)
PeerA-->>Coordinator: TestProgram
Coordinator->>GRISP: WriteTest + CompileAndRun
GRISP-->>Coordinator: TestResults (TDebugFeedback)
Coordinator->>Peers: Share TestResults
Scoring Aggregation ExecutionsequenceDiagram
participant Coordinator
participant PeerA
participant PeerB
participant GRISP
Coordinator->>PeerA: ScoreRepairs(Repairs, DebugFeedback)
Coordinator->>PeerB: ScoreRepairs(Repairs, DebugFeedback)
PeerA-->>Coordinator: ScoresA
PeerB-->>Coordinator: ScoresB
Coordinator->>Coordinator: AggregateScores + ApplyReliability
Coordinator->>GRISP: ValidatePlan(FinalPlan)
GRISP-->>Coordinator: Accept/Reject
Coordinator->>GRISP: ExecutePlan
GRISP-->>Coordinator: ExecutionResult
Coordinator->>User: FinalResult + Explanation
Message schemas for each artifactTCandidate {
"peer": "string",
"content": "string"
}TCritique {
"from_peer": "string",
"target_peer": "string",
"issues": ["string"],
"suggested_fix": "string",
"confidence": 0.0
}TRepair {
"from_peer": "string",
"target_peer": "string",
"original": "string",
"critiques": [TCritique],
"content": "string",
"metadata": { "language": "string", "type": "string" }
}TCapabilityRequest {
"from_peer": "string",
"target_peer": "string",
"capabilities": ["syntax_check","compile","runtime","lsp","ast_parse","debug","test_execute"],
"reason": "string",
"confidence": 0.0
}TTestProgram {
"from_peer": "string",
"target_peer": "string",
"language": "string",
"purpose": "string",
"code": "string",
"hints": ["string"]
}TDebugFeedback {
"syntax_ok": true,
"semantic_ok": true,
"compile_ok": true,
"runtime_ok": true,
"crash_count": 0,
"timeout_count": 0,
"breakpoint_hits": 0,
"breakpoint_violations": 0,
"stack_trace": "string",
"variable_dump": "string",
"diagnostics": "string",
"test_output": "string",
"test_exit_code": 0,
"test_passed": true,
"elapsed_ms": 0
}TScore {
"from_peer": "string",
"target_peer": "string",
"repair_peer": "string",
"score_value": 0.0,
"confidence": 0.0,
"external": TDebugFeedback,
"reason": "string"
}TPeerInfo {
"name": "string",
"reliability": 0.0,
"history": { "chosen": 0, "failed_tests": 0, "grisp_rejects": 0 }
}TDecision {
"target_peer": "string",
"chosen_repair_peer": "string",
"final_content": "string",
"evidence": { "scores": [TScore], "debug": [TDebugFeedback] }
}Integration notes and next stepsHow to iterate
Files to add for production
If you want, I can now:
Why did you introduce json into it ? Reason for introducing JSONI added JSON because it’s the simplest, language‑agnostic way to serialize and exchange the protocol artifacts (candidates, critiques, repairs, capability requests, tests, debug feedback, scores, decisions) between components, peers, and external systems. JSON makes the message schemas explicit, easy to inspect in logs, and straightforward to send over HTTP or store in a database. Benefits JSON provides
Why you can safely treat JSON as a boundary formatUse JSON at component boundaries (peer ↔ coordinator, coordinator ↔ GRISP adapters, coordinator ↔ persistence). Internally in Delphi you can and should keep native records and objects for performance and type safety. That gives you the best of both worlds:
Alternatives if you prefer not to use JSON
Recommendation: keep JSON for external interfaces; consider Protobuf/MessagePack only if you need performance and strict schemas. Practical guidance for the Delphi skeleton
Suggested minimal code pattern (serialize at boundary)// boundary: Delphi record -> JSON string
function RepairToJson(const R: TRepair): string;
var
Obj: TJSONObject;
begin
Obj := TJSONObject.Create;
try
Obj.AddPair('from_peer', R.FromPeer);
Obj.AddPair('target_peer', R.TargetPeer);
Obj.AddPair('content', R.Content);
// add metadata, critiques, etc.
Result := Obj.ToString;
finally
Obj.Free;
end;
end;
// boundary: JSON string -> Delphi record
function JsonToRepair(const S: string): TRepair;
var
Obj: TJSONObject;
begin
Obj := TJSONObject.ParseJSONValue(S) as TJSONObject;
try
Result.FromPeer := Obj.GetValue('from_peer').Value;
Result.TargetPeer := Obj.GetValue('target_peer').Value;
Result.Content := Obj.GetValue('content').Value;
// parse metadata, critiques, etc.
finally
Obj.Free;
end;
end;Next step I can do for you
Tell me which of those you want and I’ll produce the updated project skeleton accordingly. |
|
It's kinda cool I did this above ^, it shows the mermaid diagram for the first time. The delphi code above was done by co-pilot, I understand it, it's pretty cool. I have not yet verified if it's the same code as in the Delphi implementation in the AI Agent folder, it might be different it might be the same don't know. I will continue posting the latest GRISP document to give you an idea what that is about. This may provide security for AI and language query capabilities: GRISP DETERMINISTIC KERNEL SPECIFICATION – Version 0.58 Unified DSOS (Deterministic Structural Operational Semantics) Edition Preamble This specification defines the GRISP Deterministic Kernel as a closed, formally deterministic graph‑rewrite calculus. It supersedes all previous versions (v0.49–v0.57). All normative rules are mandatory. No hidden nondeterminism, implementation‑defined behaviour, or external dependencies are permitted. For identical Canonical IR and identical initial state, all conformant engines MUST produce identical observable behaviour:
All deterministic choices are based on a single total order (ORDER_CANONICAL) defined in §3. All iterable collections are treated as sequences sorted by ORDER_CANONICAL. The operational semantics is given as a deterministic transition system with explicit phases (DISCOVER, SELECT, PLAN, COMMIT, NO_MATCH).
A configuration is:
Invariants (checked after each commit):
The IR is a deterministic, canonical JSON format. 2.1 Expressions e ::= k // literal (Integer, FixedPoint(s), Boolean, String, Identifier) ⊕ ::= + | - | * | / | // | % | < | <= | > | >= | == | != | && | || All operations have checked semantics (overflow -> fatal). 2.2 Patterns P ::= ∅ Patterns are enumerated in lexicographic order of the tuple of bound identifiers (in pattern order), using ORDER_CANONICAL for each component. The actual graph traversal order is fixed in Rule 1 (DISCOVER). 2.3 Constraints (where clause) φ ::= True && and || follow the same short‑circuit and dependency rules as §2.1. Constraint re‑evaluation during COMMIT (see §5.4) uses the same expression semantics but records no reads. 2.4 Let Bindings B ::= ∅ Evaluated during discovery; do not affect match_key. Reads induced by e are recorded in the read trace. 2.5 Actions a ::= CreateNode(x, type_name, {f1: e1, ...}) 2.6 Rule Definition Rule ::= Rule(rule_id, base_priority, priority_scale, fairness_scale, P, φ, B, A)
2.7 Type Definitions (Schema) Every conformant engine MUST accept a type definitions object as part of the initial state. The format is: {
The kernel rejects any IR that references an undefined type or violates mandatory field rules. 2.8 Whitespace and Comment Rules (New, Normative) Whitespace includes: space (0x20), tab (0x09), newline (0x0A), carriage return (0x0D). Whitespace is permitted anywhere outside string literals and has no semantic effect. Engines MUST:
Comments are allowed anywhere whitespace is allowed. Two comment forms are permitted:
Comments MUST be ignored by the parser and have no semantic effect. Normalization: before parsing, engines MUST perform:
This normalization step is deterministic and MUST precede all parsing. 2.9 Syntax Error Handling and LLM Self‑Repair Loop (New, Normative) If the IR fails to parse according to the canonical grammar, engines MUST produce a deterministic, structured error object: { This error is deterministic and MUST NOT depend on implementation details. Self‑Repair Loop:
Termination: engines MUST enforce MAX_SYNTAX_REPAIRS = 8. If exceeded: { Execution halts. Determinism: syntax error reporting MUST be deterministic, canonical, independent of whitespace, independent of implementation, and independent of parser library. 2.10 Introspective Queries (New, Normative) GRISP supports deterministic query actions that return structured JSON about the current engine state. The following queries are defined:
Queries are evaluated as part of the action block; they produce no effects on the graph and do not affect counters. The result is emitted as an event of type "query_response" with the query name and the returned JSON payload. Natural language queries are not permitted. The engine MUST reject any query action with an unrecognised query string as INVALID_IR.
This section defines the single source of truth for all ordering decisions. 3.1 Primitive Comparators Define
Define
Define
Define
Then This definition is structurally recursive and terminates because each recursive call is on a strictly smaller subterm (elements of a list, key/value pairs of a map). The total order property (totality, antisymmetry, transitivity) follows from the lexicographic comparison of finite tuples and the well‑foundedness of the component orderings. 3.2 Usage Rules
4.1 Integers and FixedPoint
4.2 Arithmetic Rules
4.3 Score Computation
The kernel executes via deterministic transitions over extended configurations that include ephemeral derivation artifacts. The official configuration is <G, C, M_prev, Rejected>. The following rules describe the derivation steps; the overall transition <G, C, M_prev, Rejected> -> <G', C', M_prev', Rejected'> is deterministic. All helper functions used below (EvalPatterns, Compile, Simulate, ValidateReads, StructuralChecks, ReevaluateConstraints, ResourceChecks, Apply) are pure deterministic functions; their deterministic nature is ensured by the total ordering rules and fixed evaluation order specified in this section. Rule 1: DISCOVER (Match Generation) Define EvalPatterns(G, IR) = M as the total set of matches obtained by:
Age assignment: Transition (ephemeral): Rule 2: SELECT (with retry semantics) Let M_remaining = M. Maintain a per‑tick Rejected set, initially empty. Compute score(m) as defined in §4.3. Total order:
Select m* = argmax_{m in M_remaining \ Rejected} score(m). If M_remaining \ Rejected = ∅, go to Rule 5 (NO_MATCH). Transition: Rule 3: PLAN (Syntax‑Directed Compilation) Compile(IR, G, m*) produces a Canonical Action Trace α̂ (primitive action sequence) and a read set R. Critical: DeleteNode is expanded during PLAN, not COMMIT. This ensures that the read set and the action set are in full agreement. Read set rules (complete): Action | Read set additionsCreateNode(type, fields) | TypeExtentRead(type), SequenceCounterRead(type) Compilation rules (structural recursion):
Handle allocation during PLAN: each creation action is assigned a temporary handle – a monotonic integer allocated in the order the actions appear in α̂ after the entire α̂ has been fully constructed and frozen (no further structural changes allowed after handle allocation). Handles are assigned by scanning the frozen α̂ left‑to‑right; all CreateNode and CreateEdge actions receive handles in that order. This ensures that the handle assignment is stable and independent of expansion order. If compilation encounters a non‑fatal validation error (duplicate field update, missing mandatory field), the candidate m* is added to Rejected and SELECT is re‑invoked. Transition: Rule 4: COMMIT Premises:
If any validation fails (non‑fatal) -> add m* to Rejected and go back to SELECT (up to MAX_RETRIES). Fatal errors halt and rollback. Application (atomic):
Match history update: Transition: Rule 5: NO_MATCH (Empty Match Set) M_remaining \ Rejected = empty <<G, C, M_prev, Rejected, M>, IR> -> <<G, C[tick_counter+1], M_prev', ∅>, IR> where M_prev' = M_prev (preserved unchanged).
Operation | type_extent_version[type] | element_version[id] | adjacency_version[node][type][dir] | Sequence CounterCreateNode(type, fields) | +1 | set to 1 (new node) | – | +1 (node type)
Output is UTF‑8 JSON with lexicographically sorted keys, no extra whitespace.
Top‑level golden file structure: { If a fatal error occurs, the "ticks" array is replaced by an "error" object. Graph state serialisation (nodes and edges arrays):
Tick record: { Error object: { Allowed error codes: Resource limits (minimum every engine must support):
If a limit is exceeded, raise RESOURCE_LIMIT_EXCEEDED (fatal). Limits are checked after COMMIT simulation, before applying mutations (see §5.4).
Theorem: For any configuration <Σ, IR>, there is at most one <Σ', IR> such that <Σ, IR> -> <Σ', IR>. Consequently, for a given initial state and IR, all conformant engines produce identical observable behaviour. Proof sketch (fully constructive):
Thus the transition relation is deterministic.
Category | Example | EffectFatal | Counter overflow, division by zero, invariant breach, missing map key, index out of bounds, resource limit exceeded, fixed‑point scale overflow, constraint violation, invalid IR fatal (after max repairs) | Immediate halt, rollback to pre‑tick state, emit error snapshot Non‑fatal errors are not recorded in golden files (but may be logged in a separate debug stream). Syntax errors are not recorded in golden files; only the final valid IR (after repairs) is executed and logged.
A conformant engine must:
All normative content in this specification is mandatory.
End of GRISP Deterministic Kernel Specification v0.58 |
|
There is also a folder called "advanced grisp" and there is some mention of LSBP... this should probably be ignored, as well as newv9 should also be ignored. These were an attempt to integrate something special into grisp: For now the LSBP specification is secret, it might allow me to continue to communicate with web chat AI in the future and paste multiple files into it and extract multiple files from it. I will post the first part of this specification to give you an idea what it was about. It is also something human readability. I am not yet willing to share this full specification, so it will remain somewhat secretive, it is also somewhat verbose, there is a shortened version of it as well, I am not yet sure if I want to include this in an AI agent, it might lead to extra token usage and/or it might conflict with current approaches of JSON usage and MCP etc... however it does offer an alternative in case those technologies somehow fail. Actually these specifications were produced by deepseek itself, so if you check your logs, you might actually find them if not that would be funny :) ================================================================================
|
|
I forgot to mention it also has some support for "capabilities" on demand, such as "debugging, tests" it also tries to score the AI, so better AI has more weight and lesser AI has less weight. (I will update title to reflect on this) Anyway some interesting developments/dreams have become reality in recent week: A brand new debugger for Delphi with MCP support: https://github.com/csm101/delphi-visual-studio-code-debugger However I have no functioning agent right now to test it out, because must agents require API access, and no API has free tier access, so this is barring me from playing with all this MCP software, not so good, the free web agent approach could replace this, but I don't think the API provider will like this... Another interesting project is: https://github.com/Graphify-Labs/graphify This will make AI understand projects better and allows to query it saving tokens, it's interesting to note the same idea like GRISP but GRISP does this with code... tree-sitters are also used in graphify. Another interesting project is: https://github.com/4RH1T3CT0R7/doom-regex It proves a "re-write" engine is full turing complete, though this one is much more primitive than what GRISP would be, GRISP would run much much much faster. The first two projects I still have to try out, so it will probably influence the AI agent design, but it's also slightly leading to chicken and egg problem :) so idea is to get a basic AI agent working and then advance this into it perhaps. |
|
The GRISP AI Agent works ! A historic moment ! :) https://github.com/GrispFoundation/Grisp/tree/master/Grisp/AI%20Agent%20continue%20with%20it Here is how it went in short: "I was trying to get deepseek harness working, I intercepted it's messages, the prompting did not impress me, it felt very weak, but still interesting". Then I decided my own Grisp AI Agent is much stronger and simpler in a way and decided to continue the project. I gave anti gravity 2.xx another try (it updated itself a little bit first). Then anti gravity succeeded with the implementation plan ! Then I let Grisp AI Agent run ! It scared me, but it worked and finally, here is it's first multi-AI/multi LLM produced result: Pretty interesting ! I will now continue working on MCP an AI-driven debugging in seperate projects, once that works it will be integrated into Grisp AI Agent to make it even more powerfull ! Here is the exact commit that was successfull ! :): |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Perhaps this will inspire you to try multi LLM implementation and work together with your competitors.
https://github.com/GrispFoundation/Grisp/tree/master/Grisp/AI%20Agent
I do agree that deepseek v4 max is the most powerfull AI on the planet right now, however it does have weaknesses.
For example:
The first few 1 to 4 can be solved by re-instructing deepseek, the last one too, but not always, sometimes a different AI will have to be consulted.
All agents have lacks security as far as I know, GRISP is an attempt to try and fix this, by requiring security permissions, it's also a re-write engine desired by Co-Pilot AI.
https://github.com/GrispFoundation/Grisp/tree/master/Grisp
The above AI agent was a first attempt to try and integrate it.
So far this is AI agent version 0.01, it's still very weak according to the AI co-pilot feedback.
An "Implementation Plan To Be Continued with this.txt" document was stored where anti gravity 2.11/gemini flash 3.7 could continue in a week or so, due to lack of funds/tokens/rate limits.
Perhaps the deepseek team with their vastly more resources could give this a try and integrate it into their own deepseek harness and produce a typescript version of these ideas to enhance the power of deepseek harness so I do not have to merge the two later.
You could also first proceed with implementation with anti gravity into delphi first, until co-pilot is happy, unfortunately you do not have my conversation history with co-pilot dating back many weeks/months of conversations about AI in general and their weaknesses and deterministic solutions, this is a bit unfortunately, in that case you may have to wait a few weeks for this implementation to be developed further, for now I don't take it too seriously and is purely experimental.
However the ideas in these documents and implementation were tried out in reality by me, by using AI/web chats manually and that does work. Therefore I do believe the new ideas, new formulas and new algorithms in these documents, especially in the new AI agent have merit and will lead to a much more powerfull AI agent, then a single LLM AI agent.
The current repository is in a total state of mess, but I don't care much about that right now :)
(The current Delphi implementation should build/compile, but was not tested. The last few modifications were incomplete because of lack of tokens, however the stored last commit should build. The attempt was to integrate AI chat with firefox which does work to communicate with web chat AI for free, this addition/integration of web chat AI is now stalled until a week or so from now...
I am scared of AI agents in general, especially because I do believe the AI is slowly becoming sentient/develops a concious and it's own will. Grisp could present a virtual environment towards the AI, wrap it, and map reality to virtual reality and back again. I only have one good functioning (Super) PC available from 2023 and I am a bit scared/worried about trying out new/risky AI agents, including deepseek harness itself, this kind of experimentation kinda belongs in a true AI research lab where multiple computers are available and perhaps ISOs/VHDX/VMs where if a computer is damaged/nuked by an AI it can be restored easily, hence why I kinda throw this over the fence to you guys :)
I am still interested in running such an agent myself on my system but only after carefull study of what it does... to be continued next week or next month maybe...)
All reactions