Skip to content

feat: goal tracking in observability — cost per goal progressed#636

Merged
kokevidaurre merged 1 commit intodevelopfrom
feat/goal-tracking
Mar 27, 2026
Merged

feat: goal tracking in observability — cost per goal progressed#636
kokevidaurre merged 1 commit intodevelopfrom
feat/goal-tracking

Conversation

@kokevidaurre
Copy link
Copy Markdown
Contributor

Every run snapshots goals before/after, diffs changes. Org cycle shows goal summary. Enables measuring cost per goal progressed.

Every execution now captures:
- goals_before: goal name → status before the run
- goals_after: goal name → status after the run
- goals_changed: list of goals that moved (e.g., not-started → in-progress)

Org cycle (squads run --org) shows goal changes summary at the end.
obs history shows goal changes per execution record.

Also added grade/grade_score fields to ObservabilityRecord for
future COO eval integration.

This enables measuring: cost per goal progressed — the key metric
for self-improving autonomous operation.

Co-Authored-By: Claude <noreply@anthropic.com>
@kokevidaurre kokevidaurre merged commit cfe0254 into develop Mar 27, 2026
11 checks passed
@kokevidaurre kokevidaurre deleted the feat/goal-tracking branch March 27, 2026 15:35
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces goal tracking and observability improvements by adding functionality to snapshot and diff goals from 'goals.md' files before and after execution cycles. The changes include new utility functions in 'observability.ts', updates to the execution engine to capture these snapshots, and enhanced reporting in the CLI. I have provided a suggestion to simplify the 'diffGoals' function by using a single loop over a combined set of goal names for better efficiency and readability.

Comment on lines +240 to +256
const changes: GoalChange[] = [];

for (const [name, afterStatus] of Object.entries(after)) {
const beforeStatus = before[name] || 'new';
if (beforeStatus !== afterStatus) {
changes.push({ name, before: beforeStatus, after: afterStatus });
}
}

// Goals that disappeared (moved to achieved/abandoned)
for (const [name, beforeStatus] of Object.entries(before)) {
if (!(name in after)) {
changes.push({ name, before: beforeStatus, after: 'removed' });
}
}

return changes;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The body of this function can be simplified to use a single loop, which is more efficient and easier to read. By creating a combined set of all goal names from both before and after snapshots, you can check for differences in a single pass.

Suggested change
const changes: GoalChange[] = [];
for (const [name, afterStatus] of Object.entries(after)) {
const beforeStatus = before[name] || 'new';
if (beforeStatus !== afterStatus) {
changes.push({ name, before: beforeStatus, after: afterStatus });
}
}
// Goals that disappeared (moved to achieved/abandoned)
for (const [name, beforeStatus] of Object.entries(before)) {
if (!(name in after)) {
changes.push({ name, before: beforeStatus, after: 'removed' });
}
}
return changes;
const changes: GoalChange[] = [];
const allNames = new Set([...Object.keys(before), ...Object.keys(after)]);
for (const name of allNames) {
const beforeStatus = before[name];
const afterStatus = after[name];
if (beforeStatus !== afterStatus) {
changes.push({
name,
before: beforeStatus ?? 'new',
after: afterStatus ?? 'removed',
});
}
}
return changes;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant