fix: encode honors context-vs-input governance distinction#98
Merged
Conversation
Server now follows klappy://odd/encoding-types/how-to-write-encoding-types section 'Context vs Input': input generates artifacts; context only informs quality scoring. Before this change, fullInput (input + context) was passed to both the parser and the scorer, causing context paragraphs to become separate standalone artifacts. The governance says context is metadata, not content. Changes: - runEncodeAction: parsers receive input only (not fullInput) - runEncodeAction: scoring receives input + context per artifact so background information still counts toward quality - scoreArtifactQuality: accepts optional scoringText parameter that defaults to artifact.body when not provided - Inline comments cite the governance doc to prevent regression Closes the gap between governance and code surfaced during PR #96 testing.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
oddkit | e95af36 | Commit Preview URL Branch Preview URL |
Apr 16 2026, 03:11 AM |
…t from corrupting negative/positive checks
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Non-empty check passes for empty artifacts with context
- Changed the non-empty criterion check on line 428 from
text.length > 0back toartifact.body.length > 0so that context cannot substitute for actual artifact content.
- Changed the non-empty criterion check on line 428 from
Preview (e95af368db)
diff --git a/workers/src/orchestrate.ts b/workers/src/orchestrate.ts
--- a/workers/src/orchestrate.ts
+++ b/workers/src/orchestrate.ts
@@ -402,15 +402,22 @@
function scoreArtifactQuality(
artifact: ParsedArtifact,
criteria: Array<{ criterion: string; check: string; gapMessage: string }>,
+ scoringText?: string,
): { score: number; maxScore: number; level: string; gaps: string[]; suggestions: string[] } {
const gaps: string[] = [];
const suggestions: string[] = [];
let score = 0;
+ // Governance: context informs quality scoring. When scoringText is provided
+ // (artifact.body + context), criteria check against that combined text so
+ // background information in context (rationale, alternatives, evidence)
+ // counts toward the artifact's quality without becoming separate artifacts.
+ // See: klappy://odd/encoding-types/how-to-write-encoding-types#context-vs-input
+ const text = scoringText ?? artifact.body;
if (criteria.length === 0) {
- if (artifact.body.split(/\s+/).length >= 10) score++;
+ if (text.split(/\s+/).length >= 10) score++;
else suggestions.push("Expand — more detail improves quality");
- if (/because|due to|since/i.test(artifact.body)) score++;
+ if (/because|due to|since/i.test(text)) score++;
else suggestions.push("Add rationale");
return { score, maxScore: 2, level: score >= 2 ? "adequate" : "weak", gaps, suggestions };
}
@@ -419,11 +426,11 @@
const ck = c.check.toLowerCase();
let passed = false;
if (ck.includes("non-empty")) passed = artifact.fields.length > 3 || artifact.body.length > 0;
- else if (ck.includes("10")) passed = artifact.body.split(/\s+/).length >= 10;
- else if (ck.includes("number") || ck.includes("concrete")) passed = /\d/.test(artifact.body);
+ else if (ck.includes("10")) passed = text.split(/\s+/).length >= 10;
+ else if (ck.includes("number") || ck.includes("concrete")) passed = /\d/.test(text);
else if (ck.includes("interpretation") || ck.includes("does not contain")) passed = !/should|better|worse|means|implies/i.test(artifact.body);
else if (ck.includes("prohibition") || ck.includes("requirement")) passed = /must|must not|never|always|shall/i.test(artifact.body);
- else passed = artifact.body.split(/\s+/).length >= 5;
+ else passed = text.split(/\s+/).length >= 5;
if (passed) score++;
else { gaps.push(c.gapMessage); suggestions.push(c.gapMessage); }
}
@@ -1415,19 +1422,26 @@
state?: OddkitState,
): Promise<ActionResult> {
const startMs = Date.now();
- const fullInput = context ? `${input}\n${context}` : input;
+ // Governance: input generates artifacts; context only informs quality scoring.
+ // See: klappy://odd/encoding-types/how-to-write-encoding-types#context-vs-input
+ // Do not pass fullInput to parsers — that would create separate artifacts
+ // for each context paragraph instead of letting context inform scoring.
const types = await discoverEncodingTypes(fetcher, canonUrl);
- const structured = isStructuredInput(fullInput);
+ const structured = isStructuredInput(input);
const artifacts = structured
- ? parseStructuredInput(fullInput, types)
- : parseUnstructuredInput(fullInput, types);
+ ? parseStructuredInput(input, types)
+ : parseUnstructuredInput(input, types);
- // Score each artifact using its type's quality criteria
+ // Score each artifact using its type's quality criteria.
+ // When context is provided, append it to the artifact's body for scoring
+ // so background information (rationale, alternatives, evidence) counts
+ // toward the artifact's quality without becoming separate artifacts.
const scoredArtifacts = artifacts.map((a) => {
const typeDef = types.find((t) => t.letter === a.type);
const criteria = typeDef ? typeDef.qualityCriteria : [];
- const quality = scoreArtifactQuality(a, criteria);
+ const scoringText = context ? `${a.body}\n${context}` : undefined;
+ const quality = scoreArtifactQuality(a, criteria, scoringText);
return { title: a.title, type: a.type, typeName: a.typeName, content: a.body, fields: a.fields, quality };
});You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit e6e9e6c. Configure here.
The non-empty check in scoreArtifactQuality was using text (which includes appended context) instead of artifact.body. This meant an artifact with an empty body would pass the non-empty check whenever context was supplied, defeating the purpose of the validity check. Restored the check to use artifact.body.length so context informs quality scoring but cannot substitute for actual artifact content.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Closes the gap between governance and code surfaced during PR #96 testing.
The Gap
Governance (klappy://odd/encoding-types/how-to-write-encoding-types#context-vs-input):
Server code (after PR #96):
Result: context paragraphs became separate standalone artifacts instead of informing the primary artifact's score.
The Fix
runEncodeAction: parsers receiveinputonly (notfullInput)runEncodeAction: scoring receivesinput + contextper artifactscoreArtifactQuality: accepts optionalscoringTextparameterVerification
TypeScript compiles clean. Manual verification on preview after merge.
Note
Medium Risk
Behavior of
encodeoutput and quality ratings changes, which could affect downstream expectations/tests, but the change is localized and not security-sensitive.Overview
Fixes
encodeto honor the governance rule that context should not generate artifacts.runEncodeActionnow parses artifacts frominputonly, and passescontextonly into quality scoring by appending it to each artifact’s body at scoring time.scoreArtifactQualityis updated to accept optionalscoringText, so length/rationale/concreteness checks can considerbody + contextwithout changing the persisted artifact content.Reviewed by Cursor Bugbot for commit e95af36. Bugbot is set up for automated code reviews on this repo. Configure here.