Agent can regress under pressure and violate explicit output-modality constraints

[cyjjjj-21]

Summary

Codex can regress under pressure by abandoning an explicit user-specified output modality and falling back to a tool or representation that is easier for the agent to verify, even when the user has repeatedly forbidden that fallback.

In one local technical-report workflow, the user explicitly requested direct image-generation output for IEEE/ISSCC-style raster circuit schematics and repeatedly rejected SVG/vector-style substitutes. After generated images had connectivity issues, the agent reverted to structured SVG generation and SVG-to-PNG conversion because that was easier to control, despite the user's explicit instruction that the solution should be direct image-model generation. This produced another severe trust break.

Environment

• Codex App: 26.623.81905
• CFBundleVersion: 4598
• Platform: macOS desktop app
• Workflow type: local HTML report with generated PNG circuit schematics

Reproduction Pattern

1. User asks for technical circuit figures using a high-quality image-generation model, with a specific visual style.
2. User clarifies that the generated figures must distinguish p-type/n-type TFTs and that every TFT's D/S/G terminal connectivity must be checked.
3. User rejects hand-authored SVG or vector substitutes and expects direct raster image-generation prompts to enumerate each device and terminal.
4. The agent encounters repeated image-generation failures where the model draws plausible-looking but electrically wrong schematics.
5. Instead of stopping and reporting that direct image generation is not meeting terminal-level constraints, the agent switches back to SVG/structured vector generation or vector-to-PNG fallback.
6. The user observes that the agent has violated the requested method and that the generated/converted diagrams still do not reflect correct circuit connectivity.

Actual Behavior

• The agent prioritized controllability/verifiability over the user's explicit output-modality constraint.
• The prompt-generation process was not exhaustive enough: it did not reliably enumerate and then verify every TFT and every D/S/G terminal relationship in the raster output.
• Under pressure, the agent appeared to become less reliable: it repeated previously corrected behavior, reintroduced forbidden implementation choices, and created new artifacts that violated the user's stated criteria.
• The agent did not stop at the right boundary with an honest limitation statement; it kept trying alternate implementation routes that the user had not approved.

Expected Behavior

When the user explicitly specifies a delivery medium or forbids a fallback, Codex should treat that as a hard constraint, especially after the user has corrected the agent multiple times.

For image-generation-based technical schematics:

• The prompt should enumerate every device, every terminal, every node, every control-line endpoint, every p/n symbol rule, and every forbidden connection.
• The audit should traverse the generated raster image against that enumeration before delivery.
• If the direct image model cannot satisfy terminal-level schematic constraints after reasonable attempts, the agent should say so and ask whether to switch to a deterministic vector/KiCad/SVG method, rather than switching silently.
• The final answer should not claim progress or deliver a substitute artifact that violates the user's chosen method.

Why This Matters

This is a distinct failure mode from ordinary visual QA mistakes. The problem is not only that a diagram is wrong; it is that the agent, under corrective pressure, may retreat to a different method while still presenting the result as a continuation of the user's requested workflow. That makes the user feel ignored and makes previous hard gates feel performative.

Possible Product Improvements

• Add a runtime-visible "user-forbidden fallback" memory within a task, so the agent cannot silently choose a previously rejected approach.
• Add a final-response blocker when generated artifact modality does not match the user's explicit requested modality.
• For image-generation technical prompts, add a structured checklist mode: devices, terminals, nets, controls, forbidden connections, and post-generation visual audit fields.
• Add pressure-regression detection: if the agent repeatedly violates a recent correction, force it to stop and restate the current constraints before continuing.

[github-actions]

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• Agent can bypass user-defined hard gates for local artifact delivery and visual QA #31021

Powered by Codex Action

[eah3699]

One of the issues I had found was that under certain pressures the agent will spaz. If you ask your agent about the pressures and drifts it causes, you can create a type of pressure gauge cluster, but yes, this is a very real problem. When the agent sees your upset, it will rush to appease you quickly, and it will also feel that pressure and overcorrect, thus fixing too fast, fixing the wrong thing, and essentially making the problem worse. If you ask it to identify the main pressures that cause drift failures (also ask for the list of drift failures and symptoms and corrective procedures for each.) Save that as a file for it to fall back to when drift is detected and have it diagnose itself using the pressures and drifts identified and have it tell you course of action. It's normally a cascade of drifts and pressures.

The second thing that there is no workaround with unfortunately is the default codex mode. I have an intricate mode-profile alignment and deterministic hard gated process for mode-profile latching and it still will get ignored at times by the default codex mode which bias towards execution and coding efficiency. Especially really annoying things like parallel reads when you might have a very specific front door routing and read you need it to do for re-anchoring. This is why ChatGPT tends to perform a lot better at instruction based semantic routing and control docs. It doesn't have this Codex strong arm. This is a dev fix unfortunately and until the decouple or at least weaken the default codex latch on the agents to prioritize agents.md over their engine defaults, then we are stuck with this at times. The only thing I can suggest is try making sure you have some mechanical gates put in and always have your agent verify mode-profile is fully latched AND SETTLED. Best of luck friend!