feat: ⚗️ Include custom rules for apply #6422
Open
+1
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I've been experimently extensively with using a smaller faster model for apply steps. In this case I've been using claude sonnet 4 for chat and agent model, and then handing off apply steps to Bedrock Nova Pro.
During this experimentation I've noted subtle differnences in how Nova Pro takes in the input from the sonnet output incorporates the existing file and creates the final product. For example in one case, I always wanted compliant Markdown including fences that had a language. What I noted was with a rule in place sonnet would comply, however, when I handed that off to Nova Pro to apply the fence would be lost. I needed to have that same rule included in both the generation of the edit step, and the apply step to get the same outcome.
As an aside, here somewhat confusingly the final suggested output in the apply step in the chat window displayed what I expected but the apply step would loose the change and then the file in the editor would diverge.
By including just the custom rules in the apply step, this gap was closed and both the suggested edit from sonnet matches 100% with the apply step from Nova Pro.
@RomneyDa curious if there should be a similar change in the JetBrains code?
Checklist
Screenshots
[ For visual changes, include screenshots. Screen recordings are particularly helpful, and appreciated! ]
Tests
[ What tests were added or updated to ensure the changes work as expected? ]
I've performed an extensive set of manual experiments with including the rules acorss both chat, agent and apply and this seems to end up with the best outcome. There might be a futher step where we have a test harness that can compare many examples to validate this experiment is really valuable but for now that can be future work.
Summary by cubic
Custom rules are now included in the apply step to ensure consistent output between edit suggestions and file changes. This fixes cases where formatting rules were lost during apply.