Skip to content

feat: ⚗️ Include custom rules for apply #6422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

chezsmithy
Copy link
Contributor

@chezsmithy chezsmithy commented Jul 2, 2025

Description

I've been experimently extensively with using a smaller faster model for apply steps. In this case I've been using claude sonnet 4 for chat and agent model, and then handing off apply steps to Bedrock Nova Pro.

During this experimentation I've noted subtle differnences in how Nova Pro takes in the input from the sonnet output incorporates the existing file and creates the final product. For example in one case, I always wanted compliant Markdown including fences that had a language. What I noted was with a rule in place sonnet would comply, however, when I handed that off to Nova Pro to apply the fence would be lost. I needed to have that same rule included in both the generation of the edit step, and the apply step to get the same outcome.

As an aside, here somewhat confusingly the final suggested output in the apply step in the chat window displayed what I expected but the apply step would loose the change and then the file in the editor would diverge.

By including just the custom rules in the apply step, this gap was closed and both the suggested edit from sonnet matches 100% with the apply step from Nova Pro.

@RomneyDa curious if there should be a similar change in the JetBrains code?

Checklist

  • I've read the contributing guide
  • The relevant docs, if any, have been updated or created
  • The relevant tests, if any, have been updated or created

Screenshots

[ For visual changes, include screenshots. Screen recordings are particularly helpful, and appreciated! ]

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

I've performed an extensive set of manual experiments with including the rules acorss both chat, agent and apply and this seems to end up with the best outcome. There might be a futher step where we have a test harness that can compare many examples to validate this experiment is really valuable but for now that can be future work.


Summary by cubic

Custom rules are now included in the apply step to ensure consistent output between edit suggestions and file changes. This fixes cases where formatting rules were lost during apply.

@chezsmithy chezsmithy requested a review from a team as a code owner July 2, 2025 23:37
@chezsmithy chezsmithy requested review from RomneyDa and removed request for a team July 2, 2025 23:37
Copy link

netlify bot commented Jul 2, 2025

👷 Deploy request for continuedev pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 8186ebb

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jul 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

1 participant