fix: 🐛 Fix bare markdown fences and premature code generation stops

[chezsmithy]

Description

Claude 4 sonnet is very excited about creating project structure display in readme files. When this was attempted the markdown display was broken and the file output was truncated. This was due file structure being outputted with nested codeblocks without a type attached.

When I orginally implemented a fix to this code I noted that there might be some risky edge cases where the nested code blocks didn't have types. As such this PR should solve for that use case by implementing a look ahead feature to check upcoming code blocks to determine if nesting is starting or we are ending the outer code block nesting early.

Significant hardening has been done on all of the places where markdown is used. I've still noted a few issues in chat where markdown might be returned and fences are not balanced (for example an open fence is included indented, and the closing fence happens to be not indented or enclosed in quotes) that could be further improved in a new PR.

Further, I noted that Bedrock Lower cost nova models we want to use for edit and apply were causing problems with outputting markdown content that wasn't compatible with the current prompts and filtering logic. This has been solved for.

Checklist

• I've read the contributing guide
• The relevant docs, if any, have been updated or created
• The relevant tests, if any, have been updated or created

Screenshots

[ For visual changes, include screenshots. Screen recordings are particularly helpful, and appreciated! ]

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

I've added a new function for the look ahead checking, and added new tests for that. Then I added a specfic test for the nested structure output that was failing.

I've added extensive tests for all the changes included functions that have been modified. During manual testing, I have found additional edge cases where the generation was stopping pre-maturely and I've added tests to attempt to solve for those. My goal has been to allow using AI to manage the lineStream.ts, and lineStream.vitetest.ts themselves which has been a problem traditionally becuase they contained stop blocks and fences that broke code generation. I've validated that its now possible to refactor those files sucessfully.

---

Summary by cubic

Fixed an issue where project structure markdown with nested bare codeblocks would break display and truncate output.

• Bug Fixes
  • Added logic to detect and handle nested bare codeblocks without a type.
  • Updated tests to cover these edge cases.

[netlify]

👷 Deploy request for continuedev pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	1364466

[sestinj]

@chezsmithy There is a failing test, but I do like the PR. Will merge once tests are fixed

[chezsmithy]

TODO Items:

• Passing tests
• Review all ``` and STOP checking to make sure it is looking only for leading backticks or STOP keywords and not catching examples inside of code strings. Thus tests won't generate using these keywords or truncation can occur.
• Optimize lookahead function for ``` / STOP checking to not scan the whole remainder of the file if we can.

[chezsmithy]

@sestinj feeling good about this now. I've added a tone of tests and split out the filterCodeBlockLines function so that it's easier to test in issolation as its such a critical component.

[RomneyDa]

This fix will be great!
See comments, I think there's some double-duplicate code.
If possible, let's move all the markdown-patching-related code to core utils, have it in one place, and import it into GUI