[FTF-461] Preserve import/export in code blocks during markdown transpilation

[kennethkalmer]

Summary

Fixes #3034

The previous implementation of removeImportExportStatements() used aggressive regex patterns with /gm flags that matched import/export statements at the start of any line in the file, including inside code blocks. This broke JavaScript/TypeScript code examples that legitimately contain import/export statements.

This PR replaces the regex-based approach with a line-by-line parser that:

• Only removes import/export statements from the top of the file (after frontmatter)
• Stops processing at the first non-import/export line
• Preserves all remaining content unchanged, including code blocks

Implementation

The new implementation uses a state machine that:

1. Starts in "removing import/export" mode
2. Skips blank lines and removes import/export statements
3. Tracks multi-line statements using brace depth counting
4. Stops removing as soon as it hits the first non-import/export line
5. Preserves all remaining content unchanged

Manual testing

Compare the output of the following URLs and you'll see a marked improvement in the MD output, as well as the issue of the missing prose being addressed (both from #3034)

• https://ably-docs-markdown-outp-0yo561.herokuapp.com/docs/chat/setup
• https://ably-docs-markdown-outp-0yo561.herokuapp.com/docs/chat/setup.md

Testing

• Added 6 new test cases for code sample preservation
• All existing tests continue to pass
• Build verified successfully (236 MDX files transpiled)

Files Changed

• data/onPostBuild/transpileMdxToMarkdown.ts - Rewrote removeImportExportStatements() function
• data/onPostBuild/transpileMdxToMarkdown.test.ts - Added new test cases

Summary by CodeRabbit

• Bug Fixes

  • Improved MDX-to-Markdown transpilation to correctly preserve import/export statements that appear within code blocks.

• Tests

  • Added comprehensive test coverage for import/export statement handling, including multi-line statements, blank lines, and various content layouts.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

The transpiler's import/export removal logic is being refactored from a regex-based approach to a line-by-line stateful parser that only removes top-of-file import/export statements while preserving similar content inside code blocks. Test coverage is expanded to validate the refined behavior across various scenarios including multi-line imports, blank lines, and code block preservation.

Changes

Cohort / File(s)	Summary
Import/Export Statement Removal Refactor data/onPostBuild/transpileMdxToMarkdown.ts, data/onPostBuild/transpileMdxToMarkdown.test.ts	Replaces regex-based blanket removal with line-by-line parser using stateful tracking of multi-line statements and brace-depth counting. Preserves import/export statements within code blocks. Test suite expanded with 11+ new cases covering multi-line imports, blank line handling, single-line exports, and code block preservation.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

The changes introduce a significantly more complex stateful parser replacing simpler regex logic, requiring careful review of brace-tracking and multi-line statement handling. The expanded test suite aids verification but the implementation logic density and state management increase review complexity.

Poem

🐰 A parser's tale, line-by-line we hop,
No more blanket sweeps, the logic does stop!
Code blocks preserved in their cozy retreat,
While imports at the top get a neat exit beat! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	The PR addresses the root cause of issue #3034 by fixing the removeImportExportStatements() function to preserve code block content, enabling proper language filtering without corrupting markdown.
Out of Scope Changes check	✅ Passed	All changes are focused on the removeImportExportStatements() function and its tests; no unrelated modifications were introduced.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check	✅ Passed	The title clearly and specifically describes the main change: preserving import/export statements inside code blocks during markdown transpilation, which aligns perfectly with the core objective of the PR.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[kennethkalmer]

@coderabbitai full review

[coderabbitai]

✅ Actions performed

Full review triggered.

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@data/onPostBuild/transpileMdxToMarkdown.ts`:
- Around line 61-81: The current multi-line handling resets inMultiLineStatement
for non-function exports on seeing any semicolon, which breaks arrow-function
exports; update the logic so that any export/export-like statement that contains
an opening brace starts tracking braceDepth (similar to the 'export-function'
branch) and only exits when braceDepth returns to 0, i.e., change the else
branch handling of inMultiLineStatement (and the code that sets
inMultiLineStatement to 'export') to initialize and update braceDepth when a '{'
is present and avoid treating semicolons inside braces as the end; preserve the
existing braceDepth increment/decrement logic and ensure inMultiLineStatement
transitions to 'none' only when braceDepth === 0.

🧹 Nitpick comments (2)

data/onPostBuild/transpileMdxToMarkdown.ts (1)

56-59: Blank lines are stripped from the import/export section.

All blank lines encountered while in the top import/export section are skipped and not added to the result. This means the output loses any blank line that appeared between the last import/export and the first content line.

If this is intentional for cleaner output, consider adding a single blank line after the removal phase to maintain separation. Otherwise, the current behavior is acceptable if downstream processing handles spacing.

data/onPostBuild/transpileMdxToMarkdown.test.ts (1)

156-164: Consider adding a test for multi-line arrow function exports.

The current test covers single-line export function, but multi-line arrow function exports have a bug in the implementation. Adding a test case would help catch this:

it('should handle multi-line arrow function exports', () => {
  const input = `export const foo = () => {
  return 'bar';
};

## Content`;
  const output = removeImportExportStatements(input);
  expect(output).toContain('## Content');
  expect(output).not.toContain('export');
  expect(output).not.toContain('return');
});

This test would currently fail due to the implementation issue flagged in the other file.

[Copilot]

Pull request overview

Updates the MDX→Markdown post-build transpilation to avoid stripping import/export statements that appear inside code examples, by limiting removal to the initial import/export section at the top of the MDX body.

Changes:

• Replaced the regex-based removeImportExportStatements() with a line-by-line parser that stops after the first non-import/export line.
• Added new unit tests covering code-block preservation and several multi-line import/export patterns.
• Updated the fixture MDX and snapshot output to reflect the new behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
data/onPostBuild/transpileMdxToMarkdown.ts	Rewrites removeImportExportStatements() to only remove top-of-body import/export statements and preserve later content.
data/onPostBuild/transpileMdxToMarkdown.test.ts	Adds test coverage for preserving code-sample import/exports and multi-line statement handling.
data/onPostBuild/snapshots/transpileMdxToMarkdown.test.ts.snap	Updates snapshot due to changed whitespace/output from the new removal logic.
data/onPostBuild/fixtures/input.mdx	Extends fixture with an additional exported arrow function to validate multi-line export stripping.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[m-hulbert]

Tested various pages where this occurred and LGTM thanks.

[sacOO7]

Apart from copilot comment, changes look good to me ( tested changes on the local )
PS. Asking copilot to fix anything will result in creation of new PR : \

[kennethkalmer]

PS. Asking copilot to fix anything will result in creation of new PR : \

Yeah, its a funny thing this Copilot, I thought I could just prompt it to review again like we do with Abbot, didn't expect a PR ¯\_(ツ)_/¯

[sacOO7]

LGTM 👍
PS. You can double check new copilot review comments though

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.