fix(magic blocks): ensure newline characters processed as hard breaks

[maximilianfalco]

	Fix CX-2912, CX-2911, CX-2916

🧰 Changes

During some triage we saw that when newline characters are attached to the previous word, it doesnt get parsed properly as a hard break but instead as a soft break. this was the default behaviour of remark-parse and is inherently different from how RDMD legacy processed these line breaks.

this PR adds 2 things

1. bringing back remark-breaks to fully support legacy hard break behaviour
2. adds a preprocess step in magic blocks to ensure leading newlines characters are respected

Before	After

Heres an after demo in the main ReadMe app after linking locally:

🧬 QA & Testing

Testing Source Code

Line 1
Line 2

- Item 1
- Item 2
This should not be merged to Item 2

---

[block:parameters]
{
  "data": {
    "h-0": "When...",
    "h-1": "Then...",
    "0-0": "When option A is selected,",
    "1-0": "When option B is selected,",
    "2-0": "When option C is selected,",
    "0-1": "\nenter the value in the **Input** field.\n\nFor example: `items/page/1`",
    "1-1": "configure these fields:\n   a. In the **Label** field, enter a name.\n   For example: `color` or `size`.\n   b. In the **Amount** field, enter a number.\n   For example: `5`. When combining multiple values, the result may look like:\n- `color=red; size=5`\n- `type=large, color=blue`",
    "2-1": "configure these fields:\n   a. In the **Key** field, enter the lookup key.\n   For example: `region`\n   b. In the **Value** field, enter the lookup value.\n   For example: `us-east` (the full path could look like: `http://example.com/api/?region=us-east`)"
  },
  "cols": 2,
  "rows": 3
}
[/block]


---

[block:parameters]
{
  "data": {
    "h-0": "Option",
    "h-1": "Description",
    "0-0": "`format`",
    "0-1": "Sets the output format.\n• **JSON** returns structured data.\n• **XML** returns markup.\n• **CSV** returns flat rows."
  },
  "cols": 2,
  "rows": 1
}
[/block]

---

Try typing something like the attached magic block source code and we should now register these as proper newlines

• Broken on production.
• Working in this PR app.

[rafegoldberg]

This only addresses non-whitespace prefixed line breaks in magic blocks,¹ but the discrepancy between legacy and Xish still exists in all other contexts. The legacy engine had a custom compiler step that handled these breaks which we should replicate.

That said, I'm open to doing this in a separate PR if you prefer @maximilianfalco?

Footnotes

1. I've updated the title of this PR to reflect this limitation. ↩

[rafegoldberg]

Seems like we used to use the remark-breaks plugin.^↗ That might be a good starting point for the non-magic blocks side of this work?

[maximilianfalco]

@rafegoldberg ohh didnt know we had that! thanks for the info! yea I'll give it a shot first, if the diffs arent that large, I think we can combine the fix for magic blocks and non magic blocks as well in this PR but if the diff is large, may have to just open a new PR

[maximilianfalco]

it looks like integrating the remark-breaks plugin would solve this nicely and also close out this PR #1328

cc @eaglethrost @rafegoldberg @kevinports

[eaglethrost]

Nice!

[eaglethrost]

Nice thanks for integrating the remark-breaks! Seem to solve the line break issue as well, code looks good to me.

[kevinports]

Lgtm

[rafegoldberg]

This PR was released!

🚀 Changes included in v13.1.2