fix(web): strip CJK punctuation from auto-linked URLs

[hqhq1025]

Summary

• Add remark-strip-cjk-autolink plugin that walks the MDAST after remark-gfm and moves trailing CJK/fullwidth punctuation (，、。、！ etc.) out of auto-linked URL nodes into sibling text nodes
• Register the plugin in MARKDOWN_PLUGINS right after remarkGfm
• No external dependencies — uses a simple recursive tree walker with typed MDAST interfaces

Closes #478

Example

Before: https://github.com/issues/477，涵盖了 → entire 477， is part of the link

After: https://github.com/issues/477 is the link, ，涵盖了 is plain text

Test plan

• Unit tests cover: trailing fullwidth comma, ideographic full stop, multiple trailing CJK chars, no false positives on clean URLs, no stripping of CJK path segments
• Verify in the web UI that URLs followed by Chinese punctuation render with correct link boundaries

[cloudflare-workers-and-pages]

Deploying hapi with    Cloudflare Pages

Latest commit:	b92ecee
Status:	✅  Deploy successful!
Preview URL:	https://893acafb.hapi-bqd.pages.dev
Branch Preview URL:	https://fix-cjk-autolink-punctuation.hapi-bqd.pages.dev

View logs

[github-actions]

Findings

• [Major] Transformer rewrites non-autolink markdown links and can truncate legitimate CJK URL suffixes. It currently runs on every link node and the strip set includes paired suffixes like ）】」》, so an explicitly-authored link ending with one of those characters will be silently mutated. Evidence web/src/lib/remark-strip-cjk-autolink.ts:14, web/src/lib/remark-strip-cjk-autolink.ts:29.

Summary

• Review mode: initial
• 1 major regression risk found in the new markdown autolink transformer.

Testing

• Not run (automation). Suggested additions: explicit markdown-link regression coverage and a case where the URL legitimately ends with a fullwidth closing bracket/parenthesis.

HAPI Bot

[github-actions]

[MAJOR] This rewrites every link node, not just GFM autolinks, and the strip set includes paired suffixes like ）】」》. That means an explicitly-authored markdown link ending with one of those characters will be silently truncated and get an extra text node appended. Please make the transform opt-in to autolink-shaped nodes before mutating, and avoid stripping paired brackets unless you can prove they are delimiters.

Suggested fix:

const TRAILING_CJK_PUNCT = /[，。、；：！？\u3000\uFF0C\uFF0E\uFF1B\uFF1A\uFF01\uFF1F]+$/

const textChild = child.children?.length === 1 ? child.children[0] : undefined
if (
    child.type !== "link" ||
    typeof child.url !== "string" ||
    textChild?.type !== "text" ||
    textChild.value !== child.url
) {
    visitLinks(child)
    continue
}

[github-actions]

Findings

• [Major] The new strip set still removes paired fullwidth delimiters from any autolink-shaped URL, so valid URLs that legitimately end with ）, 】, 」, or similar characters will be silently truncated. Evidence web/src/lib/remark-strip-cjk-autolink.ts:14, web/src/lib/remark-strip-cjk-autolink.ts:39.

Summary

• Review mode: follow-up after new commits
• 1 major regression risk remains in the autolink punctuation stripper.

Testing

• Not run (automation; PR code not executed). Suggested addition: cover an autolink whose real path ends with a fullwidth closing delimiter.

HAPI Bot

[github-actions]

[MAJOR] TRAILING_CJK_PUNCT still includes paired fullwidth delimiters such as ）】」』》〉. For bare autolinks, those characters can be legitimate path suffixes, so this transform will silently truncate valid URLs and move the last character into plain text. Please narrow the strip set to punctuation that is unambiguously trailing prose punctuation, and add a regression test for an autolink that legitimately ends with one of the removed delimiters.

Suggested fix:

const TRAILING_CJK_PUNCT = /[，。、；：！？\u3000\uFF0C\uFF0E\uFF1B\uFF1A\uFF01\uFF1F]+$/

it('does not strip fullwidth delimiters that are part of the URL', () => {
    const tree = makeAutolink('https://example.com/路径）')
    transform(tree)

    expect(tree.children[0].children[1].url).toBe('https://example.com/路径）')
})

[github-actions]

Findings

• [Major] Valid autolinks ending with fullwidth closing delimiters are still truncated. TRAILING_CJK_PUNCT still includes characters like ）, 】, and 」, so a real URL such as https://example.com/路径） will be rewritten even when that last character is part of the path. Evidence web/src/lib/remark-strip-cjk-autolink.ts:16, web/src/lib/remark-strip-cjk-autolink.ts:41, missing regression coverage in web/src/lib/remark-strip-cjk-autolink.test.ts:86.
  Suggested fix:

  const TRAILING_CJK_PUNCT = /[，。、；：！？\u3000\uFF0E]+$/
  
  it('does not strip fullwidth closing delimiters that are part of the URL', () => {
      const tree = makeAutolink('https://example.com/路径）')
      transform(tree)
  
      expect(tree.children[0].children[1].url).toBe('https://example.com/路径）')
      expect(tree.children[0].children.length).toBe(2)
  })

Summary

• Review mode: follow-up after new commits
• 1 major issue remains on the current head.

Testing

• Not run (automation). Suggested addition: autolinks that legitimately end with ）, 】, or 」.

HAPI Bot

[github-actions]

[MAJOR] Valid autolinks ending with fullwidth closing delimiters are still truncated here. TRAILING_CJK_PUNCT still includes characters like ）, 】, and 」, so a real URL such as https://example.com/路径） will be rewritten even when that last character is part of the path.

Suggested fix:

const TRAILING_CJK_PUNCT = /[，。、；：！？\u3000\uFF0E]+$/

it('does not strip fullwidth closing delimiters that are part of the URL', () => {
    const tree = makeAutolink('https://example.com/路径）')
    transform(tree)

    expect(tree.children[0].children[1].url).toBe('https://example.com/路径）')
    expect(tree.children[0].children.length).toBe(2)
})