mermaid extension erase all unicode in html output

[retsyo]

>>> import markdown
>>>
>>> md = markdown.Markdown( extensions=['md_mermaid'] )
>>> strMd = '# hello world'
>>> print(md.convert(strMd))
<h1>hello world</h1>
>>> strMd = 'a你好b'
>>> print(md.convert(strMd))
<p>ab</p>
>>> strMd = u'a你好b'
>>> print(md.convert(strMd))
<p>ab</p>
>>>

[orobardet]

Same here, even with accented characters. It even removes all non basic ascii characters from the whole markdow file (not only from mermaid code).
It seems to be a big regression in the commit 9f279f1 (I don't understand why this code was added). As it's the only difference between version 0.1.0 et 0.1.1, a workaround for now is to pin the 0.1.0 version of the package.

[gmat]

@oruelle why is it useful to remove all characters except acsii with strip_notprintable() ? This breaks unicode pages. Can you give us some cases where is useful ? Maybe strip_notprintable should be call in some if else conditions to be defined.

Thanks

[mEDI-S]

i have a lot problems with the current version too and have as fix only add strip_notprintable(line) in the re.mach funktions added and the raw lines not modificid, this help a lot to found the correct end and start from a code block

i remoed all other strip_notprintable()
and add only

m_start = MermaidRegex.match( strip_notprintable(line) )
m_end = re.match(r"^["+mermaid_sign+"]{3}[\ \t]*$", strip_notprintable(line) )

hope this help

[rayalan]

I'm running into a variant of this with the latest mindmap syntax, which relies heavily on the leading whitespace. The current line.strip() call wipes out the whitespace, turning

mindmap
  root
    topic
      subtopic

into

mindmap
root
topic
subtopic

which won't render.

As others before me have said, I'm not what problem we're trying to solve by stripping when we are inside of mermaid code -- why can't the mermaid code be sent as-is to the mermaid parser?

[retsyo]

Same here, even with accented characters. It even removes all non basic ascii characters from the whole markdow file (not only from mermaid code). It seems to be a big regression in the commit 9f279f1 (I don't understand why this code was added). As it's the only difference between version 0.1.0 et 0.1.1, a workaround for now is to pin the 0.1.0 version of the package.

This commit should be cancelled since it gives bad results