Wrong conversion of German characters äöüß

[ai-arie]

I'm using the Python API to convert some German documents, and I narrowed down the problem in my conversion to MarkItDown.

md = MarkItDown()
result = md.convert(file_name)
markdown_text = result.text_content

Before the conversion the document in "file_name" contains: ...für künstliche Intelligenz und zur Änderung...

After the conversion markdown_text contains: ...fΟΦr kΟΦnstliche Intelligenz und zur Ο³nderung...

Any help with that?
Preferably, the solution should not change the output when converting English text, only German (potentially other languages? did not test...)

Thanks!

[Pindar777]

I had the same issue in terms of saving the converted text as .md and found the following solution:

...

with open(md_file, 'w', encoding="utf-8") as f:
     f.write(result.text_content)

...

[ai-arie]

thanks @Pindar777 - I'm saving as .txt - but that does not work.
Have not checked for .md yet, will try it out.

[Pindar777]

@ai-arie .md or .txt should behave in the same way. As a matter of fact I could not save any output without the encoding parameter.

[cboct]

Command-line users can enforce Unicode output through the flag -c utf-8 / --charset utf-8.
(I wonder if there is a reason why utf-8 is not the default encoding, but this is at least a working solution.)