I received a message from a mailing list and I believe that it is displayed incorrectly in mu4e-view. This is the start of the message body, copied from mu4e-raw-view:
I am wondering about the consequences of using non-orthogonal contrasts in =
I come from psychological experiments in which we often code factorial effe=
And below you see what is displayed by mu4e-view. Everything is on one line and each newline in the raw text is replaced by a period:
Dear List,..I am wondering about the consequences of using non-orthogonal contrasts in an LMM..I come from psychological experiments in which we often code factorial effe
Any ideas? I actually receive more and more of these mails. If you could offer some directions for debugging, I could try to solve it myself. I had a look at mu4e-view.el but haven't found the culprit yet. Thanks.
Does this go through some html-renderer? Can you reproduce it with mu view?
Thanks for the response, Dirk-Jan. I don't think it goes through an html renderer. (When I set mu4e-html2text-command to a dummy function, I still get the same display of the message.) If I display the email using mu view on the command line, I also get the wrong display with all text on one line and newlines replaced by periods. BTW, I'm running the current development version of Emacs and updated mu/mu4e a few days ago.
My locale is en_US.UTF-8.
Ok, I found it. In mu_msg_mime_part_to_string, the mime part is converted to utf-8 (line 501, in mu-msg-file.c):
buffer = convert_to_utf8 (part, buffer);
This is where the problem is introduced. convert_to_utf8 doesn't recognize the charset of the part and "ugly * hack: replace all non-ascii chars with '.'" What's strange is that the newlines should not be replaced as they are in the ASCII range. The issue is that mu_str_asciify_in_place does more than it says: it does not only replace non-ascii characters but also characters below the whitespace and that seems wrong. The relevant part is this:
if (!isascii(*c) || *c < ' ')
*c = '.';
Is it enough to instead write
*c = '.';
or would this introduce other problems?
* lib: Don't replace ascii characters < ' ' with '.' in mu_str_asciif…
Fix for issue #351.
There may be another problem involved: the mime part is plain ascii and should therefore g_utf8_validate. Thus, the replacement of nonascii characters shouldn't be triggered at all.
I've changed the check slightly (git), does that fix things for you?
(I suspect there must be some non-ascii char somewhere in the decoded message)
That solves my problem. Thanks!
I will open a new issue if I find out more about the other problem (which I think is real because there are no non-ascii characters in the message.)