No email content displayed

[sketchdes1gns]

No content in emails at all

they poll as expected and I will get read/unread counts along with the usual titles and headers with the two line preview (as set in the options)
but when openning the email to read - there is nothing at all present.

Checked this with basic text emails, with/without images, with/without attachments etc but nothing shows.

This has only just started to happen recently and the play store reports that I am using a "test version" so am unable to write a review.
Up until this started happening I've always used this app and have got others to use it instead of their normal applications.

The emails are fine using my desktop pc and through other apps but k9 just isnt having any of it, lol

Also tried to bump up the max fetch size to see if that was the issue and also cleared cache out as suggested by a few others, all to no avail.

running latest updated version on android version 5.1.1 samsung s5 trying to access my googlemail account

[philipwhiuk]

Same issue for an e-mail I got today.

I got a stack trace when I hooked up ADB. I've created a gist with the email and stack trace. https://gist.github.com/philipwhiuk/43f98fff2260c67f5f68decd6906143c

Not possible to know whether it's the same cause.

We could choose handle errors in a way that allows us to get the reason why the message wasn't parsed (e.g. by adding a message to K9mail-errors folder) or we could just fix it for the cases we get stack traces for.

[Valodim]

at a first glance, that email seems to violate the unique boundary property

[sketchdes1gns]

it happens with nigh on every single email I get now though with a certain amount of annoyance and irony the email I got telling me I had a reply here seems to open just fine.
Is there any info I could provide to help me get a bit further in the process of getting it fixed as its a massive inconvenience not being able to read emails on my phone at the minute

[Valodim]

It would be very helpful if you could provide a debug log, because without that we are flying blind here. The relevant bits will look something like the stack trace @philipwhiuk has in the gist he linked above

On a related note, if you use K-9 productively, it might not be a wise decision to join the tester group. It's cool if you help us figure out bugs so they never make it into an actual release, but if it gets to a point where it's detrimental to your experience you might want to reconsider (see here)

[sketchdes1gns]

to be completely honest, I hadnt even realised I had a test version of k9 installed and not sure how/why I installed it either (I've used k9 for ages now and certain I hadnt installed anything but the regular playstore version) - it has updated itself recently and was looking to roll it back to a useable state but thought I'd check it out first and had never experienced problems up until recently.
I can certainly post up a debug log, is there anything in particular you would suggest I should do before the log to generate a desired outcome such as checking for new mail, sending myself a test email, openning mail that does "open" etc?

[Valodim]

just capture a log right after you opened a mail which shows up blank. hopefully your log contains a stacktrace of the cause then.

another thought: does this happen with new mail as well as old? or do newly loaded messages show up correctly perhaps?

[sketchdes1gns]

does it with new and old and I've just gone to email myself the log screenshots from catlog (was having issues with logcat and I have root so tried catlog) and it threw back an error there as well, lol,
This is the error from trying to send an email to myself (I copied to clipboard text using airdroid and just dumped it into a notepad file, there wasnt any personal info there as far as I could see quickly)...

K9error.txt

this was about the best I could manage to get from catlog (for some reason I couldnt get anything out in text format but I've put screenshots together - hope thats ok) the 'sketch' part is only the name for that particular inbox so didnt bother hiding that and the only thing I've covered in it was the accountuuid, I wasnt sure if that was specific to my personal account or not so figured I would cover it to be safe.
no offence by the way, sorry

I oppened two emails before that log - one that didnt show content (from epic about beta access) and one that did (the notification about a reply to this post)
both obviously were proper emails, its pure random choice that the epic one contains images - even other emails that dont contain images wont load, these were just the first two emails

hope this is a bit more helpful

[philipwhiuk]

So that issue is from #1164 I think it should be fixed in the next beta?

[sketchdes1gns]

Doesn't only happen with "large" emails though bit for now I have had to begrudgingly revert back to the "normal" play store version of the app as I rely on access to my emails too much

[Valodim]

Regarding the non-unique boundary issue, the problem arises in LocalFolder: When a message part is saved, its boundary/preamble/postamble fields are only filled if the body is actually a Multipart, which may not be the case regardless of declared mimetype of the part. When the message is recalled though, a MimeMultipart body is always constructed if the mimeType is multipart, and that fails if the boundary field is not set.

This error is triggered by non-unique boundaries: A message such as multipart/mixed(multipart/alternative(text/plain, text/html)) with non-unique boundaries might get parsed by an imap server as multipart/mixed(multipart/alternative,text/plain,text/html) which results in a multipart/alternative MimeBodyPart object with a null body.

Making sure that all MimeBodyPart objects always has a MimeMultipart body seems difficult, I would propose adding a check if the mimeType is multipart/* to missingPartToContentValues, and add a generated a boundary if that's the case.

I gave that fix a shot, works for me. PR coming up.

[Valodim]

on current master, email and stacktrace:

From: "test account" <test@example.org>
To: test@mugenguild.com
Message-Id: <test3-E2b0uAD@sifjsdlf>
Subject: second and empty content-type header
Content-Type: 
Content-Type: multipart/related; type="text/html"; boundary="__part_M1HhxI3trB/gNWWyh03Y5w95"
Date: Fri, 15 Apr 2016 13:00:30 +0200

Hello!

java.lang.IllegalArgumentException: boundary can't be null
at com.fsck.k9.mail.internet.MimeMultipart.<init>(MimeMultipart.java:28)
at com.fsck.k9.mail.internet.MimeMessage$MimeMessageBuilder.startMultipart(MimeMessage.java:529)
at org.apache.james.mime4j.parser.MimeStreamParser.parse(MimeStreamParser.java:171)
at com.fsck.k9.mail.internet.MimeMessage.parse(MimeMessage.java:113)
at com.fsck.k9.mail.internet.MimeMessage.parse(MimeMessage.java:83)
at com.fsck.k9.mail.store.imap.ImapFolder.fetch(ImapFolder.java:760)
at com.fsck.k9.controller.MessagingController.fetchUnsyncedMessages(MessagingController.java:1328)
at com.fsck.k9.controller.MessagingController.downloadMessages(MessagingController.java:1176)
at com.fsck.k9.controller.MessagingController.synchronizeMailboxSynchronous(MessagingController.java:966)
at com.fsck.k9.controller.MessagingController$8.run(MessagingController.java:749)
at com.fsck.k9.controller.MessagingController.run(MessagingController.java:250)
at java.lang.Thread.run(Thread.java:818)

For an email like this (note the duplicate content-type header), the mime parser gives us a startMultipart() callback, but the content-type header we parse does not have a boundary.

The standards violation we need to deal with here is lack of a boundary parameter. We can't do much about a malformed mail like this I guess, but at least we shouldn't crash.

More generally at the moment, MimeMultipart crashes if it gets handed a null boundary parameter, so the handling must happen at every calling site. This is currently handled in none of the three calling sites (mimeparser, decryptstreamparser, loadpart), so we should probably find a common strategy for this.

We can either:

• Drop the "must have boundary" constraint of MimeMultipart. Not a good idea since it shifts the problem to writeTo()
• Weaken the constraint and simply generate a boundary if none exists
• Ensure that every calling site makes the check and generates a boundary or otherwise handles the situation if a malformed message comes in. This is what I do partially in PR ensure that multipart/* parts always have a non-null boundary field #1381, ensuring a part in the database always has a boundary thus solving the problem for 'load message'. Obviously that isn't sufficient and this needs to be dealt with in at least two other places (mimeparser, decryptstreamparser)

[cketti]

We should make sure that our code comes to the same conclusion as the MIME4J parser when it comes to the Content-Type header. That way there's no disconnect between interpretation of the headers and message parts in our message structure.

[Valodim]

That's not a call we can make when we get the structure of an email from the imap server, so there is at least one way we can get parts that isn't mime4j-dependent

[Valodim]

I looked at this in some more detail, please disregardmy previous statement.

So, the main handling for this is in org.apache.james.mime4j.stream.FallbackBodyDescriptorParser which during parsing is handed all field values and remembers the relevant parts of content-* fields. A content-type with value multipart/* is only regarded as valid if it has a boundary value and otherwise disregarded, if more than one content-type header occurs the last one is preferred.

What we do in getContentType() is return the first header for content-type. This is a particular problem in MimeMessageBuilder.startMultipart() which is called to parse ImapMessage content from ImapFolder and similar places. Now in this particular position, we could simply use the values from the BodyDescriptor parameter we get instead of the ones from the message, and that would probably even be the more correct solution there. However it still leaves the getContentType() problem unsolved.

The simple solution would be to pass all content-type headers to FallbackBodyDescriptorParser, and use the content-type value from there. BUT unfortunately, that class is package-local, which makes this a real pain...

[cketti]

Should be fixed in master.