Failed to parse headers

[gigi81]

I'm using MimeKit 2.0.3
I'm getting this error trying to parse an email headers.
System.FormatException : Failed to parse headers.
at MimeKit.MimeParser.ParseHeaders(Byte* inbuf, CancellationToken cancellationToken) in C:\GitHub\MimeKit\MimeKit\MimeParser.cs:line 1413
at MimeKit.MimeParser.ParseHeaders(CancellationToken cancellationToken) in C:\GitHub\MimeKit\MimeKit\MimeParser.cs:line 1445

I tried to debug a little bit the issue and I narrowed it down to this bit of code in MimeParser.cs:

if (inptr == inend) {
    // we don't have enough input data
    int available = (int) (inend - start);

    inputIndex = (int) (start - inbuf);

    if (left == available) {
        state = MimeParserState.Error;
        headerIndex = 0;
        return false;
    }

    left = available;
    needInput = true;
    break;
}

If I change this line in the file:
const int BlockSize = 4096;
To
const int BlockSize = 4096 * 2;

The headers are parsed correctly and I don't get any exception.
I hope this is enough info but do please let me know if you need me to provide more info.
Unfortunately I cannot share the email itself.

[jstedfast]

I'm going to need a test case. You could just X-out the private info and I likely won't need the message body at all (just the headers).

In other words: replace abc@def.org with xxx@xxx.xxx (in other words, make sure that the words in your modified header are the same length).

[gigi81]

I will have a look and see if I can obfuscate the email and add a unit test specific for that.
Anyway, if you set the blocksize to 16 or 32, the TestJwzMbox unittest fails.
I think I found the problem and I have a fix.
Can you confirm if you are happy for me to add a blockSize (optional) parameter to the constructors of the MimeParser class? This way I can also provide a unit test to reproduce the issue.

[jstedfast]

I would prefer not to have a public API that did that, but if the idea is to just unit test it, you could add an internal .ctor that takes a blockSize. The unit tests can invoke internal APIs.

[gigi81]

I found the commit that introduced the bug: 801ac47
Not sure what you were trying to fix there but my fix was actually to revert this change.

[jstedfast]

Ok, I think I have an idea of what the problem is but if you could provide me with a test case that would still rock so that I could have a unit test for this and be sure that any fix I cook up actually works.

[jstedfast]

Thanks for digging into this btw, I've been pretty swamped :)

[gigi81]

No problem at all. I'm just waiting to get approval from my company to send a PR

[jstedfast]

Should be fixed now. Thanks for the PR with a way to replicate!

[gigi81]

It would be great if you can have a new nugget package version for this so I can put it to test. Thanks for having a look into this

[FSGRSolutions]

Experiencing the same issue, so following this.

[jstedfast]

I'm currently away on a business trip of back-to-back meetings (w/ only my macbook) so can';t make/publish any nugets until at the earliest this weekend or early next week. I will do so as soon as I can - this bug is serious enough that it's worth making a new release on its own.

[FSGRSolutions]

You da man sir!

[jstedfast]

I've just published MimeKit 2.0.4 to nuget.org (might take up to an hour before it shows up)