MimeMessage::ToString() throwing System.NullReferenceException

[LeeBear35]

I have a message that when loaded has a Text part, but the BodyParts Enumeration is null, so the ToString call throws and error.

[jstedfast]

BodyParts can never return null. Can you email me the message that is causing this so I can see for myself what is really going on?

[LeeBear35]

Here is one of the messages, it is a draft, converted from an MSG file, I have enclosed the code used to create the MimeMessage from the MSG.

        Public Shared Function ConvertToMimeKit(ByRef message As Independentsoft.Msg.Message) As MimeKit.MimeMessage
            Dim mkMessage As MimeKit.MimeMessage = Nothing

                If message.TransportMessageHeaders IsNot Nothing Then
                    Dim transportMessageHeaders As String = message.TransportMessageHeaders

                    If message.TransportMessageHeaders.StartsWith("Microsoft Mail Internet Headers") Then
                        transportMessageHeaders = message.TransportMessageHeaders.Substring(message.TransportMessageHeaders.IndexOf(vbCrLf) + 2)
                    End If

                    If Not transportMessageHeaders.EndsWith(vbCr & vbLf & vbCr & vbLf) Then
                        transportMessageHeaders += vbCr & vbLf & vbCr & vbLf
                    End If

                    Dim codePageEncoding As System.Text.Encoding = GetEncoding(message.InternetCodePage)

                    If Not codePageEncoding.IsSingleByte Then
                        codePageEncoding = Encoding.UTF8
                    End If

                    Dim transportMessageHeadersBuffer As Byte() = codePageEncoding.GetBytes(transportMessageHeaders)
                    Using s As Stream = New MemoryStream(codePageEncoding.GetBytes(transportMessageHeaders))
                        mkMessage = MimeKit.MimeMessage.Load(s)
                    End Using
                Else
                    mkMessage = New MimeKit.MimeMessage()
                End If

                Dim mkMailBox As MimeKit.MailboxAddress

                For Each recipient As Independentsoft.Msg.Recipient In message.Recipients
                    If String.IsNullOrWhiteSpace(recipient.DisplayName) Then
                        If String.IsNullOrWhiteSpace(recipient.SmtpAddress) Then
                            mkMailBox = New MimeKit.MailboxAddress(recipient.SmtpAddress, recipient.SmtpAddress)
                        Else
                            mkMailBox = New MimeKit.MailboxAddress(recipient.EmailAddress, recipient.EmailAddress)
                        End If
                    Else
                        If String.IsNullOrWhiteSpace(recipient.SmtpAddress) Then
                            mkMailBox = New MimeKit.MailboxAddress(recipient.DisplayName, recipient.EmailAddress)
                        Else
                            mkMailBox = New MimeKit.MailboxAddress(recipient.DisplayName, recipient.SmtpAddress)
                        End If
                    End If
                    If mkMailBox IsNot Nothing AndAlso mkMailBox.Address IsNot Nothing Then
                        Select Case recipient.RecipientType
                            Case RecipientType.To
                                mkMessage.To.Add(mkMailBox)
                            Case RecipientType.Cc
                                mkMessage.Cc.Add(mkMailBox)
                            Case RecipientType.Bcc
                                mkMessage.Bcc.Add(mkMailBox)
                            Case RecipientType.P1
                            Case RecipientType.None
                            Case Else
                        End Select
                    End If
                Next

                If mkMessage.From Is Nothing Then
                    If String.IsNullOrWhiteSpace(message.SenderName) Then
                        If message.InternetAccountName IsNot Nothing AndAlso message.InternetAccountName.Contains("@") Then
                            mkMailBox = New MimeKit.MailboxAddress(message.SenderEmailAddress, message.InternetAccountName)
                        Else
                            mkMailBox = New MimeKit.MailboxAddress(message.SenderEmailAddress, message.SenderEmailAddress)
                        End If
                    Else
                        If message.InternetAccountName IsNot Nothing AndAlso message.InternetAccountName.Contains("@") Then
                            mkMailBox = New MimeKit.MailboxAddress(message.SenderName, message.InternetAccountName)
                        Else
                            mkMailBox = New MimeKit.MailboxAddress(message.SenderName, message.SenderEmailAddress)
                        End If
                    End If
                    mkMessage.From.Add(mkMailBox)
                End If

                If message.Subject IsNot Nothing Then mkMessage.Subject = message.Subject
                mkMessage.[Date] = message.ClientSubmitTime

                If mkMessage.MessageId Is Nothing Then
                    Try ' Some ID's do not comply
                        mkMessage.MessageId = message.InternetMessageId
                    Catch
                    End Try
                End If

                Dim BodyBuild As New MimeKit.BodyBuilder()

                If message.Body IsNot Nothing Then BodyBuild.TextBody = message.Body
                If message.BodyHtml IsNot Nothing Then BodyBuild.HtmlBody = message.BodyHtmlText

                For Each Attachment As Independentsoft.Msg.Attachment In message.Attachments
                    Dim mkEntity As New MimeKit.MimePart

                    If Attachment.Data IsNot Nothing Then
                        If Attachment.MimeTag IsNot Nothing Then
                            MimeKit.ContentType.TryParse(Attachment.MimeTag, mkEntity.ContentType)
                        End If

                        Try ' Some ID's do not comply
                            mkEntity.ContentId = Attachment.ContentId
                        Catch
                        End Try

                        Uri.TryCreate(Attachment.ContentLocation, UriKind.RelativeOrAbsolute, mkEntity.ContentLocation)

                        If Not String.IsNullOrWhiteSpace(Attachment.LongFileName) Then
                            mkEntity.FileName = Attachment.LongFileName
                        ElseIf Not String.IsNullOrWhiteSpace(Attachment.DisplayName) Then
                            mkEntity.FileName = Attachment.DisplayName
                        ElseIf Not String.IsNullOrWhiteSpace(Attachment.FileName) Then
                            mkEntity.FileName = Attachment.FileName
                        End If

                        mkEntity.ContentObject = New MimeKit.ContentObject(Attachment.GetStream(), MimeKit.ContentEncoding.Default)
                        mkEntity.ContentTransferEncoding = MimeKit.ContentEncoding.Base64

                        If mkEntity.IsAttachment Then
                            BodyBuild.Attachments.Add(mkEntity)
                        Else
                            BodyBuild.LinkedResources.Add(mkEntity)
                        End If
                    End If
                Next

                mkMessage.Body = BodyBuild.ToMessageBody

            Return mkMessage
        End Function

From: Jeffrey Stedfast [mailto:notifications@github.com]
Sent: Monday, May 04, 2015 02:21 PM
To: jstedfast/MimeKit
Cc: Carpenter, Lee
Subject: Re: [MimeKit] MimeMessage::ToString() throwing System.NullReferenceException (#135)

BodyParts can never return null. Can you email me the message that is causing this so I can see for myself what is really going on?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/135#issuecomment-98823408.

[jstedfast]

I do all of my development on a Mac, so don't have access to a VB.NET compiler... could you just send me the results of mkMessage.WriteTo ("message.txt") ?

[LeeBear35]

I have attached it and the text is below as well.

From:
To:
Date: Mon, 01 Jan 0001 00:00:00 -0600
Subject:
Message-Id: SGOEMW3WZUT4.Y0Y73SL1JEB6@ATL01L21037
MIME-Version: 1.0
Content-Type: text/plain

From: Jeffrey Stedfast [mailto:notifications@github.com]
Sent: Monday, May 04, 2015 03:01 PM
To: jstedfast/MimeKit
Cc: Carpenter, Lee
Subject: Re: [MimeKit] MimeMessage::ToString() throwing System.NullReferenceException (#135)

I do all of my development on a Mac, so don't have access to a VB.NET compiler... could you just send me the results of mkMessage.WriteTo ("message.txt") ?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/135#issuecomment-98833804.

[jstedfast]

It doesn't throw an exception for me. Which version of MimeKit are you using?

[LeeBear35]

I am running 4.0.30319. I think it might be an issue with one of the internal Try converts getting trapped by the debugger. Microsoft has a way of hooking into and catching errors before the bubble up to the registered error handler making it look like an unhandled exception.

Since I am processing a PST file with a few thousand messages I will reprocess it and see if I can get all the details. Oh and here is the VB code as C#.

public static MimeKit.MimeMessage ConvertToMimeKit(ref Independentsoft.Msg.Message message)
{
MimeKit.MimeMessage mkMessage = null;

                            if (message.TransportMessageHeaders != null) {
                                            string transportMessageHeaders = message.TransportMessageHeaders;

                                            if (message.TransportMessageHeaders.StartsWith("Microsoft Mail Internet Headers")) {
                                                            transportMessageHeaders = message.TransportMessageHeaders.Substring(message.TransportMessageHeaders.IndexOf(Constants.vbCrLf) + 2);
                                            }

                                            if (!transportMessageHeaders.EndsWith(Constants.vbCr + Constants.vbLf + Constants.vbCr + Constants.vbLf)) {
                                                            transportMessageHeaders += Constants.vbCr + Constants.vbLf + Constants.vbCr + Constants.vbLf;
                                            }

                                            System.Text.Encoding codePageEncoding = GetEncoding(message.InternetCodePage);

                                            if (!codePageEncoding.IsSingleByte) {
                                                            codePageEncoding = Encoding.UTF8;
                                            }

                                            byte[] transportMessageHeadersBuffer = codePageEncoding.GetBytes(transportMessageHeaders);
                                            using (Stream s = new MemoryStream(codePageEncoding.GetBytes(transportMessageHeaders))) {
                                                            mkMessage = MimeKit.MimeMessage.Load(s);
                                            }
                            } else {
                                            mkMessage = new MimeKit.MimeMessage();
                            }

                            MimeKit.MailboxAddress mkMailBox = default(MimeKit.MailboxAddress);

                            foreach (Independentsoft.Msg.Recipient recipient in message.Recipients) {
                                            if (string.IsNullOrWhiteSpace(recipient.DisplayName)) {
                                                            if (string.IsNullOrWhiteSpace(recipient.SmtpAddress)) {
                                                                            mkMailBox = new MimeKit.MailboxAddress(recipient.SmtpAddress, recipient.SmtpAddress);
                                                            } else {
                                                                            mkMailBox = new MimeKit.MailboxAddress(recipient.EmailAddress, recipient.EmailAddress);
                                                            }
                                            } else {
                                                            if (string.IsNullOrWhiteSpace(recipient.SmtpAddress)) {
                                                                            mkMailBox = new MimeKit.MailboxAddress(recipient.DisplayName, recipient.EmailAddress);
                                                            } else {
                                                                            mkMailBox = new MimeKit.MailboxAddress(recipient.DisplayName, recipient.SmtpAddress);
                                                            }
                                            }
                                            if (mkMailBox != null && mkMailBox.Address != null) {
                                                            switch (recipient.RecipientType) {
                                                                            case RecipientType.To:
                                                                                            mkMessage.To.Add(mkMailBox);
                                                                                            break;
                                                                            case RecipientType.Cc:
                                                                                            mkMessage.Cc.Add(mkMailBox);
                                                                                            break;
                                                                            case RecipientType.Bcc:
                                                                                            mkMessage.Bcc.Add(mkMailBox);
                                                                                            break;
                                                                            case RecipientType.P1:
                                                                                            break;
                                                                            case RecipientType.None:
                                                                                            break;
                                                                            default:
                                                                                            break;
                                                            }
                                            }
                            }

                            if (mkMessage.From == null) {
                                            if (string.IsNullOrWhiteSpace(message.SenderName)) {
                                                            if (message.InternetAccountName != null && message.InternetAccountName.Contains("@")) {
                                                                            mkMailBox = new MimeKit.MailboxAddress(message.SenderEmailAddress, message.InternetAccountName);
                                                            } else {
                                                                            mkMailBox = new MimeKit.MailboxAddress(message.SenderEmailAddress, message.SenderEmailAddress);
                                                            }
                                            } else {
                                                            if (message.InternetAccountName != null && message.InternetAccountName.Contains("@")) {
                                                                            mkMailBox = new MimeKit.MailboxAddress(message.SenderName, message.InternetAccountName);
                                                            } else {
                                                                            mkMailBox = new MimeKit.MailboxAddress(message.SenderName, message.SenderEmailAddress);
                                                            }
                                            }
                                            mkMessage.From.Add(mkMailBox);
                            }

                            if (message.Subject != null)
                                            mkMessage.Subject = message.Subject;
                            mkMessage.Date = message.ClientSubmitTime;

                            if (mkMessage.MessageId == null) {
                                            // Some ID's do not comply
                                            try {
                                                            mkMessage.MessageId = message.InternetMessageId;
                                            } catch {
                                            }
                            }


                            MimeKit.BodyBuilder BodyBuild = new MimeKit.BodyBuilder();

                            if (message.Body != null)
                                            BodyBuild.TextBody = message.Body;
                            else
                                            BodyBuild.TextBody = string.Empty;
                            if (message.BodyHtml != null)
                                            BodyBuild.HtmlBody = message.BodyHtmlText;
                            else
                                            BodyBuild.HtmlBody = string.Empty;

                            foreach (Independentsoft.Msg.Attachment Attachment in message.Attachments) {
                                            MimeKit.MimePart mkEntity = new MimeKit.MimePart();

                                            if (Attachment.Data != null) {
                                                            if (Attachment.MimeTag != null) {
                                                                            MimeKit.ContentType.TryParse(Attachment.MimeTag, mkEntity.ContentType);
                                                            }

                                                            // Some ID's do not comply
                                                            try {
                                                                            mkEntity.ContentId = Attachment.ContentId;
                                                            } catch {
                                                            }

                                                            Uri.TryCreate(Attachment.ContentLocation, UriKind.RelativeOrAbsolute, mkEntity.ContentLocation);

                                                            if (!string.IsNullOrWhiteSpace(Attachment.LongFileName)) {
                                                                            mkEntity.FileName = Attachment.LongFileName;
                                                            } else if (!string.IsNullOrWhiteSpace(Attachment.DisplayName)) {
                                                                            mkEntity.FileName = Attachment.DisplayName;
                                                            } else if (!string.IsNullOrWhiteSpace(Attachment.FileName)) {
                                                                            mkEntity.FileName = Attachment.FileName;
                                                            }

                                                            mkEntity.ContentObject = new MimeKit.ContentObject(Attachment.GetStream(), MimeKit.ContentEncoding.Default);
                                                            mkEntity.ContentTransferEncoding = MimeKit.ContentEncoding.Base64;

                                                            if (mkEntity.IsAttachment) {
                                                                            BodyBuild.Attachments.Add(mkEntity);
                                                            } else {
                                                                            BodyBuild.LinkedResources.Add(mkEntity);
                                                            }
                                            }
                            }

                            mkMessage.Body = BodyBuild.ToMessageBody;


            return mkMessage;

}

From: Jeffrey Stedfast [mailto:notifications@github.com]
Sent: Monday, May 04, 2015 03:19 PM
To: jstedfast/MimeKit
Cc: Carpenter, Lee
Subject: Re: [MimeKit] MimeMessage::ToString() throwing System.NullReferenceException (#135)

It doesn't throw an exception for me. Which version of MimeKit are you using?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/135#issuecomment-98837642.

[jstedfast]

There is no MimeKit version 4.0.30319. The latest is 1.0.13.

[LeeBear35]

Sorry I was looking at the runtime version under the properties, the package is reporting 1.0.13.0:

[cid:image001.png@01D08684.01AC79B0]

From: Jeffrey Stedfast [mailto:notifications@github.com]
Sent: Monday, May 04, 2015 03:45 PM
To: jstedfast/MimeKit
Cc: Carpenter, Lee
Subject: Re: [MimeKit] MimeMessage::ToString() throwing System.NullReferenceException (#135)

There is no MimeKit version 4.0.30319. The latest is 1.0.13.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/135#issuecomment-98845330.

[jstedfast]

I'm going to need a test case that I can compile and run locally that reproduces this problem and/or a full stack trace from the exception you are getting because I cannot figure out how it could possibly be throwing a NullReferenceException without more information :-(

[LeeBear35]

I think I have one…the message has a plain text component, but not HTML, so the BodyBuilder HTMLBody is getting set to String.Empty. The MSG loads all the parts, and then on the call to ToString() a “Buffer cannot be null” is thrown.

? mkMessage.ToString()
{"Buffer cannot be null.
Parameter name: buffer"}
_className: Nothing
_COMPlusExceptionCode: -532462766
_data: {System.Collections.ListDictionaryInternal}
_dynamicMethods: Nothing
_exceptionMethod: Nothing
_exceptionMethodString: Nothing
_helpURL: Nothing
_HResult: -2147467261
_innerException: Nothing
_ipForWatsonBuckets: 8791552849748
_message: "Buffer cannot be null."
_remoteStackIndex: 0
_remoteStackTraceString: Nothing
_safeSerializationManager: {System.Runtime.Serialization.SafeSerializationManager}
_source: Nothing
_stackTrace: {SByte()}
_stackTraceString: Nothing
_watsonBuckets: Nothing
_xcode: -532462766
_xptrs: 0
Data: {System.Collections.ListDictionaryInternal}
HelpLink: Nothing
HResult: -2147467261
InnerException: Nothing
IPForWatsonBuckets: 8791552849748
IsTransient: False
m_paramName: "buffer"
Message: "Buffer cannot be null. Parameter name: buffer"
ParamName: "buffer"
RemoteStackTrace: Nothing
s_EDILock: {Object}
Source: "mscorlib"
StackTrace: " at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count) at MimeKit.IO.FilteredStream.Write(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) at MimeKit.IO.FilteredStream.Write(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) at MimeKit.ContentObject.WriteTo(Stream stream, CancellationToken cancellationToken) at MimeKit.ContentObject.DecodeTo(Stream stream, CancellationToken cancellationToken) at MimeKit.MimePart.WriteTo(FormatOptions options, Stream stream, CancellationToken cancellationToken) at MimeKit.Multipart.WriteTo(FormatOptions options, Stream stream, CancellationToken cancellationToken) at MimeKit.MimeMessage.WriteTo(FormatOptions options, Stream stream, CancellationToken cancellationToken) at MimeKit.MimeMessage.ToString()"
TargetSite: {Void Write(Byte[], Int32, Int32)}
WatsonBuckets: Nothing

The WriteTo posts the same error, but handles it, here is the file out, I also used the IndependentSoft library to convert the MSG file to EML.

It looks like one of the attachments is having a problem, because the IndependentSoft library generated the email with no attachments.

From: Jeffrey Stedfast [mailto:notifications@github.com]
Sent: Monday, May 04, 2015 04:28 PM
To: jstedfast/MimeKit
Cc: Carpenter, Lee
Subject: Re: [MimeKit] MimeMessage::ToString() throwing System.NullReferenceException (#135)

I'm going to need a test case that I can compile and run locally that reproduces this problem and/or a full stack trace from the exception you are getting because I cannot figure out how it could possibly be throwing a NullReferenceException without more information :-(

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/135#issuecomment-98853782.

[jstedfast]

Should be fixed now.

[LeeBear35]

Great, I will test if first thing in the morning…Hey, just to let you know, we process millions of emails internally to audit, and there is a 300% increase in EML ingestion, and 200% increase on MSG, PST and NSF ingestion. Looks like I need to dig in a little more and fix those other libraries. MSG and PST are both using IndependentSoft.

Thank you again.

Leland Carpenter ♦ Sr. Software Engineer ♦ PRGX USA, Inc.
4904 Hickory Way ♦ Johnsburg, IL 60051-8967
O: 815.307.7634 ♦ Lee.Carpenter@prgx.commailto:Lee.Carpenter@prgx.com
[cid:image001.jpg@01D08691.4C5875B0]

From: Jeffrey Stedfast [mailto:notifications@github.com]
Sent: Monday, May 04, 2015 05:12 PM
To: jstedfast/MimeKit
Cc: Carpenter, Lee
Subject: Re: [MimeKit] MimeMessage::ToString() throwing System.NullReferenceException (#135)

Should be fixed now.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/135#issuecomment-98865476.

[LeeBear35]

Is there a new nuget release, or would I need to download and build everything?

[jstedfast]

There's no nuget release yet, so you'll have to grab the source from github and build it (make sure to follow the README which explains what you'll need to get)

[jstedfast]

Just released MimeKit 1.0.14 with this fix