Ist Reader quits without any message

[alexanderbjelos]

Hey folks,

when I want to open the incoming-message folder, the Xst Reader quits without any message. The message folder contains over 60.000 items. May be this could be the problem? Please give me a reply if you need more information.

Alex

[Dijji]

Hi Alex

Let me first confirm what I think you mean. You can open the outlook file with no issues (hence the screenshot), and you can open other folders within the file okay, but opening the particular folder with 60,000 messages crashes the program. Is that right?

It could be related to the number of messages. When you open a folder, the program reads some information about every message in it, which can consume a lot of memory. I have tested XstReader with a folder of 250,000 messages, which causes the program to use 480 MB of memory. It shouldn't matter what the content of the emails is, as at this point, it is only reading header material. Do you have memory limitations that might cause a problem? Are you running a 64-bit machine? In any case, it might be interesting to use Task Manager to monitor memory consumption while XstReader attempts to open the folder.

I'm also rather worried about exiting without a message, which is not supposed to happen. It implies that I've missed something somewhere, and I'd like to enlist your help digging around until we find out what, if you are willing.

Dijji

[Dijji]

Looking at the code, what you should see when you click on the folder is the text "Reading messages..." appear in the status bar at the bottom and the mouse change to an hourglass. Then the actual reading starts, and when it is complete, potentially a while later, the status text disappears, the mouse changes to a pointer, and the list of mails appears.

If the reading fails, a message box titled "Error reading messages" should pop up containing an error report.

What do you see, and at what point does the program terminate?

Dijji

[alexanderbjelos]

I've created for you a short video where you can see what happened.
Link

[Dijji]

Thank you for the video. That makes things very clear.

I have created a new release, Version 1.9.0.1, with some extra debugging traps (the source is up there if you want to see what I've done). Could you please try this version and see what you get? I'm hoping for some sort of message box. Be aware though, that sometimes you have to close the application to see it (assuming that I've managed to stop it closing itself).

Dijji

[flywire]

https://www.screentogif.com/

[alexanderbjelos]

Hello Dijji,

thank you for the debugging release. Here is the error message:

Unhandled exception
Out of bounds error attempting to map type

[Dijji]

Thank you, that is very revealing.

I have now been able to block up the leakage that left the exception unhandled in the first place, you should now get the intended "Error reading messages" status bar text.

I have also put in much richer error reporting in the indicated area, so you should also get a rather full message box containing a lot of detailed diagnostics.

I have updated the debug release to version 1.9.0.2, and would be very grateful if you could try it once more and tell me what you see.

One question. It looks as though the problem is occurring reading the message table for the folder, which may have some corruption at quite a deep level. Or it might equally be one of those 'less than' that should be 'less than or equal' bugs in my code. The new debug release should give me a clearer indication. However, the question is, what is the desired behaviour when I encounter a corruption error? Should I abort reading the folder, as I do now, or should I attempt to soldier on, skipping the damaged area, if there is one, to try and read at least some messages, albeit with the potential for some misreadings? I would be interested in your opinion.

Dijji

[flywire]

what is the desired behaviour when I encounter a corruption error?

Try to emulate Outlook.

It is important the original file is not inadvertently overwritten although it might be renamed. Regardless, a Recover as much as possible process is desirable even if it is an external utility.

[Dijji]

I can say that whatever I might do, it will be read-only. I will not ever change the original file, or even produce a copy containing changes.

The most I will do is to try to 'skip over' any damaged area in order to extract as much information as possible, and always let the user know that I have done so, with, if possible, an indication of what might be missing.

By the way, I know that this debugging process is rather slow and tedious, but I really don't want to see anybody's private data, so I have to resort to a sort of slow motion remote debug session. Please bear with me, it shouldn't take more than one more round to finish.

Dijji

[Dijji]

An additional tip: when the message box with the diagnostics comes up, if you put the focus on it, and press CTRL+C the contents will be copied to the clipboard so that you can paste them straight into this thread without going to the trouble of creating an image.

[alexanderbjelos]

Hello Dijji,

thank you very much for your help.
The XstReader is a very useful tool for me.
Here is the error message:

Error reading messages
XstReader.XstException: Out of bounds error attempting to map BBTENTRYUnicode4K from buffer length 4056 at offset 4032 bei XstReader.Map.MapType[T](Byte* buffer, Int32 buflen, Int32 offset)
bei XstReader.NDB.ReadBTPageUnicode4K(FileStream fs, UInt64 fileOffset, TreeIntermediate parent)
bei XstReader.NDB.ReadDeferredIndex(TreeIntermediate inter)
bei XstReader.BTree1.LookupTreeNode(TreeIntermediate parent, UInt64 key, Action1 readDeferred)
bei XstReader.BTree1.LookupTreeNode(TreeIntermediate parent, UInt64 key, Action1 readDeferred)
bei XstReader.BTree1.LookupTreeNode(TreeIntermediate parent, UInt64 key, Action1 readDeferred)
bei XstReader.BTree1.Lookup(UInt64 key, Action1 readDeferred)
bei XstReader.NDB.ReadSubNodeBtreeUnicode(FileStream fs, UInt64 subDataBid, TreeIntermediate parent)
bei XstReader.NDB.ReadSubNodeBtree(FileStream fs, UInt64 subDataBid)
bei XstReader.NDB.LookupNodeAndReadItsSubNodeBtree(FileStream fs, NID nid, BTree1& subNodeTree) bei XstReader.XstFile.<>c__DisplayClass18_1.<ReadMessages>b__1(Message m) bei System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()
bei System.Collections.Generic.List1..ctor(IEnumerable1 collection)
bei System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
bei XstReader.XstFile.ReadMessages(Folder f)
bei XstReader.MainWindow.<>c__DisplayClass11_0.<treeFolders_SelectedItemChanged>b__0()

My desired behaviour when a corruption error encounter is to skipping the damaged area and try to read at least some messages. Otherwise I would not be able to view any messages.

Alex

[flywire]

@alexanderbjelos Does that file open in Outlook? Otherwise try https://duckduckgo.com/?q=free+pst+recovery+software&ia=web

[Dijji]

Excellent! I think I know what's going on now. It is not a corrupt file, but a bug in my code.

I have applied a fix, and added more checks in case similar problems show up elsewhere. I've posted a new release, 1.9.0.3, for you to try to see if my diagnosis is accurate.

Please let me know what you find. And thanks again for your help with this. I would never otherwise have found it (or at least, think I have) in a million years.

Dijji

[Dijji]

If it's any encouragement, I believe that the latest test release should fix your problem.

Dijji

[alexanderbjelos]

The release 1.9.0.3 works very well. I have no issues to open the 64.000 mail items. Thank you for the great support.