Sporadic and incorrect Annotation placement in certain PDFs in IE11

[henry-chris]

This may be more for your internal support, but it also seems to be Webviewer centric (possibly in service workers or something, also). Let me know if I should retry there.

Link to PDF exhibiting behavior: https://pdfmarkupblobstore.blob.core.windows.net/sample-container/link_2pages.pdf

Symptom: In IE 11, annotations loaded from external file with the demo.pdftron.com server do not load correctly.
Correct Loading in Chrome (Messed up the loading modal with CSS, sorry):

Incorrect Loading in IE11:

In IE11: Using PDFNetJS loads the same file/annots perfectly. Demo server does not.
In Chrome: All annots load perfectly. (PdfNetJS and demo.pdftron.com)

Repro Steps:

1. Use my PDF Url or download it.
2. Create annotations and save them in separate XFDF.
3. Load PDF with demo server/blackbox 'doctype'/ pdftronServer 'engineType'/setServerUrl(null)
4. Load XFDF after 'DocumentLoaded' event (here is how we do it):
   (window as any).docViewer.on('documentLoaded', this.loadXfdf.bind(this, store));
   Which ends up calling - annotationManager.importAnnotationsAsync(xfdfString, callback, options);

Notes:
Possibly clashing with the annots.xfdf call that seems to come with XOD download?
Possibly Scaling issue for this type of PDF?
Possibly from PDF origin? Pdf was made in Microstation, we did create a very simplePDF in Microstation and had similar issues. PDFs from AutoCad did not seem to have this issue.

[henry-chris]

So, as a follow-up. We started taking graphics out of the linked PDF. As we removed more and more graphics we found that the annotations loaded correctly a greater percentage of the time. Once we got down to a single Rectangle on one page, the amount of times that the annotations either loaded incorrectly or didn't display at all were very small. As we put more graphics back in the amount of times the annotations loaded incorrectly went up.

[justinjung04]

Thank you for very detailed explanations and observations. This seems like an issue with the core more than the UI, so it may take a bit longer to investigate. I'll pass this down to the core team, and let you know when the fix is available :)

[henry-chris]

Thanks, should I open an issue on your website with our Viewer license and link it to here or anything? If not, this sounds fine to me. Thanks for letting me know.

As another update, this behavior has started cropping in in Chrome and other browsers than IE11, now.

[mparizeau-pdftron]

We've determined that the issue occurs for this document when using the pdftronServer option and behind the scenes a XOD document is being used. We're currently investigating a fix.

[henry-chris]

This is also happening when using the pdfnetjs and 'Downloader' option to stream the actual pdf. I also get warnings from Core if I try to export() before the full PDF is loaded and the command XML then tries to save out the moved annotations with 'NaN' positions. Since some if our PDFs take 30-40 sec to fully download, it's happening a lot.

[henry-chris]

This is also happening when using the pdfnetjs and 'Downloader' option to stream the actual pdf. I also get warnings from Core if I try to export() before the full PDF is loaded and the command XML then tries to save out the moved annotations with 'NaN' positions. Since some if our PDFs take 30-40 sec to fully download, it's happening a lot.

For Example:

1. Using Chrome pdfnetjs/useDownloader. Loading a 156MB doc with some annots possibly embedded in the document.
2. We load our site and it begins to stream the .PDF using range streaming. This takes between 25-40 seconds to finish (partial content seems to be coming in at 512KB pieces).
3. We set the XFDF to make an XHR call when 'DocumentLoaded' triggers. This grabs the full XFDF, and we poll for the XFDF every 30 seconds (we can't use SSE or Push Events yet)
4. We initially see the Annotations load into incorrect positions (possibly loaded with no positions as some are in top left corner).
5. If the PDF is still loading and we try to update the XFDF with a new Annot, all annots on the current displayed page are sent in the CMD XML with 'NAN' positions (e.g. on a comment rect='NaN, NaN, NaN, NaN'). At this point our annotations are hosed.

You can see the Cmd XML being sent out while the PDF is still being streamed.

6. IF we wait and let the PDF Fully load, the same thing happens as the above. However, if we wait a longer time (say 30 seconds after the PDF fully downloads) it seems that we can then import our annotations successfully.

The PDF I am using to test: https://pdfmarkupblobstore.blob.core.windows.net/sample-container/ZierdtRdPhase4.pdf

Questions:

1. Is there a workaround that could be easily implemented...i was thinking of keeping a dictionary of annotations in memory and having a function that imports them when a page is displayed....not sure if that would work, though. I guess I'd need to way to grab them after de-serialization occurs but before they are loaded. Maybe make them unlistable at first.

2. Is this a separate bug from the PDFTron server one or are we just using your software incorrectly?

[henry-chris]

We've determined that the issue occurs for this document when using the pdftronServer option and behind the scenes a XOD document is being used. We're currently investigating a fix.

Any news on this? If I need to write a workaround for documents not streamed by the PdfTron Server I can. I just need to know which way to go.

[mparizeau-pdftron]

The downloader issue should be solved based on the support email thread. For the PDFTron server issue we're narrowing in on a fix for the issue and unfortunately it doesn't look like it can be easily worked around on your end right now. We'll keep you updated when we have more information.

[mparizeau-pdftron]

OK, we've been able to resolve the issue and you can use this updated build which contains the fix http://www.pdftron.com/ID-zJWLuhTffd3c/WebViewer/WebViewer_4.0.2.68289.zip. Thanks for the detailed bug reports and your patience!

@henry-chris When you've confirmed it's working for you I'll close this issue.

[henry-chris]

Thanks! That's great news. We really were wondering if we were misusing the software with the way we ended up importing annotations. Hopefully this wasn't a weird use case for you guys as I'd rather go the more conventional route if possible.

It may take me a couple days to verify this because I'm also verifying the downloader fix among other things.

I'll let you know as soon as I can.

[mparizeau-pdftron]

Normally we recommend you use the serverUrl option but it depends a bit on what you want to do with internal document annotations, so loading annotations on documentLoaded is intended to be supported. We're going to be updating our guides relatively soon to go through the different options available for annotation saving/loading to make things more clear about what is supported and expected.

It turned out in this case that there was a timing issue with getting the correct page transformation when a XOD file was being used by the server so it was possible to import annotations in document loaded before the correct transformation data was available and the annotations would be placed in the wrong location.

[henry-chris]

I sent an email saying that the 3 issues we were seeing are solved, including this one.

Now, we are seeing that our annotations are being sorted incorrectly in many cases since updating to the 4.0.2 version you gave us and it only happens when downloading via Range Requests.

I can't really show you very well as we have heavily modified the Notes Panel on our end, but our issues only happened after upgrading to 4.0.2. Basically while downloading the document, notes are sorted in one way, while we tell PDFTron to sort the other way by default. While downloading we see 'toggling' of the sorting methods in the notes panel where it's usually incorrect but sometimes toggles on and off.

Once the PDF is fully downloaded, the notes 'snap' into their correct sorted display and stay there for the rest of the session.

Since this may be related to the issue here, I'll leave it and only open a new one if instructed or we can verify it's an issue on your end (and not ours).

[justinjung04]

Just to clarify, are you using a custom sorting algorithm and the sorting algorithm is switching back and forth (between custom and default algorithms) as the annotations are being imported?

[henry-chris]

Yes, basically we added a reverse time sort. 'OrderByNewest' and 'OrderByOldest'. When using the 'Downloader' option with a PDF and no PdfTron Server we would see it sort as the default sort method until the document was fully downloaded. Then, another importAnnotationsAsync() would be triggered and it would sort the annotations the 'other' way.

In 4.0.1 we did not have this problem. It seems that you recently changed how you interact with the sortmap.

I'd have to get my co-worker to comment for more details, sorry. Basically we ended up changing the sorting algorithms in 'sortMap.js' and everything works fine now. Here's an example:

I'm not sure if he debugged into the 'getLatestActivityDate' function. Let me know if you want more details. It could very well be that it was caused by something we did.

[justinjung04]

We did reproduce the issue and also observed the change from your co-worker fixes the issue. One thing to note is that the getLatestActivityDate should return the date of a last activity related to that note, including replies. We'll create a separate issue on this.