Support annotations without appearance streams

[timvandermeij]

Most annotations, even unsupported ones, render just fine in PDF.js because of their appearance stream. However, there exist PDF files with annotations that do not have an appearance stream (even though this is a deprecated practice). In the latter case, PDF.js displays nothing. We need to have a fallback to a default appearance stream in that case, or create an appearance stream ourselves using the data present for the annotation (such as quad points and color). Below is a list of known PDF files that contain annotations without appearance streams, for which the types are listed.

• http://pdfkit.org/docs/guide.pdf (Underline, Highlight and StrikeOut)
• http://www.plpdf.com/examples/PLPDF-SDK/Simple_Examples/37_Markup_annotation/markup.pdf (Underline, Highlight and StrikeOut)
• https://drive.google.com/file/d/0B6YhVF9p-y2AU0RlYjRKakFQRjQ/view?usp=sharing (Highlight, from Highlights created by qpdfview is not rendered #6382)
• https://drive.google.com/open?id=0B6YhVF9p-y2Ad3VsWDRHRkRxY2c (FreeText, from Text annotations not rendered  #6246)
• https://drive.google.com/file/d/0B-_5BgJGB33Ea2dQZnpUY2FvYkk/view (FreeText, from Issue about rendering PDF annotation/comment #6564)
• http://mirrors.rit.edu/CTAN/macros/latex/contrib/pdfcomment/doc/example.pdf (practically all annotation types, from Added text to PDF with Adobe typewriter tool, but not showing in FF #1973)

[THausherr]

new PDF:
http://www.pdfill.com/example/pdf_commenting_new.pdf

The PLPDF file is now at
http://www.plpdf.com//examples//PLPDF/plsql/BasicResults/MarkupAnnotation.pdf

[fancycarp]

Waiting for this feature..^^

[THausherr]

This one has diagonal annotations. A strategy that assumes them to be horizontal will result in weird effects (try showing the file in chrome in the built-in viewer or on an Android phone with the google viewer)
https://bugs.ghostscript.com/show_bug.cgi?id=693664

[burtonator]

This is still a pretty serious issue. Apparently poppler, one of the main PDF libraries used by Okular, etc, creates annotations without appearance streams.

When loading them into PDF.js its impossible to see the annotations.

[THausherr]

Somebody should also create an issue in poppler. The /AP is required in PDF 2.0 except for Popup, Projection or Link.

[edsontrick]

I need to add image with drag n drop. Possible solutions?

[timvandermeij]

@edsontrick The question is not related to this particular issue. Please post it on IRC instead of on an unrelated issue.

[MzunguWaNjombe]

Somebody should also create an issue in poppler. The /AP is required in PDF 2.0 except for Popup, Projection or Link.

The corresponding issue for Poppler seems to be this one:
https://gitlab.freedesktop.org/poppler/poppler/issues/522

[burtonator]

Note: "Opened 9 years ago by "

... 9 years ago. Not going to be fixed anytime soon ;)

[THausherr]

But they did work on it, there were three patch sets submitted.

[xuiqzy]

On the upstream poppler bug someone commented on the poppler issue tracker in response to someone saying the poppler bug would be responsible for projects using poppler not being able to properly add annotations:

It's not really our fault if pdf.js is buggy. This bug has nothing to do with what you're complaining though, so please don't hijack it.

So it seems not to be clear which project should/can fix what. As I understood it, pdf.js could render annotations without an appearance stream, but PDF software should create appearance streams, at least when making PDFs with version 2.0, right?

Maybe someone from this project like @timvandermeij or @THausherr could write a message to someone responsible in poppler to constructively clear up what poppler can and can't do to change the end user experience of not being able to show annotations created in one software when viewing in another?
That would be highly appreciated!

If that does not seem like a good idea, what can be done instead to improve the situation?

[THausherr]

It doesn't really matter who is at fault. From a PDF 2.0 point of view, it's poppler, and from a PDF 1.* point of view, it's nobody and everyone. The head guy on poppler is "Albert Astals Cid" and he did comment on the issue so he is aware. It is useless to pressure people in OS projects on anything, the only result is that they'll feel bad. The best thing to do is to contribute yourself, if you can. I can't contribute code either, but I can help with test PDFs and strategies on the appearance content streams and giving feedback on produced PDFs because I implemented some of the annotations in Apache PDFBox.

[haxtibal]

Hi folks! Just wanted to cross link poppler issue 839, it's roughly the counter part at poppler side. No idea yet when we'll get to it though.

[timvandermeij]

@haxtibal Thank you for the information! I agree that both sides need work, so Poppler should provide the appearance stream and PDF.js should be able to build the appearance stream based on the information in the data dictionary if it is missing.

[timvandermeij]

Highlight, underline, squiggly and strikeout annotations without appearance streams are now supported by PDF.js thanks to the PR above. This issue remains open to track the remaining types, in particular freetext annotations.

[timvandermeij]

Square and circle annotations without appearance streams are now also supported (see the PR above).

[Snuffleupagus]

Ink annotations without appearance streams are now also supported, see PR #13301.

[timvandermeij]

Line annotations without appearance streams are also supported now since PR #13033.

[timvandermeij]

Polyline and polygon annotations without appearance streams are also supported now since PR #13305.

[Snuffleupagus]

Further improvements, for many different Annotation-types without appearance streams, were made in PR #13448; these include support for opaque stroke/fill colours and handling of inconsistently specified coordinates.

[prmahima]

Is this issue being worked on?
Is there a likelihood of it being resolved this year?
Is there currently some suggested workaround?
Thanks

[Snuffleupagus]

FreeText annotations without appearance streams are now also supported, see PR #15587.

---

Is there currently some suggested workaround?

The correct thing is to generate PDF documents with appearance streams for all annotations, which is also a requirement in the PDF 2.0 specification (as far as I recall).

---

Given that FreeText was the primary remaining type missing here, let's close this old issue as fixed now (and further types should be evaluated/fixed separately).