Generated ToC can't be recognized by preview (Mac default App)

[xyliu]

here is my code: https://github.com/xyliu/pdftoc/blob/github/upt_toc.py#L88, the PDF files are from: https://github.com/Igalia/intel-osrc-gfx-prm/tree/master/010_kbl_-_2016-2017/.

The ToC is generated with standard operations documented, it can be recognized by Adobe Acrobat and Foxit Read, but failed by PDF expert in iOS and default previewer in macOS11.

Could you please have a check?

[JorjMcKie]

I tested your program on Windows - works just fine.
PDF with new TOC can be read by Adobe Reader, SumatraPDF, PDF-XChange Viewer, and Nitro 5 Reader.
The Nitro Reader couldn't read it at first - obviously because you are adding the TOC via incremental update: once I saved it non-incrementally, it was ok also.

I also had no problem at all with the standard file viewer in Linux (Ubuntu).

So I guess it should be a capability lack of the Apple program. I have neither of these, so I can't check myself.
Are you sure these apps can normally display TOCs? And also handle PDFs with TOCs given in incremental updates?

[xyliu]

Thanks for the quick response.

I add bookmark by hand though foxit reader, which can be recognized by the two apps(iOS/macOS).

The two Apps have two kind of ‘ToC’. One is 'bookmarks', the other is 'Table of Contents'. On the other way, acrobat and foxit reader have only 1. I guess that might be the cause. But I don't know the difference.

This is screenshot of Preview in Mac:

This is screenshot of Foxit Reader:

Meanwhile, I tried your idea, to save to a new .pdf file with "incremental=False", but still not work.

[JorjMcKie]

Can this "Preview" show the TOC of the Adobe Manual?
Can you please send me an example file, where you have manually entered a "bookmark" recognizable by Preview?
So far I always thought outline and bookmark are the same thing. Quote from the mentioned Adbobe Manual (section 8.2.2 on page 584):

A PDF document may optionally display a document outline on the screen, allow-
ing the user to navigate interactively from one part of the document to another.
The outline consists of a tree-structured hierarchy of outline items (sometimes
called bookmarks), which serve as a visual table of contents to display the docu-
ment’s structure to the user ...

[xyliu]

Yes. Adobe Manual's 'bookmark' can be shown as ToC, but not in bootmarks category in 'Preview'.

I tried with "https://www.foxitsoftware.com/pdf-reader/" (free to use), and the "https://github.com/Igalia/intel-osrc-gfx-prm/blob/master/010_kbl_-_2016-2017/intel-gfx-prm-osrc-kbl-vol13-mmio.pdf", the bookmark can be recognized as "ToC" in Preview.

[JorjMcKie]

I am confused:
If Preview shows the table of contents of the Adobe Manual, how can it do that if not as bookmarks? What other "categories" are there?
I am not talking about the stuff shown on pages 5-8.
Can you send a screen print of Preview doing this?

[JorjMcKie]

I suspect, when the Apple app talks of "bookmarks", it means its own method of taking bookmarks, which is independent of PDF. As I said: in PDF bookmark == outline.

If this suspicion is true, then Preview cannot show TOCs - whatever / whoever created them. My iPhone at least does not display the manuals TOC.

[xyliu]

Here is the screenshot of Preview doing with Adobe Manual. It shows the 'ToC' as Toc instead of bookmark. Other comments are shown as "highlights and notes".

[JorjMcKie]

Great, thanks a lot. Now I am convinced.
This means, that there is some difference in the TOC declarations, that Preview finds disturbing - in contrast to everyone else.

I will keep checking. I need your help here, because I do not have a Mac, and I don't want to create patches that I cannot test for their effect beforehand.

I am generating outline items from PDF source syntax in Python. This happens in file utils.py. If I send you an "experimental" new version of this file: could you manually replace the one you have in your Python installation with it and try out any effect?

[xyliu]

Sure. Glad to help.

This line https://github.com/rk700/PyMuPDF/blob/master/fitz/utils.py#L652 might have syntax issue. :)

I am generating outline items from PDF source syntax in Python. This happens in file utils.py. If I send you an "experimental" new version of this file: could you manually replace the one you have in your Python installation with it and try out any effect?

[JorjMcKie]

Oops - you are quite right. This error luckily never occurred :-)
Here is a version, that might work for you:
utils.py.zip
Thanks for your cooperation!

[xyliu]

Tried with utils.py.zip, the result is same as before. :(
I added log in utils.py to make sure the program use the new version.

$ diff -urN utils.py-old utils.py 
--- utils.py-old	2018-07-03 22:59:46.000000000 +0800
+++ utils.py	2018-07-03 23:07:41.000000000 +0800
@@ -747,6 +761,7 @@
         doc._updateObject(xref[i], txt)     # insert the PDF object
 
     doc.initData()
+    print("new version")
     return toclen 

[JorjMcKie]

Sh... disappointing.
Some more background:
Outlines have a leading entry, the so-called Outline Root. Among other things it contains a /Count of hierarchy level 1 elements. Erroneously I had specified this count as a negative number (e.g. /Count -5). This is however only valid for the outlines themselves (and controls whether they are displayed collapsed [< 0] or expanded [> 0]).
This was what had been fixed in the new utils.py. But obviously it was not what Preview finds so disturbing ...
@rk700 - would you have any advice on how to tackle this? I know you have a Mac, too.

[rk700]

Everything seems fine here on my machine:

System: 10.12.6
Preview: Version 9.0
Python: 3.7.0
PyMuPDF: 1.13.11

[rk700]

Here's the outputs of the script and the generated PDF file:

liuruikai756@mbp:~/Downloads$ python3 upt_toc.py
docs total page:  38
[2, 38]
row 22 ([3, 'BSD Logical Render Context Address (LRCA)', 18]): hierarchy steps must not be > 1
row 29 ([3, 'Copy Engine Logical Context Data', 24]): hierarchy steps must not be > 1
row 36 ([3, 'Video Enhancement Logical Context Data', 29]): hierarchy steps must not be > 1
intel-gfx-prm-osrc-kbl-vol03-gpu_overview.pdf

intel-gfx-prm-osrc-kbl-vol03-gpu_overview.pdf

[xyliu]

@rk700 The file you generated is fine on my Mac. Thanks!
My env:
System: 10.13.5 (17F77)
Preview: Version 10.0 (944.5)
Python 3.6.5 :: Anaconda custom (64-bit)
PyMuPDF: 1.13.11
intel-gfx-prm-osrc-kbl-vol03-gpu_overview.pdf