Does PyMuPDF support TOC creation from a pdf document?

[tikitong]

First of all thank you very much for this great work. I particularly appreciate your layout preserving text extraction method.
My question is: does pymupdf support TOC creation for a pdf document?

get_toc() method used such as:

import fitz

pdf_filename = 'my.pdf'
with fitz.open(pdf_filename) as doc:
    print(doc.get_toc())

seems to give results only if the TOC is already present at the beginning of the document.

On this pdf: https://github.com/pymupdf/PyMuPDF-Utilities/blob/master/text-extraction/Dart.pdf for example, is there a method in pymupdf to generate the pdf outline? In this case only the numbered titles of the paragraphs.

As well as for pdf with more complex formatting such as: https://blog.xpgreat.com/file/lstm.pdf, with numbered parts and sub-parts.

I also tested mupdf with the command: mutool show my.pdf outline but it returns nothing with or without TOC inside the pdf file in my case.

Your configuration (mandatory)

In my case, I made installation on macOS arm64 M2 (not Intel).
I create a conda osx-64 environment inspired from this amazing solution, and its work for me.

conda create -n pymupdf
conda activate pymupdf
conda config --env --set subdir osx-64
conda install python=3.9 
python -m pip install --upgrade pymupdf

print(sys.version, "\n", sys.platform, "\n", fitz.__doc__) gives me:

3.9.13 (main, Oct 13 2022, 16:12:30) 
[Clang 12.0.0 ] 
 darwin 
 
PyMuPDF 1.20.2: Python bindings for the MuPDF 1.20.3 library.
Version date: 2022-08-13 00:00:01.
Built for Python 3.9 on darwin (64-bit).

Please feel free to modify the README.md to notify macOS users with the apple chip that it also works by following this steps, I'm sure it will be useful for some :).

[JorjMcKie]

This is a typical "Discussions" item, so let me convert it before asnwering.

[JorjMcKie]

Talking about TOC in PDF can easily become confusing:

1. A PDF may have a special internal structure of interrelated objects, with forward, backward, downward, upward pointers. PDF viewers show this structure on the left in a separate detail windows, where you can click on single items for fast access of the respective page.
2. Then there may also (or only) exist pages in the PDF's content, showing a "printed" version of that structure. In most cases, the single lines in this area are PDF internal links. So clicking on them will also jump to the respective page.

A PDF technically can have both of the above things, or just one, or neither. If neither exists or just number 2 above, then doc.get_toc() will be an empty list - of course.

I think you are asking for a way to create number 1 if just number 2 exists, right?

Because TOC pages in a PDF are just ordinary text, it is left to the programmer's wits to

1. identify those pages (which may not be trivial at all in itself!), and
2. programmatically turn each line item on a TOC page into an item acceptable for doc.set_toc(). This too may be quite challenging.

And there certainly also is no function in PyMuPDF which creates a TOC if nothing of the above exists at all.
I think you were asking for something that interprets things like section headers, text properties (bold) and similar.
No this is left to you as a programmer. I think I saw your similar question on StackOverflow the other day.

---

seems to give results only if the TOC is already present at the beginning of the document.

Correct ... with the redaction.

[bsdcpp]

Is there a method in pymupdf to generate a printable toc page like number 2?
Just like the one starting on the fourth page of "mupdf_explored.pdf",
it’s very beautiful and seems to be made using Latex.
Or I need to write my own toc pages with title, page numbers and links.
Thank you.

[JorjMcKie]

@bsdcpp You should have a look at the features of the Story class.
Your intention looks like a typical use case for it. You would have to prepare the formatting of the TOC line items of course. But outputting them on predefined rectangles is a thing that Story has been built for.
Once the pages produced by such an algorithm are ready, prepend (or append) them the document in question.

[bsdcpp]

Thanks a lot for the quick reply.
Before, I've been using qpdf/gs for decryption(Some files have permissions set), pdfrw for processing, and also cooperated with pdfkit to process some HTML.
Now I can put them together into pymupdf, which is much more efficient.

[JorjMcKie]

Glad you like it!

[bsdcpp]

After reading the documentation, I found that there's already a good sample at docs/samples/code-printer.py, which is really great.

[tikitong]

Thank you for this explanation, I had difficulty in describing the problem precisely.

I think you were asking for something that interprets things like section headers, text properties (bold) and similar.

It is exactly that!

No this is left to you as a programmer.

Yes, that sounds exciting! Before starting a development, I try to find out what has already been done to solve this problem.

1. Is this an issue that has been raised before? I'm sure I'm not the first person to try to find a solution to this task.

2. Thus, do you know if there are already solutions that come close to this goal or any libraries?

3. I think that the pdf.tocgen solution is able to generate the TOC without it being present in the PDF, and apparently without internal links too, as show with its example. It is however specified that it extracts metadata (font attributes, positions), it is not pointers in the pdf, right? When I open the pdf from my side, it doesn't seem to meet either of your two above conditions. Have you already used this solution?

[JorjMcKie]

1. No. There have been people occasionally, who thought that things like a header or a footer must be miraculously identifiable in a PDF - which is not case of course, although that feature is contained in the official PDF specs, just nobody is ever using it.
2. and 3.: No, and I do not know that solution or package either. Looking into it shows that is using PyMuPDF ... and confirming that finding a solution to your problem is by no means trivial.

[tikitong]

Well noted, thanks again for the information and details.

Edit : @JorjMcKie I find this question very interesting. Have you ever thought about undertaking the development of a solution for this task with the PyMuPDF team?

[JorjMcKie]

I once (long ago) have written a GUI script which interactively allows to create a TOC:
It displays the pages on the right windows and offers to add / edit the lines of a TOC on the left windows.
May you can use it. But be prepared that there may be recent API changes that this scripts has no knowledge of.

[tikitong]

Yes I had seen this script while looking in PyMuPDF, I will think about developing a solution.