Is it posible to extract highlighted text?

[joelostblom]

I noticed this helpful wikipost on how to select text underlying a rectangle. Is it also possible to select highlighted text that is not in the shape of a rectangle? Preferably together with any comments/annotations made on that text regions.

[JorjMcKie]

You can extract the text (and images) from pages via page.getText("dict"). This works for non-PDF document also.
The result is a dictionary explained here. Except for text colors, this dictionary could be used to reconstruct a full document page in its original look, including images.
It would be your task to relate any annotations or links to those data: they are not be contained in that dict.

[joelostblom]

Thanks for the quick reply @JorjMcKie !

Just to make sure I understand correctly, when you say "It would be your task to relate any annotations or links to those data: they are not be contained in that dict." Do you mean that I will need to manually search for any text snippet I want to extract? Or is there a function within pymupdf which can extract the coordinates of the highlighted areas/texts and then I can use these coordinates to extract the relevant snippets from the dictionary containing all the extracted text from a page?

[JorjMcKie]

Not quite.

• The PDF concept of annotations represents a way to add "comments" or remarks with a reduced permission level. They don't count as "edits".
• The "normal" page text is not affected by annots - annots are like dust which can be wiped off again, if you follow this metaphor.
• Some annot types support highlighting and underlining text.

The basic PDF page's text itself can only have properties inherited from the font in use, like italic, bold, monospaced, serifed.

It cannot directly be highlighted or underlined. Any such effects come from outside the actual text specification (for which annots are just an example).
So it is your task to e.g. take the rectangle from a highlighting annotation, then dig your way through the page extraction dictionary and look for text pieces (so-called "spans") which are located within that rectangle.
This is what I was alluding to in the previous post.

[joelostblom]

Thanks for taking the time to elaborate on your explanation. This makes sense to me. I have been able to extract the highlight rectangles coordinates using PyPDF2 and python-poppler-qt5 (details here), but using those packages I had issues finding the text spans overlapping with those coordinates. Next, I will try using those coordinates in combination with the wiki page on how to extract text from a rectangle with PyMuPDF, and hopefully that should do the trick. Thanks again!

[JorjMcKie]

Bah, this is treason ... ;-)

You don't need another package for locating annotations. Why not using PyMuPDF for this?

[JorjMcKie]

scan through a page's annotations like this to find underline / highlight / strikeout / squiggly underlined annotations:

annot = page.firstAnnot

while annot:
    if annot.type[0] in (8, 9, 10, 11): # one of the 4 types above
        rect = annot.rect # this is the rectangle the annot covers
        # extract the text within that rect ...
    annot = annot.next # None returned after last annot

[joelostblom]

Ah that's great thanks! I saw that the rectangle area in the wiki example was constructed by searching for the actual words, which is why I wasn't sure if there was a way to find the highlighted area coordinates/rectangle using PyMuPDF. I will start off from your code, thanks again!

[jiedxu]

Hi, @JorjMcKie . Thanks for the wonderful package. Some enhancement may be required, because many words out of the highlighted area can be extracted as well. Here is my code.

import fitz
from itertools import groupby


def print_hightlight_text(page, rect):
    """Return text containted in the given rectangular highlighted area.

    Args:
        page (fitz.page): the associated page.
        rect (fitz.Rect): rectangular highlighted area.
    """
    words = page.getText("words")  # list of words on page
    words.sort(key=lambda w: (w[3], w[0]))  # ascending y, then x
    mywords = [w for w in words if fitz.Rect(w[:4]).intersects(rect)]
    group = groupby(mywords, key=lambda w: w[3])
    for y1, gwords in group:
        print(" ".join(w[4] for w in gwords))


def main():
    doc = fitz.open('./PDF/sample-3.pdf')
    page = doc[0]
    annot = page.firstAnnot
    print_hightlight_text(page, annot.rect)


if __name__ == "__main__":
    main()

The output of this file will be:

demand for biomass is uncertain at that time, and heat demand and electricity prices
vary drastically during the planning period. Furthermore, the optimal operation of
combined heat and power plants has to consider the existing synergies between the
power and heating systems. We propose a solution method using stochastic optimi-

Even when the fully contained words are extracted with the first method in this wiki post, the words before and after the highlight will be selected anyway:

vary drastically during the planning period. Furthermore, the optimal operation of
combined heat and power plants has to consider the existing synergies between the

Any hint as to this problem? My program will parse lots of highlights, so the introduction of that noise will have a huge impact.

[JorjMcKie]

The problems is this:

The annot rect is the blue one - not just the highlighted words!
The intersecting words are those surrounded with thinlined green rectangles.
If you change the code from intersects to fitz.Rect(w[:4]) in annot.rect, then only the two lines fully contained in the blue rect will be delivered.

[JorjMcKie]

Yor next question will probably be:
"How can I shrink the selection down to the highlighted words only?"

That should also be possible:

• The annotation was created surrounding text which spreads across more than 1 line
• This situation is reflected by a sequence of quadrilaterals - Quad objects in PyMuPDF.
• Tha annot rect is the smallest rectangle containing all those quads.
• 1 quad is given by 4 point-like objects.
• You can extract these points via annot.vertices:

>>> len(annot.vertices)  # always a multiple of 4!
8
>>> from pprint import pprint
>>> pprint(annot.vertices)
[(304.84161376953125, 274.20062255859375),
 (388.37060546875, 274.20062255859375),
 (304.84161376953125, 289.2906188964844),
 (388.37060546875, 289.2906188964844),
 (51.02360153198242, 286.20062255859375),
 (182.28359985351562, 286.20062255859375),
 (51.02360153198242, 301.2906188964844),
 (182.28359985351562, 301.2906188964844)]
>>> 

So if you do rect1 = fitz.Quad(annot.vertices[:4).rect] and rect2 = fitz.Quad(annot.vertices[4:]).rect, you get the two part rectangles and can do this:

>>> for word in wlist:
	if fitz.Rect(word[:4] in rect1:
		     
SyntaxError: invalid syntax
>>> for word in wlist:
	if fitz.Rect(word[:4]) in rect1:
		print(word[4])

		
optimal
operation
of
>>> for word in wlist:
	if fitz.Rect(word[:4]) in rect2:
		print(word[4])

		
combined
heat
and
power
plants
>>> 

This should be what you wanted ... 😎

[jiedxu]

The problems is this:

The annot rect is the blue one - not just the highlighted words!
The intersecting words are those surrounded with thinlined green rectangles.
If you change the code from intersects to fitz.Rect(w[:4]) in annot.rect, then only the two lines fully contained in the blue rect will be delivered.

I tried that one as well. Still want to reduce the extraction to the exact highlighted text. I want to analyse the highlighted text for users, so want as little pollution as possible.

Thanks for another quick solution. I will try it now.

[jiedxu]

Yor next question will probably be:
"How can I shrink the selection down to the highlighted words only?"

Thanks for the reply. The method works. I did try Annot.lineEnds, but it is not applicable to highlights.

Will write more code to make the program robust, so that it knows when to include the intersections.