blocks not identified and so disappear from exported PDF

[aviadkl]

Hi,

i'm trying to redact a few words from a PDF file.
The right words are redacted but other, unrelated words, disappear from the file, they are not being redacted, they just disappear.
trying to locate the problem i ran this code:

fitz.TOOLS.set_small_glyph_heights(True)
doc = fitz.open("file1.pdf")
page = doc[0]
page.clean_contents()
blocks = page.get_text("dict")["blocks"]
for b in blocks:
	if b["type"] !=0: continue
	for l in b["lines"]:
		for s in l["spans"]:
			bbox = fitz.Rect(s["bbox"])
			if bbox.height < s["size"]:
				bbox.y0 = bbox.y1 - s["size"]
			page.draw_rect(bbox, width=0.3, color=(1,0,0))
doc.save('file1_redacted.pdf')

in the new PDF i can see that all words have a red bounding box around them except the words that are missing from the redacted file. looking a bit further i noticed that these spans are also missing from the blocks object but these do exist if i run page.get_text("text").

my base code that i'm running to redact the file is:

import fitz
import re

fitz.TOOLS.set_small_glyph_heights(True)
doc = fitz.open("file1.pdf")
page = doc[0]
page.wrap_contents()
lines = page.get_text("text").split('\n')

def get_sensitive_data(lines):                
        reg_exp = r"PACIENTE:"
        for i,line in enumerate(lines):            
            if re.search(reg_exp, line, re.IGNORECASE):
                found_name = lines[i+1]
                found_id = lines[i+2]
                return [found_name, found_id]

sensitive = get_sensitive_data(lines)

for data in sensitive:
    areas = page.search_for(data)
    for rect in areas: 
        page.add_redact_annot(rect, fill = (0, 0, 0))                   

                     
page.apply_redactions() 
doc.save('file1_redacted.pdf')

i'll add a few screenshots to illustrate this. notice that the words: "Rh (D)" and "Biometría Hemática" disappear.

i'm using version 1.21.1.

any help would be much appreciated.
thanks,

Aviad

original file

bounding box file

[JorjMcKie]

Please provide a reproducing file. The images are not helpful to track down a possible error.

Independent from that, I recommend changing the logic somewhat:
Read the lines in a way that delivers the line bbox together with the line text. Instead of decoupling these things, because this introduces possible sources of error.

blocks = page.get_text("dict", flags=fitz.TEXTFLAGS_TEXT)["blocks"]  # blocks of text on page
for block in blocks:
    for line in block["lines"]:
        text = "".join([span["text"] for span in line["spans"]])  # text in the line
        if is_sensitive(text):  # check if line text must be redacted
            page.add_redact_annot(line["bbox"])  # if yes, redact it
page.apply_redactions()

If the problem remains, please provide an example file.

[aviadkl]

hi,

i tried your code but it doesn't solve the problem. the disappearing words are already missing after the first row:
blocks = page.get_text("dict", flags=fitz.TEXTFLAGS_TEXT)["blocks"]

which is weird because as i mentioned they do appear when running lines = page.get_text("text")

i'm starting to think that this problem is specific to the producer of these PDF files, unfortunately i can't upload the file itself as it contains personal medical data.

Thanks again

[JorjMcKie]

Hm, understand. Maybe you are encountering a duplicate of a pending bug.
No way to tell w/o a reproducer.
So would have to wait for the next version.