Why there is additional spaces after each character when trying to get textline in pdf?

[nissansz]

p2.pdf
'''
doc = fitz.open(path)

j = 0
k = 0
s3 = set()
for i in range(0, doc.page_count):
    print(f'开始识别页码：{str(i + 1)}/{doc.page_count}')
    k = k + 1
    page = doc[i]
    for block in page.get_text("dict", flags=fitz.TEXTFLAGS_TEXT)["blocks"]:
        for line in block["lines"]:
            j = j + 1
            bbox = line["bbox"]  # the line bbox
            text = ''.join([span["text"] for span in line["spans"]])  # text in line

            # print(bbox)
            print(text)

'''

line result example is '【 請 求 項 １ 】'
When copying directly with Adobe, result is "【請求項１ 】", no additional spaces

[nissansz]

cnp1.pdf
Another problem is, the bbox is smaller than real text, and the cropped image may cut the final character to half.

Line 2 in attached pdf is wrongly cropped, and result is as follows.

[JorjMcKie]

This is largely a problem introduced by the standard text flags. Things will improve if you only use fitz.TEXT_DEHYPHENATE | fitz.TEXT_MEDIABOX_CLIP.

Otherwise, you may be interested in looking at the new package / repository pymupdf4llm.
It will produce markdown text output, but also detects multiple text columns and tables and creates a unified output across all of this content. When using it with your 2 examples like this:

import pathlib
import pymupdf4llm

data=pymupdf4llm.to_markdown("cnp1.pdf")
pathlib.Path("cnp1.pdf.md").write_bytes(data.encode())

data=pymupdf4llm.to_markdown("p2.pdf")
pathlib.Path("p2.pdf.md").write_bytes(data.encode())

You will receive these output files:
cnp1.pdf.md
p2.pdf.md

[nissansz]

How to use fitz.TEXT_DEHYPHENATE | fitz.TEXT_MEDIABOX_CLIP to parse each text line?

[JorjMcKie]

set text extraction flags to this value!

[nissansz]

How to revise in below code?

doc = fitz.open(path)

j = 0
k = 0
s3 = set()
for i in range(0, doc.page_count):
print(f'开始识别页码：{str(i + 1)}/{doc.page_count}')
k = k + 1
page = doc[i]
for block in page.get_text("dict", flags=fitz.TEXTFLAGS_TEXT)["blocks"]:
for line in block["lines"]:
j = j + 1
bbox = line["bbox"] # the line bbox
text = ''.join([span["text"] for span in line["spans"]]) # text in line

        # print(bbox)
        print(text)

[JorjMcKie]

Instead of flags=fitz.TEXTFLAGS_TEXT use flags=fitz.TEXT_DEHYPHENATE | fitz.TEXT_MEDIABOX_CLIP

[nissansz]

Same wrong result.