Possible bug: two letters in 'text' field of 'char'

[LivingDeadCloud]

Hello everybody

I was using PDFPlumber to extract text from a PDF with some tables/tablish text. Here is a glimpe (I'm afraid I cannot share the entire document, since it is a medical report of a patient):

I noticed that for some specific set of characters, a single char has two letters in text. In particular, this happens for double f, for example in the word differita (marked with red underline in picture and with dashes in the following code):

[{'fontname': 'BAAAAA+DejaVuSans',
  'adv': Decimal('6.340'),
  'upright': True,
  'x0': Decimal('126.040'),
  'y0': Decimal('460.850'),
  'x1': Decimal('132.380'),
  'y1': Decimal('470.850'),
  'width': Decimal('6.340'),
  'height': Decimal('10.000'),
  'size': Decimal('10.000'),
  'object_type': 'char',
  'page_number': 3,
  'stroking_color': (0, 0, 0),
  'non_stroking_color': (0, 0, 0),
  'text': 'd',
  'top': Decimal('371.011'),
  'bottom': Decimal('381.011'),
  'doctop': Decimal('2054.733')},
 {'fontname': 'BAAAAA+DejaVuSans',
  'adv': Decimal('2.770'),
  'upright': True,
  'x0': Decimal('132.340'),
  'y0': Decimal('460.850'),
  'x1': Decimal('135.110'),
  'y1': Decimal('470.850'),
  'width': Decimal('2.770'),
  'height': Decimal('10.000'),
  'size': Decimal('10.000'),
  'object_type': 'char',
  'page_number': 3,
  'stroking_color': (0, 0, 0),
  'non_stroking_color': (0, 0, 0),
  'text': 'i',
  'top': Decimal('371.011'),
  'bottom': Decimal('381.011'),
  'doctop': Decimal('2054.733')},
 {'fontname': 'BAAAAA+DejaVuSans',
  'adv': Decimal('6.880'),
  'upright': True,
  'x0': Decimal('135.130'),
  'y0': Decimal('460.850'),
  'x1': Decimal('142.010'),
  'y1': Decimal('470.850'),
  'width': Decimal('6.880'),
  'height': Decimal('10.000'),
  'size': Decimal('10.000'),
  'object_type': 'char',
  'page_number': 3,
  'stroking_color': (0, 0, 0),
  'non_stroking_color': (0, 0, 0),
  -----------------------------------'text': 'ff',-----------------------------------
  'top': Decimal('371.011'),
  'bottom': Decimal('381.011'),
  'doctop': Decimal('2054.733')},
 {'fontname': 'BAAAAA+DejaVuSans',
  'adv': Decimal('6.150'),
  'upright': True,
  'x0': Decimal('142.020'),
  'y0': Decimal('460.850'),
  'x1': Decimal('148.170'),
  'y1': Decimal('470.850'),
  'width': Decimal('6.150'),
  'height': Decimal('10.000'),
  'size': Decimal('10.000'),
  'object_type': 'char',
  'page_number': 3,
  'stroking_color': (0, 0, 0),
  'non_stroking_color': (0, 0, 0),
  'text': 'e',
  'top': Decimal('371.011'),
  'bottom': Decimal('381.011'),
  'doctop': Decimal('2054.733')},

From what I read in the documentation, every char should have a single letter in its text field.
I've seen that in a previous post you talked about malformed PDFs. What is exactly a malformed PDF? Maybe this could be the cause of this problem?

Thanks a lot

[jsvine]

Hi @LivingDeadCloud, I think you've come across a ligature — a typographic convention in which two letters combined into a single symbol: https://en.wikipedia.org/wiki/Orthographic_ligature

So the char you see is, indeed, one symbol — but it represents two letters. I don't know exactly where the conversion happens, but it's "before" pdfplumber. My best guess, without doing further research, is that it's "before" pdfminer.six as well and instead part of the font's definition.