Encounter a valid pdf file but PyPDF2 fail on it

[ulion]

that file can be decompressed by pdftk, but the FlateDecode of PyPDF2 failed:

  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/pdf.py", line 1751, in mergePage
    self._mergePage(page2)
  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/pdf.py", line 1801, in _mergePage
    originalContent, self.pdf))
  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/pdf.py", line 1714, in _pushPopGS
    stream = ContentStream(contents, pdf)
  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/pdf.py", line 2158, in __init__
    stream = BytesIO(b_(stream.getData()))
  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/generic.py", line 850, in getData
    decoded._data = filters.decodeStreamData(self)
  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/filters.py", line 310, in decodeStreamData
    data = FlateDecode.decode(data, stream.get("/DecodeParms"))
  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/filters.py", line 102, in decode
    data = decompress(data)
  File "/Users/ulion/Develop/gapps/email-pdf-sign.deploy/pdfsign/PyPDF2/filters.py", line 47, in decompress
    return zlib.decompress(data)
zlib.error: Error -5 while decompressing data: incomplete or truncated stream

here's the data to be decompressed (repr print):

'H\x89\xedW\xcbr\x1b\xb7\x12\xdd\xf3+f)W\x85#\xbc\x1fZ]\x8a\x0f\xc5%\x9a\x94I:\xded\xa3X\xd4\xe3F\x12mIN\xe2\xbf\xbf\xc0\x003\x83\x19\x00M\xd9I*Qr7*q\x80n4\xbaO\x9f>8\xde\x0c\x0eg\xb8\xc0\xa8D\xaa\xd8\\\x0e\x84\x94%Q\x05\xd3\xa4\xd4\xa2\xd8L\x8a\x83\xc9\xeeU\xb1\xf9\xef`hvp\xb3\xb2\xf9P\x1c|8\xaa>\x1d\xceHk\x88UIX\x81\xac\t\xaa6aM\x84\x92\xc4\xef4G\xe0\x121\xbbsH\x15/5)8-\xa5;b\xd4\x9c\xa0\xb4?\xe2\xa6\xfa\x84J\x8c\x04r_\x1e\xb2\x9b\xaa\xff\xef\xaf\xeau\x8c%&\xd5\xb7\xa2\x8d\x9c\x0bg2\tL\xcec\x8b\xa7`y\xfb\x18\xaf\x1f\x15\xc1\x06\x8a\xa3 \x87\x8d\tB\xd4\x99\xbc\x89N\xec\xdcj\x18,\x13\x84Y\xee\x16n\xc7f\x07\xaf\x13\x99\t\xc9-\x8f>>3\x80\xaa`\xc4V\x8b\xf0\xd2T\xb9\x02\x85)U\x95~V\xed]\x07v\xd7;\xef\xb7^l\x8beQd\x13\x1bF9Kow\x8bw}\xd3\xd0\xf2\xcd\xf6\xe2\xe6<\xda\xb0\xaeo\xe5\x12\xf2!\x8cl{\xf1\xf9v[}\x98n\x06\x9f\n\xac\x95\xc1\'*\x86D\x95\xaa\xa8\xfep\xa5\x8a\x0fw\xc5\xe1\xcd\x1d.&\xbb\xe2\xed\xe0\xb8\xd7\x13D\x97\xdc\xc0\x95\xc9RTI8 X\xd2\xa6\x0eT\x11\x16\xc5\xb9>on\xc8\x99[\xbe\x8d\xda\xe8\xe7\xa6\x18X\xc6W\x1dE\xfb\x7f\t\xc1\x19\xd9~\xb7\'\xa2\xcfQD7O\x91\xc3\xac9E\x08\xde0\x0e\xba\tw\xcb\n&\xe11\xf0\xf1\xd3\xf9E|\xad\xce!#8\x08M\x10R\xd5on\xc9fhHI\x92\xaad\xe38\x94\x90=\xb6\xf7\xd1\xc1OG~\xddxiYm\xdc\x14Vr*\x02\xc0\xba\xe5\x8fq\xdd\xc3c\xae\xee\xe3$\xfdx\x90\xc9Iw\xd7+\x17\x8e\xb9N\x18\xcer\xb4\x8e\x02\x7f\x1dV\x1d\xce\xd7|z2\x9a\xc3[f\xa0\xff\xc5h1\x9e\xd6\x0e\xa8L9X\x9eMW{\xa0\xb3\xdc\x13\xe5|\x9c\xd8\xe0\xf3\xe6\x0f\x99\xcf\x16?\xbe\xea\x16,\x9d\x1c\x9f\x11\x93\xc7z[L\xd0\t\x10\'\xfa6\xd3\x87OM\xa1\x82a8\x9a\xc1y\x9c\x83\xab\xa3\xcd\x9e\x1c/\xa6\x9b\xda\x01V\x9e\xd8\xdf\x87\xe9Y\x9d\xd6\x14\x88J\xa6\x8a_\x0b\xc2lg0D\xed$\x96\x0c\x97\xe6\xb6RX\xc2{\xd8\x0e\xd6\x96\x02)+\xb9(\x98\xe0\xa5\x1b\xd2q.\x9f\xa2\xc4uVC\xe0\xef\xee\xbbY98^\xbe\xab\x91\x13\x13`\'\xff\xc3\xf0\xa2\x81\x99+!\xad9y\xbcK\xb4p\xaa\xb2\xad\x8ay^\xe9\xc6\xcb\xf9|:]\x80\\}:\x1f\x9dLCd5\xea\xe9\xeczw\xbf\xed\xbaMu\xbfs#\x98Hu>x4c*\x91\xb3`\x834c2\xf6\xd0i\xf187\xbf=\xa72\xcf\x8f\xddE\xaa\xd0\xdeHu\xaa\xb45L\xd2\x94\x9b`\xdc\xdb(\xe1\xfd\xa2;P\xc63\xe16l\xed\xed}\x94\x97\x9fc\x930\x94\xabmd\x11\xce\xb7\xffd.\x92\x18\xf2\x99\xcbn/\xe2\xdb\x82 \x7f\x043\xd5\xb9n\xce6\x00A\x99i"\xf3\x85H\xc1\xdc\xa7\x9d\x0b\x98RU2\xc30%%m3\x1d\x7f\x8e<|I\x8a\x90\x96~]-)\xea=/&\xab\x86=z\x94g\xcc\x05N\xb1\xe4\xe9\xbb\x15H\xb3\'\xcd](\xf5\xcb~tifo\xb1\xa7{\xa5H\xe0{O\xf7\nDc\x9b\xb0\'0g\xb8\x19kMGLrc\xca\xa7\xf1\xd7L\xe5|\x1e>g\xcds\xba%U\x98\x10\xd8\xd71\xf2B\xb5\n\xe3r{q\x15\x8f\xd1\x1c\xd4P\x881C\xb4\xdc>"\x03\xc2\xdd\xc0\x91\xc6s\xea\xf2\xf2\xa6\xa9\xbb\xee>p\xbd\xf7\xbf\xa6\xee\x84\xf8+Ra\xc1\x17T?~\x19<>f\xca\x07\xb7sg>\xb6\xd2\xb4\xd1X~\x1e\xb2\xce<\\\xedr\x95\xc9\x80\xa7\xd9\x9f\xd6.\x9b\xd8_\x07:\x0f\x7fr\x07j\x02W\x82b\x8d\xe2\x0e\xfc\xc6,\xe4Zr_\x12\xfe\xa6\xed\xc7\x8dH4}7$\xc4\xb3\xfc\xc1\xfac<G\xe0\xc4$\xa6R\x11\xb2r<\x84s@\xce\xe6\'\x1e:\t\xed\xd7\x95\xab\xf1\xa1\xae\x17\x04\xd2V*c\xaa=\x12|S\xc4\xb7\xfe%\xee\xa7\x10w\x93\x84b\x8e&a\xb8z\x14Z\xa3=\xa2o\xf41\x91\x91p\x03A\x98\xc1\x01z\x86\xea\x84\x10,s\x04\x9b\x9f\xb9g\x95\x96\xf6\x811\xa4\x8e\x9c\xb3i\xea^\xd4+I\x82dI-\xb9\x1b\x11\xc1\x82\xa7ZgB\x8f\xdc\x84f\x8e#q\xc9\x03\xb5A\xdd\xe8\xd4\xc8\xbeu\x9a\xb5\x96?[/g\x80\x17\xe2\xbc\x0c\x85\xb0X\x7f\x8e\x1b?\x93\x9a\x9d\xb2\xfaj\x86I\x10\x85\xaf\xd3\xd7\x85R\xd7\x8c\x0bT*\x15\xc42\x89[\xfc\xcbc="\xadrj\'d(-\xdf\x84\x14T\x17"\xf9\x82\xdc\xcc\xd6\xee}\x83\r\x07"\xaf\xc4\x0e~\xc8\xea\x82T\xbff\x88\xf1\xde\xfbU\xf6\xf5\xe9\xfc\xce\xbd\xf4&f\xf6\xa9\xfa\xe3*\xd11\xee|\xf3\xac\x15\x8a[\xa0QA\xac\xc1]\xf3\x9b\xe2\xd2\x9cy\xeb\x9e\xb4\x92T\xab\xd46o\xd0\xbb_\xdb\x89\x89\xe9\x95\x9d\xaa\xb7\x1e\xca\xd4B\xa2U)M\xdd\x8d\xbe\xe3\xce\xec\xc6\'.Y\xf8\xe7\x1dc\xf2`r^\x1a\x1d\xde\xe4\xa1\xfe\xdd\xe6\xe1S\x81\xcdo"LKI\xa6\xed)\x15\xa3=l\x8b\xeb\xe2}q?\xc0:H\xdc\xdd@\x1aO]\x07o\x8bO\x85I&QU\x88m23\xaaz=v\xbf\xb8nqsv\xbd\xbbwy\xe5\xaa\x99\xeb\x07cx\x08\xee\xd1\xa0\xf5\xe3\x8bR\xeaXC\xd9\xae\xb7~Y\x94\xbb\xefB\x9eC\x9e\xc9$"e\xfdd\xfc\xd6&\xc9\xb2r\x08\x1f\xf7\x89U\x13$\x10VH#4\xca:p[\x04\xe2gn\x0eU\x97ty[,\xe3\xa8\x8a0\xae\xefG\x1b\x10\xc4\x98 p\xfd\x07\xec[T\x94\xa2~\x84e\xc6B\xed\x8a\x18X\xd4\xd2\xd9\x0c\x1b\xd4\xc7\xa7y\xcf\x99Vh\xf1\x89\xb9\x05\xa1\xefS\xa1u\x89\r\xb4H\xc5r\xceC\xbf\xd1k\x07\xf5\xef\x8e\x03nN\'*\xe1`\x88\xb5j\x8a|\xa0PB/\x86\xf9\xe6b\x8fLG\xba\x96\x8f!\xbc\x11\xc2pB\xd7\xc7\x94H\xe1\x93*\xdbJ\xce\xb3\xaa2l\xf3\x06\xe1\xdcf\xc9\xda\xc5\xa3:D8\xe1\t\x84\xef\x03\xe9\xd7O\x86\x00\xd0\x16\xa8\xe8\x0cN\x9dA\n\x1f\xbd<,S\xc2\xfbXv\xd0k\xb1\x8c*h\xf6\xb0\x8c*Q`=\xe0\x08\xcb\xb5\x83\xfaw\xc7A\x8d\xe5\xc8\xc1?\x03\xcbF\xdb\xf1\x0e\x96\x1b\xacVs\xe8w\xb1q\x02\x98\xab\xe5\xfae`\x8e\xb1V\x0eE\xec\x89\xba\xecITH~\x12a\x0b\x18bd\xafp\xb2\x99\xf7\x01\x87\xba\xe4\xd9\xb5\xe7\x92\xdaZD\xf6F\xc4\x10[\xab\x97\x8d737L\xc6\x02\xbca\x98<y}O\xdb\xc8\x8a\xfd_\x1e\xfc\t\x94\x1a\xc9\x03\x0f\xc8\x16\xe02A\xa9D\xf2\xfa\xf9\xc3\xfa\x08o\x1c\xd4\xbf;\x0e\xb8\xfb\x15;0\x10\xb7O\xcd\x17\x8e\xf0\xbe:  \xc0\xd5\x1f\xa8\x7f\x9f\x85\xef\xdf\xd5\x04\xf49M\x80^\xa2\xae\xe0Q\x13\xc8\x1e\xcb\xf3\x14\xcb\x0b\x0b\xd8\n\xae\xfd\x16\x90=\x92\xe7)\x92\xef\x99\xff\xcd9~q\x1e\x99=\xdd\x04v\xbb{\xdf\x04\x873R`d\'\xe8\xe6r`\x84\xa6M\xc0\x103\x7fW\xe1\xf7\xe0v\x0f&\xd2\xbeo\xdd1\x18l\x18\x9d\x92\xd3/j"lF\xa7S0\xff\xe3\xe5b\xb3Z\xce\x1b\xdfT\x11\x96\xed\x88?PJ{\x8c\xb6\x90g!m\xd7\x90g"\xa8b\x00x\xde\xe3\xfc\x8e1Q\xd8\x92}m\x8c\x10\x82!\xb8\x19\x9b\x98\x17\xdfJ\xc2i\xfc5\x8c\xfco\x07\xe0d5\x8a\xc3z\x1d\x98-N\xc0\xe2|?ZL\xfeZ\xb6\x8e\xa1\xcbzlMRlm\xe4\x85Ri\xf4\xb2\x1e]w\xeck\xf4\xb6\xf6{\x01\xbc\x9c\x8d7\xf8\x9b\x01l\x02#,\xb8\xc7\xdd@\nQ\xf6\xafu8\xa3\x85\xb4\xd1\x1a\x00ss\x88\xb0y(\x89c\xd9\xb3\xa5y\xa1\x12\xa1$\x196\x97e\x92\xb9\x07L\x85\xdb\xca\xbf\xb07z\xd8\x16\x97f\xb1\xaaF\xbd(IwQ\xda\x83\xdd"\xae2\x14,*dO\xcd-\x12\xc0\xad\xa2\xad\xdbx\x91\xdb\xa1\x95s[5lnQ\x03n5i\xa3\x8d\x17)\xe0Vs\xc8\xad\x04\xa25\x9a\x99\xe5,\xb9Ag\xd6-G\xb4u\x1b/r\xc8-PO\x8e\x80zr\x0c\xd4\x93\xe3\xa0\x9e\xf1"\x07\xa2\xc5\x12\x88\x16+ Z\xac\x81h\tPON\x80zr"\x81h\r\x91\xe4\x93`>\xe7\xa3\xa5P=)\xcd\xc3\xc4L\x10\xc8\xad\x84\xdcj\xc0-\xc3@\xe2\x19TO&\x80$0\xa8d\x0chA\xce1\x10-\xa7@\xb4\x1chA\xce\x81\x16\xe4\x1cjA\x01\xd5S@\xf5\x14\x0c@\x9f\xe0\x00\xfa\x04\xd4\x82\x12\x01n%\x06\xdcJ\xa8\x05\xa5\x80\xdc\x02\x94\xca%TO\x05\xb5\xa0\x82\xea\xa9\x18\x80>\x05\xd5SA\xf5\xd4\x08p\xab\x19\x00j\r\xb5\xa0\x86ZP\xeb|\xe2\x05\x02ZP \xa0\x05\x8d\x82\x83\xdc\x02\xfd)\x10\xd0\x9f\x02c\xc0-&\x80[\x0c\x94L`\x80R+\xc9\xc2\x1a\xc9Bt%\xaa\x84\x7f\x16\x8e\x1f\xbc\xa0BR\x0b\xa7\x8c\xb6\xa12z\x8a\x97/\x8a\x9f\xbe\x04[\x8e\xea\x1d\x9a\xe0P\x91Yy\x8c\x04%\xd5\xa7I`\xb0\xaa\xdfaHI\xec\x96\xdf\x87"\xee\xf4\xdd\xaav@0\xf3\x0e^7\xba\xcf\xfd>\t\x0c\xa6\xab8\x84"\xf4\xc8\x0e\xd5!A\x98e"u\x9b\xe8\x11GG\x8c\x83{\xce\xde\xc4\x8f\x18a\x80\xcd\x85y\xe9X\xa8U\x1a\xf0\xfcj\x0b\x0bu\xf8\x8d\xb9\x8b\x8c/\x0bR\x8b\xc9\xb7\xc5\xff\x00L\xc2\xa0'

the pdf file can also be opened by osx preview correctly.

[muhammedfurkan]

I get same error: zlib.error: Error -5 while decompressing data: incomplete or truncated stream @mstamy2

[Stef-Sijben]

I have the same issue. I attached 3 PDF files that illustrate the problem. form.pdf contains a form with an empty text field. PyPDF2 loads this just fine. Then I opened the file in Evince, filled in the form field and saved. Also loads in PyPDF2 without issue.
Then I opened the file in Acrobat Reader on Android, edited the form and saved. I can open this file in Evince without issue, and see the value I entered in the form field. When I open this file with PyPDF2 I get the same error mentioned in previous comments:

from pathlib import Path
import PyPDF2

for path in Path(__file__).parent.glob('form*.pdf'):
    with open(path, 'rb') as f:
        pdf = PyPDF2.PdfFileReader(f)
        fields = pdf.getFormTextFields()
        print(path.name, fields)

Output:

form.pdf {'foo': ''}
form_evince.pdf {'foo': 'bar'}
Traceback (most recent call last):
  File "/path/to/script.py", line 8, in <module>
    pdf = PyPDF2.PdfFileReader(f)
  File "/usr/lib/python3/dist-packages/PyPDF2/pdf.py", line 1084, in __init__
    self.read(stream)
  File "/usr/lib/python3/dist-packages/PyPDF2/pdf.py", line 1807, in read
    streamData = BytesIO(b_(xrefstream.getData()))
  File "/usr/lib/python3/dist-packages/PyPDF2/generic.py", line 845, in getData
    decoded._data = filters.decodeStreamData(self)
  File "/usr/lib/python3/dist-packages/PyPDF2/filters.py", line 346, in decodeStreamData
    data = FlateDecode.decode(data, stream.get("/DecodeParms"))
  File "/usr/lib/python3/dist-packages/PyPDF2/filters.py", line 111, in decode
    data = decompress(data)
  File "/usr/lib/python3/dist-packages/PyPDF2/filters.py", line 49, in decompress
    return zlib.decompress(data)
zlib.error: Error -5 while decompressing data: incomplete or truncated stream

On Ubuntu 20.10, Python 3.8.6 and PyPDF2 1.26.0 installed through apt.

The error occurs while decoding the xref table (the last one in the file, first one to be read). I can also reproduce this if I take only the contents of the xref stream and feed that to zlib:

import zlib

data = b'x\x9cbd`\x084a``d`\x08b\x87Pi`*x6\x98\n\xb9\x08\xa6\xc2\xa4\x18\x18\x00\x00\x00\x00\xff\xff'
print(f'data: {data}')

decompressed_data = zlib.decompress(data)
print(f'decompressed_data: {decompressed_data}')

data: b'x\x9cbd`\x084a``d`\x08b\x87Pi`*x6\x98\n\xb9\x08\xa6\xc2\xa4\x18\x18\x00\x00\x00\x00\xff\xff'
Traceback (most recent call last):
  File "/path/to/script.py", line 6, in <module>
    decompressed_data = zlib.decompress(data)
zlib.error: Error -5 while decompressing data: incomplete or truncated stream

However, if I use zlib's decompressobj() function, it works:

import zlib

data = b'x\x9cbd`\x084a``d`\x08b\x87Pi`*x6\x98\n\xb9\x08\xa6\xc2\xa4\x18\x18\x00\x00\x00\x00\xff\xff'
print(f'data: {data}')

decompressed_data = zlib.decompressobj().decompress(data)
print(f'decompressed_data (obj): {decompressed_data}')

data: b'x\x9cbd`\x084a``d`\x08b\x87Pi`*x6\x98\n\xb9\x08\xa6\xc2\xa4\x18\x18\x00\x00\x00\x00\xff\xff'
decompressed_data (obj): b'\x01\x00\x00Q4\x00\x00\x01\x00\x00R\x07\x00\x00\x01\x00\x00Rf\x00\x00\x01\x00\x00S\x9b\x00\x00\x01\x00\x00T\xd1\x00\x00\x01\x00\x00V\x1a\x00\x00'

I don't understand why this happens, since zlib's documentation says:

Returns a decompression object, to be used for decompressing data streams that won’t fit into memory at once.

and this stream certainly fits in my memory. But it resolves the issue for me, so maybe it would be good to always use this when decompressing flate encoded streams.

form.pdf
form_acrobatReader.pdf
form_evince.pdf