AttributeError: 'PdfFileWriter' object has no attribute 'stream' #670

rtibbles · 2022-04-07T23:47:04Z

I am very happy to see that PyPDF2 is being maintained and released again - however, it seems that the latest release appears to contain a regression. We had an unpinned dependency on PyPDF2 in our library, and when 1.27.0 was released, we started seeing this error:

../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:482: in write
    self._sweepIndirectReferences(externalReferenceMap, self._root)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:572: in _sweepIndirectReferences
    self._sweepIndirectReferences(externMap, realdata)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:548: in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:572: in _sweepIndirectReferences
    self._sweepIndirectReferences(externMap, realdata)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:548: in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:557: in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, data[i])
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:572: in _sweepIndirectReferences
    self._sweepIndirectReferences(externMap, realdata)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:548: in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:589: in _sweepIndirectReferences
    newobj = self._sweepIndirectReferences(externMap, newobj)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:548: in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:548: in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:589: in _sweepIndirectReferences
    newobj = self._sweepIndirectReferences(externMap, newobj)
../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:548: in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <PyPDF2.pdf.PdfFileWriter object at 0x7f3f9fe45460>
externMap = {<ricecooker.utils.pdf.CustomPDFReader object at 0x7f3f9fec98b0>: {0: {116: IndirectObject(7, 0), 118: IndirectObject(6, 0), 119: IndirectObject(3, 0), 120: IndirectObject(5, 0)}}}
data = IndirectObject(8, 0)

    def _sweepIndirectReferences(self, externMap, data):
        debug = False
        if debug: print((data, "TYPE", data.__class__.__name__))
        if isinstance(data, DictionaryObject):
            for key, value in list(data.items()):
                origvalue = value
                value = self._sweepIndirectReferences(externMap, value)
                if isinstance(value, StreamObject):
                    # a dictionary value is a stream.  streams must be indirect
                    # objects, so we need to change this value.
                    value = self._addObject(value)
                data[key] = value
            return data
        elif isinstance(data, ArrayObject):
            for i in range(len(data)):
                value = self._sweepIndirectReferences(externMap, data[i])
                if isinstance(value, StreamObject):
                    # an array value is a stream.  streams must be indirect
                    # objects, so we need to change this value
                    value = self._addObject(value)
                data[i] = value
            return data
        elif isinstance(data, IndirectObject):
            # internal indirect references are fine
            if data.pdf == self:
                if data.idnum in self.stack:
                    return data
                else:
                    self.stack.append(data.idnum)
                    realdata = self.getObject(data)
                    self._sweepIndirectReferences(externMap, realdata)
                    return data
            else:
>               if data.pdf.stream.closed:
E               AttributeError: 'PdfFileWriter' object has no attribute 'stream'

../../.virtualenvs/ricecooker_clean/lib/python3.9/site-packages/PyPDF2/pdf.py:575: AttributeError

It seems to be entirely internal to PyPDF2, so might be causing some issues for others as well.

The text was updated successfully, but these errors were encountered:

MartinThoma · 2022-04-08T05:10:27Z

Damn. Do you have a pdf + code to share that shows this issue?

Daniel-Steinberger · 2022-04-09T15:27:58Z

This issue is also discussed here: https://stackoverflow.com/questions/40168027/pypdf2-pdffilewriter-has-no-attribute-stream
I'll try to create a simplified example to reproduce this next week.

Daniel-Steinberger · 2022-04-12T13:59:50Z

pypdf2_files.zip

% python3 -m venv venv
% source ./venv/bin/activate
[venv] % pip install pypdf2==1.26.0
[venv] % python test.py

1.26.0

[venv] % pip install pypdf2==1.27.3
[venv] % python test.py

1.27.3
Traceback (most recent call last):
File "test.py", line 39, in
main()
File "test.py", line 35, in main
split(pdf_data, split_info)
File "test.py", line 22, in split
pdf_writer.write(tmp_io)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 480, in write
self._sweepIndirectReferences(externalReferenceMap, self._root)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 572, in _sweepIndirectReferences
self._sweepIndirectReferences(externMap, realdata)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 548, in _sweepIndirectReferences
value = self._sweepIndirectReferences(externMap, value)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 572, in _sweepIndirectReferences
self._sweepIndirectReferences(externMap, realdata)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 548, in _sweepIndirectReferences
value = self._sweepIndirectReferences(externMap, value)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 557, in _sweepIndirectReferences
value = self._sweepIndirectReferences(externMap, data[i])
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 572, in _sweepIndirectReferences
self._sweepIndirectReferences(externMap, realdata)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 548, in _sweepIndirectReferences
value = self._sweepIndirectReferences(externMap, value)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 589, in _sweepIndirectReferences
newobj = self._sweepIndirectReferences(externMap, newobj)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 548, in _sweepIndirectReferences
value = self._sweepIndirectReferences(externMap, value)
File "/home/dst/Desktop/pypdf2/venv/lib/python3.8/site-packages/PyPDF2/pdf.py", line 575, in _sweepIndirectReferences
if data.pdf.stream.closed:
AttributeError: 'PdfFileWriter' object has no attribute 'stream'

MartinThoma · 2022-04-20T16:44:50Z

This is a simplified version:

#!/usr/bin/env python3

from PyPDF2 import PdfFileReader, PdfFileWriter, __version__ as pypdf_version


print(pypdf_version)

reader = PdfFileReader("crazyones.pdf", strict=False, overwriteWarnings=False)
for _ in range(2):
    pdf_writer = PdfFileWriter()
    page = reader.getPage(0)
    print(page)
    pdf_writer.addPage(page)
    with open("foo.pdf", "wb") as f_pdf:
        pdf_writer.write(f_pdf)

The example works with the crazyones.pdf example in Ressources:

python foo.py
1.27.2
{'/Resources': IndirectObject(8, 0), '/Type': '/Page', '/Parent': IndirectObject(10, 0), '/Contents': [IndirectObject(7, 0)], '/MediaBox': [0, 0, 612, 792]}
{'/Resources': IndirectObject(5, 0), '/Type': '/Page', '/Parent': IndirectObject(1, 0), '/Contents': [IndirectObject(20, 0)], '/MediaBox': [0, 0, 612, 792]}
Traceback (most recent call last):
  File "foo.py", line 15, in <module>
    pdf_writer.write(f_pdf)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 480, in write
    self._sweepIndirectReferences(externalReferenceMap, self._root)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 572, in _sweepIndirectReferences
    self._sweepIndirectReferences(externMap, realdata)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 548, in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 572, in _sweepIndirectReferences
    self._sweepIndirectReferences(externMap, realdata)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 548, in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 557, in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, data[i])
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 572, in _sweepIndirectReferences
    self._sweepIndirectReferences(externMap, realdata)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 548, in _sweepIndirectReferences
    value = self._sweepIndirectReferences(externMap, value)
  File "/home/moose/.pyenv/versions/3.6.15/lib/python3.6/site-packages/PyPDF2/pdf.py", line 575, in _sweepIndirectReferences
    if data.pdf.stream.closed:
AttributeError: 'PdfFileWriter' object has no attribute 'stream'

It's interesting that reading the same page twice gives different results

MartinThoma · 2022-04-20T16:59:45Z

It works with PyPDF2==1.26.0, but breaks with PyPDF2==1.27.0. More specifically:

26e5077 broken
5bbd5af broken

MartinThoma · 2022-04-20T17:09:14Z

ce5f7ec - mixed tabs / spaces

MartinThoma · 2022-04-20T17:10:06Z

b030b7f - worked

MartinThoma · 2022-04-20T17:13:18Z

Here is what broke it: b030b7f...26e5077

MartinThoma · 2022-04-20T17:16:55Z

Removing this from PdfFileWriter._sweepIndirectReferences would remove the exception you're encountering:

if data.pdf.stream.closed:
    raise ValueError("I/O operation on closed file: {}".format(data.pdf.stream.name))

Closes #670

Closes py-pdf#670

rtibbles · 2022-11-29T16:58:04Z

Apologies for not seeing any of the follow up here - very glad to hear that this is fixed!

MartinThoma added the is-bug From a users perspective, this is a bug - a violation of the expected behavior with a compliant PDF label Apr 8, 2022

MartinThoma added this to the Last PyPDF2 version 1.X release milestone Apr 9, 2022

MartinThoma changed the title ~~Regression in latest release~~ AttributeError: 'PdfFileWriter' object has no attribute 'stream' Apr 11, 2022

MartinThoma added the Regression label Apr 11, 2022

MartinThoma added the Has MCVE A minimal, complete and verifiable example helps a lot to debug / understand feature requests label Apr 20, 2022

MartinThoma added a commit that referenced this issue Apr 20, 2022

BUG: 'PdfFileWriter' object has no attribute 'stream'

b1a2526

Closes #670

MartinThoma mentioned this issue Apr 20, 2022

BUG: 'PdfFileWriter' object has no attribute 'stream' #787

Merged

MartinThoma closed this as completed in #787 Apr 20, 2022

MartinThoma added a commit that referenced this issue Apr 20, 2022

BUG: 'PdfFileWriter' object has no attribute 'stream' (#787)

b36a564

Closes #670

MartinThoma mentioned this issue Apr 22, 2022

PdfFileWriter object has no attribute stream #555

Closed

VictorCarlquist pushed a commit to VictorCarlquist/PyPDF2 that referenced this issue Apr 29, 2022

BUG: 'PdfFileWriter' object has no attribute 'stream' (py-pdf#787)

fe3ea7f

Closes py-pdf#670

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AttributeError: 'PdfFileWriter' object has no attribute 'stream' #670

AttributeError: 'PdfFileWriter' object has no attribute 'stream' #670

rtibbles commented Apr 7, 2022

MartinThoma commented Apr 8, 2022

Daniel-Steinberger commented Apr 9, 2022

Daniel-Steinberger commented Apr 12, 2022

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022 •

edited

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022

rtibbles commented Nov 29, 2022

AttributeError: 'PdfFileWriter' object has no attribute 'stream' #670

AttributeError: 'PdfFileWriter' object has no attribute 'stream' #670

Comments

rtibbles commented Apr 7, 2022

MartinThoma commented Apr 8, 2022

Daniel-Steinberger commented Apr 9, 2022

Daniel-Steinberger commented Apr 12, 2022

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022 • edited

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022

MartinThoma commented Apr 20, 2022

rtibbles commented Nov 29, 2022

MartinThoma commented Apr 20, 2022 •

edited