stack overflow / crash on malformed input in QPDFWriter::enqueueObject(QPDFObjectHandle)

[hannob]

The attached file will crash qpdf. It seems it's running into an endless recursion and thus a stack overflow.
Found with afl.

qpdf-stackoverflow.zip

==24283==ERROR: AddressSanitizer: stack-overflow on address 0x7ffda4d32f78 (pc 0x00000050ba42 bp 0x7ffda4d337c0 sp 0x7ffda4d32f60 T0)
    #0 0x50ba41 in operator new(unsigned long) (/r/qpdf/qpdf+0x50ba41)
    #1 0x62c243 in PointerHolder<QPDFObject>::PointerHolder(QPDFObject*) /f/qpdf/include/qpdf/PointerHolder.hh:74:17
    #2 0x62c243 in QPDFObjectHandle::QPDFObjectHandle(QPDF*, int, int) /f/qpdf/libqpdf/QPDFObjectHandle.cc:46
    #3 0x62c243 in QPDFObjectHandle::newIndirect(QPDF*, int, int) /f/qpdf/libqpdf/QPDFObjectHandle.cc:1183
    #4 0x5b73af in QPDFObjectHandle::Factory::newIndirect(QPDF*, int, int) /f/qpdf/include/qpdf/QPDFObjectHandle.hh:520:13
    #5 0x5b73af in QPDF::getObjectByID(int, int) /f/qpdf/libqpdf/QPDF.cc:1889
    #6 0x674910 in QPDFWriter::enqueueObject(QPDFObjectHandle) /f/qpdf/libqpdf/QPDFWriter.cc:1057:27
    #7 0x67491b in QPDFWriter::enqueueObject(QPDFObjectHandle) /f/qpdf/libqpdf/QPDFWriter.cc:1057:3
    #8 0x67491b in QPDFWriter::enqueueObject(QPDFObjectHandle) /f/qpdf/libqpdf/QPDFWriter.cc:1057:3

[jberkenbilt]

You're doing me a great service by finding all these cases. I hope you keep it up. I should be releasing 7.0.0 pretty soon. It would be great to fix as many of these as possible. I've coded a fix for this, but I need to do some extra code inspection to make sure my fix is good. It definitely doesn't break any existing tests, and the test suite is very thorough, and it also catches this problem, but I'm not sure it fully addresses the underlying issue, so I'll do more checking before I push to master.

[jberkenbilt]

You're doing me a great service by finding all these cases. I hope you keep it up. I should be releasing 7.0.0 pretty soon. It would be great to fix as many of these as possible. I've coded a fix for this, but I need to do some extra code inspection to make sure my fix is good. It definitely doesn't break any existing tests, and the test suite is very thorough, and it also catches this problem, but I'm not sure it fully addresses the underlying issue, so I'll do more checking before I push to master.

[ghost]

This has been assigned CVE-2017-18183

[ghost]

This has been assigned CVE-2017-18183