Qpdf throwing the range conversion error

[santoshturamari]

I am trying to encrypt the pdf file. But it throws the range error.
"integer out of range converting 4294967295 from a 8-byte signed type to a 4-byte signed type"
Please clarify why it appears and how to fix.

@jberkenbilt

TIA

[jberkenbilt]

This is most likely due to a malformed input PDF file, but to be able to help you, I would need the input file and to know what version of qpdf you are using.

[santoshturamari]

We are using the version qpdf version 9.1.1.
The PDF is in a proper format when opened. But I may not be able to share it here, as it is company asset.

Thanks

[jberkenbilt]

You might try qpdf 10.0.3 (just out) and see if it does better. I feel like I fixed some problem with out-of-spec encryption dictionary parameters. There are lots and lots of files that are not compliant with the PDF spec but that open fine in most readers. qpdf is very good at handling these most of the time, but there are always special cases that it doesn't (yet) handle. I try to enhance qpdf to cover these when I learn about them.

If you are not able to share the PDF, you can try running qpdf --json file.pdf and send that. This won't include any content (page text or images), though it will include things like text strings from bookmarks, if any, and some metadata. Look over the output and send it if you can. Most of the time, people are able to share this even when they can't share the file. Usually I can diagnose most problems by just looking at the json.

Alternatively, you can try, as a workaround, first running qpdf --decrypt in.pdf out.pdf and then encrypting the result of that. Sometimes, when qpdf just rewrites a file, it will fix a number of problems of this type.

[santoshturamari]

Hi @jberkenbilt
We have tried your commands, they throw the same error even when upgrading to higher version.
Even qpdf --json file.pdf also throws the same error.
But we observed other pdfs are working fine.
I have attached the PDF here. This is created from the template generated in Docusign

LOA-workplace-pension_v6.pdf

integer out of range converting 4294967295 from a 8-byte signed type to a 4-byte signed type
The same error is happening for the document encryption.

[jberkenbilt]

Thanks, I am studying the problem, but in the meantime, I was able to manually recover the file and am attaching the repaired file:

LOA-workplace-pension_v6-fixed.pdf

I'll update this ticket when I have a proper fix and/or a usable workaround.

[santoshturamari]

Thanks for the reply.
But this thing goes manually in our application. The template comes from docusign.
What needs to be done in that case?
Is there any problem with docusign template?

[jberkenbilt]

I know what the problem is. It's an easy fix in qpdf, but I'm not sure there's a good workaround without building a new version of qpdf. Technically, the issue is that the PDF file contains an out of range value in a place where the value is actually unused. Unfortunately, qpdf is range-checking the value even though it doesn't really have to. The secondary problem is that the range error is not triggering file recovery because of the type of exception that gets thrown in the code. I'm going to try to fix both problems.

Where do you get qpdf from? Do you build your own? What platform? Are you able to use a patched version of qpdf? It's going to be a very simple fix, but this error is happening so early in qpdf's reading of the file that I don't think there is going to be a suitable workaround with 10.0.3 or earlier. I'd rather not release 10.0.4 so soon after 10.0.3 just for this bug if I can avoid it, especially since this problem has actually been in qpdf since 9.0.0 when I started doing the range checks. But let me know.

[santoshturamari]

We install the qpdf - apt-get install qpdf
Platform - Linux
We can use the patched version of qpdf.
Please let me know, if there is any place in the PDF which is causing the problem? So that we can avoid that field.

Because the templates without signature are working fine, the one with signature is causing problem. That's just the assumption.

Thanks

[jberkenbilt]

The problem is that one of the xref streams has /W[1 4 4]. The third 4 indicates use of 4 bytes for an offset within the object stream. When the entry type is 0, meaning "deleted object", the value of the third field is all 1s (4294967295), which is causing the range error. When writing out a deleted object in the xref stream, the software should write all 0s, not a -1 in the final position. It is the combined fact that the third element of /W in the xref stream is >= 4 and that the resulting value is greater than 2^31 that is causing the problem. I'm not sure whether this is within your control to change.

[jberkenbilt]

The value of the offset/generation bytes when the xref entry type is 0 is not used, so there's no reason for qpdf to be stumbling over this. I have just added code to ignore the last field if the entry type is 0.

[santoshturamari]

So do we need to install the patched version now? can you please give me the version number to check.
Thanks

[jberkenbilt]

I haven't generated it yet. I am about to push to CI. It will be about 30 minutes before I have a final version. Then I need to figure out how to get it to you.

[jberkenbilt]

How big of a problem is it for you if this is not installable via apt-get? Which distribution are you on? Are you using a binary or linking C++ with the library?

[jberkenbilt]

If you are able to build qpdf yourself, you can download 10.0.3 and apply the patch from the commit that is referenced above. Or maybe you have the ability to change how the files are written so the issue doesn't get triggered. Using 4 bytes for the third field in an xref entry is strange and probably never necessary in any PDF file.

[santoshturamari]

I am unable to build qpdf or apply the changes. If I could install via apt-get, it would be more helpful for me.
Thanks.

[jberkenbilt]

Ordinarily I would not release a new version of qpdf to fix a single issue unless that issue was a regression from the most recent previous release. I just released 10.0.3 on October 31, so I think it's a little soon to release a new version. If I haven't had a chance to work on new features in the next few weeks, I could release 10.0.4 with this fix and anything else that comes up, but barring some catastrophic issue, I would not want to do a release for a few weeks at least. It's disruptive to packagers and some users to get updates too frequently.

In the near term, I can easily supply a patched source distribution or a patched AppImage built from CI. The AppImage would enable you to run qpdf as an executable.

[msmahon]

@jberkenbilt Just chiming in to say we are having the same issue with DocuSign docs. Glad to see a patch is coming in the near future.

[jberkenbilt]

@msmahon Thanks for letting me know. Knowing that this is affecting multiple people may accelerate my release of a new version. I guess I'll try to get a 10.0.4 out soon that has this fix. I am a consumer of DocuSign documents myself, though I guess I've never run qpdf on them. :-)

[jberkenbilt]

I am preparing 10.0.4 now and will have it out in the next hour or so. That will include the fix to this problem.

[santoshturamari]

@jberkenbilt : Can you please confirm, the current latest version has the fix to this problem?

[jberkenbilt]

@santoshturamari Yes, the fix for this is in 10.0.4, which is the current latest version.

[rcx]

This fix worked for me too, thank you for your hard work :)

[donal-s]

Thanks for dealing with this - had this issue with multiple readers for multiple Docusign PDFs. Now qpdf --linearize creates a document that other PDF readers can read. Kudos.

[jberkenbilt]

@donal-s Thanks -- I appreciate the feedback. It's good to know this is helping people.