[BUG] SubprocessOutputError: Ghostscript PDF/A rendering failed - Cant upload PDF Files since 1.17

[amigappc2023]

Description

if i upload a PDF i get the failure "SubprocessOutputError: Ghostscript PDF/A rendering failed"
I just updated the Docker Container from 1.16.5 to 1.17 and it works in the beginning fine. after a restart both diff. instanzes arent working anylonger.

Steps to reproduce

i uploaded several diff. PDFs, and i used 2 different systems with the same Docker Image and i get always the same result.

Webserver logs

[2023-08-06 08:41:08,862] [INFO] [celery.worker.strategy] Task documents.tasks.consume_file[8536feb8-15f8-4720-a8df-20a2894129f3] received
[2023-08-06 08:41:09,399] [INFO] [paperless.consumer] Consuming 201812_Kontoauszug_1822direkt.pdf
[2023-08-06 08:41:11,234] [INFO] [ocrmypdf._sync] Start processing 2 pages concurrently
[2023-08-06 08:41:11,295] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,297] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,299] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,303] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,305] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,305] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,305] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,306] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,307] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,308] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
[2023-08-06 08:41:11,337] [INFO] [ocrmypdf._sync] Postprocessing...
[2023-08-06 08:41:11,725] [ERROR] [ocrmypdf._exec.ghostscript] GPL Ghostscript 10.0.0 (2022-09-21)
Copyright (C) 2022 Artifex Software, Inc.  All rights reserved.
This software is supplied under the GNU AGPLv3 and comes with NO WARRANTY:
see the file COPYING for details.
GPL Ghostscript 10.00.0: Text string detected in DOCINFO cannot be represented in XMP for PDF/A1, discarding DOCINFO
GPL Ghostscript 10.00.0: Text string detected in DOCINFO cannot be represented in XMP for PDF/A1, discarding DOCINFO
Error: /typecheck in --runpdf--
Operand stack:
   --nostringval--   --nostringval--   --nostringval--   --nostringval--
Execution stack:
   %interp_exit   .runexec2   --nostringval--   runpdf   --nostringval--   2   %stopped_push   --nostringval--   runpdf   runpdf   false   1   %stopped_push   1990   1   3   %oparray_pop   1989   1   3   %oparray_pop   1977   1   3   %oparray_pop   1978   1   3   %oparray_pop   runpdf   runpdf   runpdf   runpdf
Dictionary stack:
   --dict:777/1123(ro)(G)--   --dict:0/20(G)--   --dict:76/200(L)--   --dict:18/20(L)--
Current allocation mode is local
GPL Ghostscript 10.00.0: Unrecoverable error, exit code 1

[2023-08-06 08:41:11,751] [ERROR] [paperless.consumer] Error while consuming document 201812_Kontoauszug_1822direkt.pdf: SubprocessOutputError: Ghostscript PDF/A rendering failed
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_exec/ghostscript.py", line 232, in generate_pdfa
    p = run_polling_stderr(
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/subprocess/__init__.py", line 114, in run_polling_stderr
    raise CalledProcessError(proc.returncode, args, output=None, stderr=stderr)
subprocess.CalledProcessError: Command '['gs', '-dBATCH', '-dNOPAUSE', '-dSAFER', '-dCompatibilityLevel=1.6', '-sDEVICE=pdfwrite', '-dAutoRotatePages=/None', '-sColorConversionStrategy=LeaveColorUnchanged', '-dPDFSTOPONERROR', '-dAutoFilterColorImages=true', '-dAutoFilterGrayImages=true', '-dJPEGQ=95', '-dPDFA=2', '-dPDFACompatibilityPolicy=1', '-o', '-', '-sstdout=%stderr', '/tmp/ocrmypdf.io.afgv3tys/fix_docinfo.pdf', '/tmp/ocrmypdf.io.afgv3tys/pdfa.ps']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 320, in parse
    ocrmypdf.ocr(**args)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/api.py", line 347, in ocr
    return run_pipeline(options=options, plugin_manager=plugin_manager, api=True)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 391, in run_pipeline
    optimize_messages = exec_concurrent(context, executor)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 308, in exec_concurrent
    pdf, messages = post_process(pdf, context, executor)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 239, in post_process
    pdf_out = convert_to_pdfa(pdf_out, ps_stub_out, context)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_pipeline.py", line 789, in convert_to_pdfa
    context.plugin_manager.hook.generate_pdfa(
  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 116, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/builtin_plugins/ghostscript.py", line 77, in generate_pdfa
    ghostscript.generate_pdfa(
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_exec/ghostscript.py", line 246, in generate_pdfa
    raise SubprocessOutputError('Ghostscript PDF/A rendering failed') from e
ocrmypdf.exceptions.SubprocessOutputError: Ghostscript PDF/A rendering failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/paperless/src/documents/consumer.py", line 424, in try_consume_file
    document_parser.parse(self.path, mime_type, self.filename)
  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 377, in parse
    raise ParseError(f"{e.__class__.__name__}: {e!s}") from e
documents.parsers.ParseError: SubprocessOutputError: Ghostscript PDF/A rendering failed
[2023-08-06 08:41:12,332] [ERROR] [celery.app.trace] Task documents.tasks.consume_file[8536feb8-15f8-4720-a8df-20a2894129f3] raised unexpected: ConsumerError('201812_Kontoauszug_1822direkt.pdf: Error while consuming document 201812_Kontoauszug_1822direkt.pdf: SubprocessOutputError: Ghostscript PDF/A rendering failed')
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_exec/ghostscript.py", line 232, in generate_pdfa
    p = run_polling_stderr(
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/subprocess/__init__.py", line 114, in run_polling_stderr
    raise CalledProcessError(proc.returncode, args, output=None, stderr=stderr)
subprocess.CalledProcessError: Command '['gs', '-dBATCH', '-dNOPAUSE', '-dSAFER', '-dCompatibilityLevel=1.6', '-sDEVICE=pdfwrite', '-dAutoRotatePages=/None', '-sColorConversionStrategy=LeaveColorUnchanged', '-dPDFSTOPONERROR', '-dAutoFilterColorImages=true', '-dAutoFilterGrayImages=true', '-dJPEGQ=95', '-dPDFA=2', '-dPDFACompatibilityPolicy=1', '-o', '-', '-sstdout=%stderr', '/tmp/ocrmypdf.io.afgv3tys/fix_docinfo.pdf', '/tmp/ocrmypdf.io.afgv3tys/pdfa.ps']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 320, in parse
    ocrmypdf.ocr(**args)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/api.py", line 347, in ocr
    return run_pipeline(options=options, plugin_manager=plugin_manager, api=True)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 391, in run_pipeline
    optimize_messages = exec_concurrent(context, executor)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 308, in exec_concurrent
    pdf, messages = post_process(pdf, context, executor)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 239, in post_process
    pdf_out = convert_to_pdfa(pdf_out, ps_stub_out, context)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_pipeline.py", line 789, in convert_to_pdfa
    context.plugin_manager.hook.generate_pdfa(
  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 116, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/builtin_plugins/ghostscript.py", line 77, in generate_pdfa
    ghostscript.generate_pdfa(
  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_exec/ghostscript.py", line 246, in generate_pdfa
    raise SubprocessOutputError('Ghostscript PDF/A rendering failed') from e
ocrmypdf.exceptions.SubprocessOutputError: Ghostscript PDF/A rendering failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/asgiref/sync.py", line 349, in main_wrap
    raise exc_info[1]
  File "/usr/src/paperless/src/documents/consumer.py", line 424, in try_consume_file
    document_parser.parse(self.path, mime_type, self.filename)
  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 377, in parse
    raise ParseError(f"{e.__class__.__name__}: {e!s}") from e
documents.parsers.ParseError: SubprocessOutputError: Ghostscript PDF/A rendering failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 760, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/src/paperless/src/documents/tasks.py", line 156, in consume_file
    document = Consumer().try_consume_file(
  File "/usr/src/paperless/src/documents/consumer.py", line 454, in try_consume_file
    self._fail(
  File "/usr/src/paperless/src/documents/consumer.py", line 108, in _fail
    raise ConsumerError(f"{self.filename}: {log_message or message}") from exception
documents.consumer.ConsumerError: 201812_Kontoauszug_1822direkt.pdf: Error while consuming document 201812_Kontoauszug_1822direkt.pdf: SubprocessOutputError: Ghostscript PDF/A rendering failed

Browser logs

No response

Paperless-ngx version

1.17

Host OS

Debian Docker and Unraid Docker

Installation method

Docker - official image

Browser

No response

Configuration changes

No response

Other

No response

[amigappc2023]

i switched back to 1.16.5 and now it works again.
so i think, there is something with the docker image

[Samurai1201]

i encountered the same problem, but when i removed some pdf/a conformity stamp in the documents (pops up on top of document when opened via adobe acrobat) in the documents by hand it worked again.
until this is fixed i will just switch back to 1.16.5 where i never had this issue

[stumpylog]

I suggest looking into the issues at ocrmypdf. It's the one actually doing anything with ghostscript, not paperless, so fixes would need to be from upstream.

[stumpylog]

Also, without a sample file, it's really hard to even begin poking around in either paperless or ocrmypdf to investigate. PDFs are surprisingly not very standardized

[amigappc2023]

Also, without a sample file, it's really hard to even begin poking around in either paperless or ocrmypdf to investigate. PDFs are surprisingly not very standardized

for sure, i tried with several pdf files and no one works, since im going back to 1.16.5 i have no issue no longer, everything works fine now again.

so maybe there is something wrong with the 1.17 container. if its not right here, do you have a sugesstion, where i have to adress the problem ? thx in advance.

i put a file in as an attachment, its an closed bank account, so im not worry about spreading the information.

200706_Kontoauszug_1822direkt.pdf

[shamoon]

I had no issue consuming that file:

[amigappc2023]

i understand.

i got on 2 diff. systems the same problem.
i jumped back to 1.16.5 and now i dont have the problem anylonger.

so then im close the problem and i will stay on 1.16.5 because no one got the issue.
i hope with a new version , i can upgrade, if not, i will stay on 1.16.5

[wodan66]

And this happens with another file:

[2023-08-06 19:56:22,988] [INFO] [paperless.consumer] Consuming Onlineabschluss_Waermepumpe_Natur12_Preview_13.07.2023_12_45.pdf

[2023-08-06 19:56:22,990] [DEBUG] [paperless.consumer] Detected mime type: application/pdf

[2023-08-06 19:56:22,991] [DEBUG] [paperless.consumer] Parser: RasterisedDocumentParser

[2023-08-06 19:56:22,994] [DEBUG] [paperless.consumer] Parsing Onlineabschluss_Waermepumpe_Natur12_Preview_13.07.2023_12_45.pdf...

[2023-08-06 19:56:23,168] [DEBUG] [paperless.parsing.tesseract] Calling OCRmyPDF with args: {'input_file': PosixPath('/tmp/paperless/paperless-ngxxjz78923/Onlineabschluss_Waermepumpe_Natur12_Preview_13.07.2023_12_45.pdf'), 'output_file': PosixPath('/tmp/paperless/paperless-p_hmhx70/archive.pdf'), 'use_threads': True, 'jobs': 4, 'language': 'deu', 'output_type': 'pdfa', 'progress_bar': False, 'skip_text': True, 'clean': True, 'deskew': True, 'rotate_pages': True, 'rotate_pages_threshold': 12.0, 'sidecar': PosixPath('/tmp/paperless/paperless-p_hmhx70/sidecar.txt')}

[2023-08-06 19:56:23,747] [DEBUG] [paperless.parsing.tesseract] Deleting directory /tmp/paperless/paperless-p_hmhx70

[2023-08-06 19:56:23,751] [ERROR] [paperless.consumer] Error while consuming document Onlineabschluss_Waermepumpe_Natur12_Preview_13.07.2023_12_45.pdf: SubprocessOutputError: Ghostscript PDF/A rendering failed

Traceback (most recent call last):

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_exec/ghostscript.py", line 232, in generate_pdfa

    p = run_polling_stderr(

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/subprocess/__init__.py", line 114, in run_polling_stderr

    raise CalledProcessError(proc.returncode, args, output=None, stderr=stderr)

subprocess.CalledProcessError: Command '['gs', '-dBATCH', '-dNOPAUSE', '-dSAFER', '-dCompatibilityLevel=1.6', '-sDEVICE=pdfwrite', '-dAutoRotatePages=/None', '-sColorConversionStrategy=LeaveColorUnchanged', '-dPDFSTOPONERROR', '-dAutoFilterColorImages=true', '-dAutoFilterGrayImages=true', '-dJPEGQ=95', '-dPDFA=2', '-dPDFACompatibilityPolicy=1', '-o', '-', '-sstdout=%stderr', '/tmp/ocrmypdf.io.h5j6ox7w/fix_docinfo.pdf', '/tmp/ocrmypdf.io.h5j6ox7w/pdfa.ps']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 320, in parse

    ocrmypdf.ocr(**args)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/api.py", line 347, in ocr

    return run_pipeline(options=options, plugin_manager=plugin_manager, api=True)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 391, in run_pipeline

    optimize_messages = exec_concurrent(context, executor)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 308, in exec_concurrent

    pdf, messages = post_process(pdf, context, executor)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_sync.py", line 239, in post_process

    pdf_out = convert_to_pdfa(pdf_out, ps_stub_out, context)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_pipeline.py", line 789, in convert_to_pdfa

    context.plugin_manager.hook.generate_pdfa(

  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__

    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)

  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec

    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)

  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 116, in _multicall

    raise exception.with_traceback(exception.__traceback__)

  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall

    res = hook_impl.function(*args)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/builtin_plugins/ghostscript.py", line 77, in generate_pdfa

    ghostscript.generate_pdfa(

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_exec/ghostscript.py", line 246, in generate_pdfa

    raise SubprocessOutputError('Ghostscript PDF/A rendering failed') from e

ocrmypdf.exceptions.SubprocessOutputError: Ghostscript PDF/A rendering failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/consumer.py", line 424, in try_consume_file

    document_parser.parse(self.path, mime_type, self.filename)

  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 377, in parse

    raise ParseError(f"{e.__class__.__name__}: {e!s}") from e

documents.parsers.ParseError: SubprocessOutputError: Ghostscript PDF/A rendering failed

[shamoon]

You can see the exact arguments with which ocrmypdf is getting called, you can try that directly but I'm not sure why this wouldn't be easily reproducible between docker installs.

Regardless, if it can be reproduced directly against ocrmypdf the issue should be raised upstream

[amigappc2023]

You can see the exact arguments with which ocrmypdf is getting called, you can try that directly but I'm not sure why this wouldn't be easily reproducible between docker installs.

Regardless, if it can be reproduced directly against ocrmypdf the issue should be raised upstream

i closed it because i saw it comming.

like i wrote, i have an unraid system on an asrock 1805 and an promox with an J4105 (Lenovo Futro S740) and BOTH are giving the SAME problem.

so i thought it have something todo with the made container

Why i am thought so ?!
Because im back on 1.16.5 and the problems are gone.

so a lot hearing, yeah, it have nothing todo with that, so i closed it.
i hope for the next release and if its not solving the issue on my end, i will stuck on 1.16.5 or find an other solution.

p.s.:
i had it on 200 diff. pdfs. from different banks and all are generated by the bank system itself.

[wodan66]

I just installed a fresh LXC with debian bookworm, updated it, installed docker and paperless.
I can confirm I had the issues with scanning several PDF. Installing ver 1.16.5 everything is fine and all problems disappear.
The issue is related to ghostscript. Unfortunately the PDF I have the issues with have personal information, I am not willing to share. As far as I see, selfscanned PDF are ok, it is related to PDF I received from companies.

[stumpylog]

I also can't reproduce it with the one sample file about. Any configuration changes related to OCR or archive file creation?

[amigappc2023]

I also can't reproduce it with the one sample file about. Any configuration changes related to OCR or archive file creation?

hello,

i just repulled the docker image. nothing else changed.

[kennnyshiwa]

Adding on to this, ran into this exact issue on 1.17 today. With the following PDF X570D4U

I also did confirm that when running this PDF against raw OCRMYPDF that the error does occur as well.

[stumpylog]

If you add the option --continue-on-soft-render-error to the OCRMyPDF call does it work? But is the resulting PDF rendered correctly?

[BWibo]

Same here with this PDF on Raspberry Pi4, 4 GB, Ubuntu 22 x64, paperless-ngx v1.17.0.
Allgemeine Versicherungsbedingungen.pdf

[Mavarius1989]

It happens just to a few pdf files here. I then opended the file with the acrobat reader, and "printed it" with the microsoft pdf printer. No problems with the files in paperless after that.

[shamoon]

Anyone who can reproduce this against ocrmypdf directly it would be good to open an issue there https://github.com/ocrmypdf/OCRmyPDF/issues/ , that way users can comment and there is a chance of it being fixed. Again, we cannot fix this in the paperless-ngx repo