[Feature Request] add tika 4 support for future compatibility

[binderth]

Description

Description

Since a few days my email and Office documents stopped being consumed and stopped with an error in the task lists (especially since I just forward Emails to paperless)
Everything else looked healthy: all containers including tika were up, tika answered on port 9998, the health check was green, and I had not changed anything in my setup. The only sign of trouble was this in the failed tasks list:

Error occurred while consuming document <name>.eml: Could not parse content with
tika server at http://tika:9998/: Client error '406 Not Acceptable' for url
'http://tika:9998/tika/text'

after digging into the logs I figured, tika was updated to 4.0.0.0 somewhere the last days (as I had tika:latest in my compose.yaml). After searching for changes from 3.x to 4.x it seems tika now needs explicit arguments and and only reports a JSON back on /tika/json, a call to /tika/text with application/json results in said 406-errors.

Feature Request

So, my request would be to perhaps add a little information in the docs or perhaps on startup that currently only tika 3.3.x is supported. In the longer term I think, tika 4.x has to be considered as well.

Other

No response

[stumpylog]

We're aware but it won't be support for both. Tika v4 was a major breaking change

[schultzter]

I see that the compose.yaml was changed a couple weeks ago to pin tika to 3.3.1.0.
d7b3612

How do I stay aware of stuff like this? I guess it's a more general question than just Paperless-ngx, but this is the only complex compose file I have - the others just start one container so the updates and dependancies are limited.

[shamoon]

There isn’t that much of a mechanism for us to “announce” stuff like that, but we will try to include it in our release notes. To be fair, it’s essentially the first time it’s ever happened, and not all things like this were anticipated when the compose files were setup years ago.

Going forward the compose files will use specific versions (maybe even for paperless itself), that will keep us aware of specific updates when they are proposed by eg dependabot

[schultzter]

For the moment I've found https://app.github-file-watcher.com so I can watch the compose file I've based my installation on. I'll probably set that up for all the apps I'm self-hosting.

[shamoon]

Cool

[binderth]

Mentioning changes not only within the core but also for setup instructions and the docker compose would be great within the release notes! 😉
Honestly I didn't check the history of the compose yaml - but found out, that 3.3.1.0 was pinned and I thought "oh, I must've changed that to latest?"... 🤷‍♀️

having it as "breaking changes" could at least alert me of checking my environment! 🤟