TIKA-2696 Add support for OSD output, contributed by @4U6U57

[4U6U57]

See https://issues.apache.org/jira/browse/TIKA-2696

Contributed by @4U6U57 on behalf of @egnyte

[tballison]

Thank you for opening this PR. Would you be able to include a unit test?

[4U6U57]

@tballison Added test.

[gdubicki]

Hi @tballison , can you please re-review after the tests were added?

[tballison]

Sorry for my delay.

Three things.

1. What would you think of "parsing" the output into metadata fields? That info feels to me far more like metadata than "content".
2. Do we know at what version of tesseract, the osd info is written to an .osd file rather than to stdout? On my ancient version (see below), that info is dumped to stderr, not written to an osd file. Should we just rely on users having a more modern version that writes an osd file?
3. If we do parse this info, do we have a sense of how much it changes across versions?

This is what I see with an old version of Tesseract:
Tesseract Open Source OCR Engine v3.04.00 with Leptonica
Orientation: 0
Orientation in degrees: 0
Orientation confidence: 28.65
Script: 1
Script confidence: 144.00

[tballison]

And from the Windows build that is hot off the press (the June 2018 beta version threw an exception with -psm 0).

tesseract v4.0.0-rc1.20181008

Page number: 0
Orientation in degrees: 0
Rotate: 0
Orientation confidence: 21.57
Script: Latin
Script confidence: 2.84

[4U6U57]

1. Interesting point in regards to Metadata, I think that it makes sense to go that route. I started this project initially with the intention of specifically supporting psm 0, which does not output any extracted content, but putting it in metadata would be more beneficial to other modes that also output OSD information.
2. I haven't had a chance to test this on older versions, I was planning to take a look over the weekend, but in my tesseract 4.0.0-beta.1 (latest on Ubuntu 18.04) and psm 0 I get the OSD info in stdout, not stderr, if the output format is specified as stdout, and in the $out.osd if output is $out. In all other modes I don't get OSD info.\

OSD Output Testing

[avalera:~/test]
$ ls
test.jpg

[avalera:~/test]
$ tesseract test.jpg stdout --psm 0 -l osd
Warning. Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 227
Page number: 0
Orientation in degrees: 0
Rotate: 0
Orientation confidence: 16.99
Script: Latin
Script confidence: 0.16

[avalera:~/test]
$ tesseract test.jpg stdout --psm 0 -l osd 2>/dev/null
Page number: 0
Orientation in degrees: 0
Rotate: 0
Orientation confidence: 16.99
Script: Latin
Script confidence: 0.16

[avalera:~/test]
$ tesseract test.jpg stdout --psm 1
Warning. Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 227
 

CHAPTER 5

IN THE MIDDLE OF THE NIGHT

Tue next day Aunt Hetty took Pam and Peter and
their cousin Brock in the pony-cart to the sea,
which was about three miles away. This was such
fun that the three children forgot all about Cliff
Castle for a day or two. And then something hap-
pened that reminded them of it.

It was something that happened in the middle
of the night. Pam woke up and felt very thirsty.
She remembered that Aunt Hetty had left a jug
of water and a tumbler on the mantelpiece and
she got up to get it.

She stood at the window, drinking the water. It

39


[avalera:~/test]
$ ls
test.jpg

[avalera:~/test]
$ tesseract test.jpg outfile --psm 0 -l osd
Tesseract Open Source OCR Engine v4.0.0-beta.1 with Leptonica
Warning. Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 227

[avalera:~/test]
$ ls
outfile.osd  test.jpg

[avalera:~/test]
$ rm outfile.osd 

[avalera:~/test]
$ tesseract test.jpg outfile --psm 1
Tesseract Open Source OCR Engine v4.0.0-beta.1 with Leptonica
Warning. Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 227

[avalera:~/test]
$ ls
outfile.txt  test.jpg

3. Same as previous, haven't tested but looking at your two examples I'd imagine not much change except for the addition of new keys.

[algo-robin]

Is there any chance of someone completing this fix?

[tballison]

I've made the fix in 2.1.1-SNAPSHOT. Please give it a try and make sure it behaves as desired.

[algo-robin]

Will do. Where can I get this version?

[tballison]

If you're using tika-app, try this: https://repository.apache.org/content/repositories/snapshots/org/apache/tika/tika-app/2.1.1-SNAPSHOT/tika-app-2.1.1-20211020.153421-53.jar

[algo-robin]

I have verified the fix. It works very nicely. Thanks a lot!

[tballison]

Thank you for the confirmation!

[Tarik37]

Hello, I am currently using the Tika 2.9.1 server version and need the output of the OSD in my metadata, particularly the value of the script (Latin, Cyrillic, etc.). So my questions are the following:
Does my server version of Tika integrate it? Is it possible?
If yes, how can I configure my Tika server?
Thanks for your work (and also english is not m'y native language)

[THausherr]

This is a closed issue from years ago, please ask this in the user's mailing list (don't forget to subscribe) or on stackoverflow.com.