Only tika-core ships in the utilities bundles, and tika-core registers no parser at all, not even for text/plain. Running the bundle as documented therefore produces parse_status=EMPTY for every file, empty extracted_text, no chunks, and a null vector on every row, with the job exiting 0 and logging nothing.
EMPTY is also the normal outcome for an image, so nothing distinguishes "this file has no text" from "this deployment cannot extract text at all".
Measured on a 2 GB corpus of 616 text and 616 binary files:
| classpath |
parse_status |
chunks |
| bundle as shipped |
EMPTY = 1232 |
0 |
plus tika-parsers-standard-package |
SUCCESS 554, TRUNCATED 62, EMPTY 616 |
115,873 |
The 616 EMPTY in the second row are exactly the binary files, which is correct. In the first row every text file was empty too.
The documented remedy, supplying the parsers at runtime with spark-submit --packages, does not work reliably. Ivy failed to resolve three of Tika's transitive dependencies. Passing the jars with --jars instead gets further and then throws NoSuchMethodError: GzipUtils.getUncompressedFileName, because Spark ships commons-compress 1.23.0 and --jars does not take precedence over Spark's own jars.
That last failure is reached in practice rather than in theory. Tika detects format by magic number, and arbitrary binary content occasionally matches a compressed signature. A 1,232-file corpus never hit it; a 64,487-file corpus did. At document-lake scale it is a certainty.
The same applies to metadata, which is easy to miss: a JPEG through TikaDocumentParser yields 25 metadata fields when the parser modules are present, and nothing at all without them.
Only
tika-coreships in the utilities bundles, andtika-coreregisters no parser at all, not even fortext/plain. Running the bundle as documented therefore producesparse_status=EMPTYfor every file, emptyextracted_text, no chunks, and a null vector on every row, with the job exiting 0 and logging nothing.EMPTYis also the normal outcome for an image, so nothing distinguishes "this file has no text" from "this deployment cannot extract text at all".Measured on a 2 GB corpus of 616 text and 616 binary files:
tika-parsers-standard-packageThe 616 EMPTY in the second row are exactly the binary files, which is correct. In the first row every text file was empty too.
The documented remedy, supplying the parsers at runtime with
spark-submit --packages, does not work reliably. Ivy failed to resolve three of Tika's transitive dependencies. Passing the jars with--jarsinstead gets further and then throwsNoSuchMethodError: GzipUtils.getUncompressedFileName, because Spark ships commons-compress 1.23.0 and--jarsdoes not take precedence over Spark's own jars.That last failure is reached in practice rather than in theory. Tika detects format by magic number, and arbitrary binary content occasionally matches a compressed signature. A 1,232-file corpus never hit it; a 64,487-file corpus did. At document-lake scale it is a certainty.
The same applies to metadata, which is easy to miss: a JPEG through
TikaDocumentParseryields 25 metadata fields when the parser modules are present, and nothing at all without them.