feat: support ocr for images using llm

[lukasdotcom]

No description provided.

[kyteinsky]

would be nice if we can also support pdfs through pdf -> jpg conversion. The rest of the path would remain the same.
a small snippet that uses imagick to do this (which preview generator of NC also uses with the imagick php extension):
(ai generated)

private function pdfToJpegFiles(string $pdfPath, int $dpi = 200, int $maxPages = 500): array {
    if (!extension_loaded('imagick')) {
        throw new \RuntimeException('Imagick extension not available');
    }
    if (empty(\Imagick::queryFormats('PDF'))) {
        throw new \RuntimeException('Imagick has no PDF support (Ghostscript missing or blocked by policy.xml)');
    }

    $probe = new \Imagick();
    $probe->pingImage($pdfPath);
    $pageCount = min($probe->getNumberImages(), $maxPages);
    $probe->clear();

    $paths = [];
    for ($i = 0; $i < $pageCount; $i++) {
        $im = new \Imagick();
        $im->setResolution($dpi, $dpi); // before readImage
        $im->readImage($pdfPath . '[' . $i . ']');
        $im->setImageBackgroundColor('white');
        $im = $im->mergeImageLayers(\Imagick::LAYERMETHOD_FLATTEN); // JPEG has no alpha
        $im->setImageFormat('jpeg');
        $im->setImageCompressionQuality(85);

        $tmpPath = $this->tempManager->getTemporaryFile('.jpg');
        $im->writeImage($tmpPath);
        $im->clear();

        $paths[] = $tmpPath;
    }
    return $paths;
}

for the 500 MB php worker memory limit, we can impose a max limit of 500 pages of pdf in the task type as it is now and convert them to jpgs like above storing them in the disk.
after this the chat completion requests could run with a batch size of 50 converting 50 images to base64 base64_encode(file_get_contents($path), run the OCR and collect the texts.
wdyt?

[lukasdotcom]

I added some code Imagick isn't preinstalled in julius's docker so you'll need to install it manually /usr/local/bin/install-php-extensions imagick (run that in the docker container)
I kept the batch size a lot smaller than this with 5 as that was actually faster than a larger batch size. Honestly it might be worth it to not batch at all as the accuracy goes up with smaller batches at only a very small cost in time (revert the newest commit). Most of the tokens are used by the image anyway.

Batch Size	Time (From one attempt)
1	2:29
2	2:02
5	1:54
50	2:02

wdyt?

[julien-nc]

Damn I didn't see this PR and did it again (without PDF support). Can you check the enh/noid/ocr-provider branch and replicate the streaming support? (the provider implements ISynchronousOptionsAwareProvider and reports the output)

[julien-nc]

I hope you don't mind: I rebased on main and added streaming support.

[lukasdotcom]

I hope you don't mind: I rebased on main and added streaming support.

I've waited to finish up on this branch until multimodality (#400) is done so I can reuse work from there.