Replies: 11 comments 8 replies
|
Running into the same issue with both 0.3.11 and 0.3.12 docker images uploading pdf files in prompt:
|
|
I'm using openai embeddings(text-embedding-3-small) and I have the same error. also docker and ubuntu |
|
Found '#1146' that reccomends turning off OCR. It worked for me |
|
Thanks @frankgestrada - turning off OCR option worked for us too, much appreciated! |
|
I don't think this can be the solution to our problem, but rather a way around it.For good RAG we need the use of OCR, and the fact that for some types of open-webui documents it doesn't work demonstrates limitations that we should be solving |
|
I don't think this can be the solution to our problem, but rather a way around it.For good RAG we need the use of OCR, and the fact that for some types of open-webui documents it doesn't work demonstrates limitations that we should be solving.Put it two times to make the issue more visible |
|
Hi, same problem for me as well. I noticed that the problem occurs only when I try to upload a PDF document with very large images. For example if I load the 77 page document I get the error “cannot reshape array of size 7818 into shape (2339,1656,newaxis)”. While if I load the same PDF in reduced size of the first 8 pages it seems to work correctly. This behavior regardless of the size of the template (7B,8B or 70B). Does anyone have any ideas? |
|
I think that was solved in dec/24 by langchain commit: langchain-ai/langchain@d49df48 |
|
Actually on a fresh install with docker i have langchain lib 0.3.7 . I wonder, is it safe and stable to make such a major upgrade manually? Shouldn't it be upgraded directly in the container image? |
|
I can't help about, my config is cloned openwebui with python 3.12 & all libs updated (a little bit tweaked), it run without errors (no more than usual). (there was other open issue about: #8877 ) |
|
Okay it's all very strange. Technically using docker should also update the various libraries. Otherwise it becomes really difficult to use and debug. |
|
Update: change the requirements to version 0.3.13 for langchain But the problem persists, with the same error log. Do you have any other ideas? |
|
try with last versions: 0.3.20 for langchain, has just been released, & 0.5.19 for langchain-community |
|
with tika work perfect, but with own ocr I have same error: |
|
disabling OCR error dissapear, |
|
I was trying with pymupdf, it work perfectly: Commit: & replace imports in requirements.txt & pyproject.toml NOTE: by the way, python lib pymupdf4llm convert pdfs to markdown, maybe it could be better for inject files to chat |
|
That Pull Request was rejected by licenses problems, I made another with unstructured pdf lib: #11240 It seem if do the work, i lightly faster |
|
same here |
Uh oh!
There was an error while loading. Please reload this page.
Bug Report
During the processing of a PDF document, an error occurs when attempting to reshape an image array. The system is unable to reshape an array of size 294 into the required shape of (300, 152, newaxis).
Installation Method**
Direct installation, not Docker, pip
Environment
Open WebUI Version: v0.3.11
Ollama (if applicable): v0.2.0
Operating System: Ubuntu 20.04
Browser (if applicable): Chrome 100.0
Confirmation:
Expected Behavior:
The file should be uploaded in the section Documents without problem
Actual Behavior:
The process halts with a ValueError when trying to reshape the image data extracted from the PDF.
Description
Bug Summary:
[Provide a brief but clear summary of the bug]
Reproduction Details
Embedding model: sentence-transformersparaphrase-multilingual-MiniLM-L12-v2
Reranker:BBAI-reranker-v2-m3
upload in documents the attach file "ESA6march2009"
Steps to Reproduce:
embedding model
Logs and Screenshots
Something went wrong :/ cannot reshape array of size 294 into shape (300,152,newaxis)
ERROR:apps.rag.main:cannot reshape array of size 294 into shape (300,152,newaxis)
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/apps/rag/main.py", line 1241, in process_doc
data = loader.load()
^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/langchain_core/document_loaders/base.py", line 30, in load
return list(self.lazy_load())
^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/langchain_community/document_loaders/pdf.py", line 202, in lazy_load
yield from self.parser.parse(blob)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/langchain_core/document_loaders/base.py", line 126, in parse
return list(self.lazy_parse(blob))
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/langchain_community/document_loaders/parsers/pdf.py", line 124, in lazy_parse
yield from [
^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/langchain_community/document_loaders/parsers/pdf.py", line 127, in
+ self._extract_images_from_page(page),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/site-packages/langchain_community/document_loaders/parsers/pdf.py", line 146, in _extract_images_from_page
np.frombuffer(xObject[obj].get_data(), dtype=np.uint8).reshape(
ValueError: cannot reshape array of size 294 into shape (300,152,newaxis)
INFO: 127.0.0.1:54686 - "POST /rag/api/v1/process/doc HTTP/1.1" 400 Bad Request
Additional Information
The error occurs while using the langchain library for processing PDF documents.
The issue appears to be related to image extraction and processing from the PDF.
The system is attempting to reshape an array of size 294 into a shape of (300, 152, newaxis), which is not mathematically possible.
This resulted in a 400 Bad Request response for the POST request to "/rag/api/v1/process/doc".
ECSS-Q-ST-70C_(6March2009).pdf
ECSS-Q-ST-70C_(6March2009).pdf
All reactions