Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Trying to run it dockerized and getting "HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url" #1872

Closed
BenBatsir opened this issue Apr 21, 2024 · 18 comments
Closed

Trying to run it dockerized and getting "HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url" #1872

BenBatsir opened this issue Apr 21, 2024 · 18 comments

Comments

@BenBatsir
Copy link

It works perfectly when running it locally (without docker), but when I tried to import a doc via UI (inside a container), I'm getting the bellow:

Generating embeddings:   0%|          | 0/23 [00:00<?, ?it/s]Traceback (most recent call last):
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
privategpt-private-gpt-1  |     conn = connection.create_connection(
privategpt-private-gpt-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
privategpt-private-gpt-1  |     raise err
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
privategpt-private-gpt-1  |     sock.connect(sa)
privategpt-private-gpt-1  | ConnectionRefusedError: [Errno 111] Connection refused
privategpt-private-gpt-1  |
privategpt-private-gpt-1  | During handling of the above exception, another exception occurred:
privategpt-private-gpt-1  |
privategpt-private-gpt-1  | Traceback (most recent call last):
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 715, in urlopen
privategpt-private-gpt-1  |     httplib_response = self._make_request(
privategpt-private-gpt-1  |                        ^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 416, in _make_request
privategpt-private-gpt-1  |     conn.request(method, url, **httplib_request_kw)
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
privategpt-private-gpt-1  |     super(HTTPConnection, self).request(method, url, body=body, headers=headers)
privategpt-private-gpt-1  |   File "/usr/local/lib/python3.11/http/client.py", line 1286, in request
privategpt-private-gpt-1  |     self._send_request(method, url, body, headers, encode_chunked)
privategpt-private-gpt-1  |   File "/usr/local/lib/python3.11/http/client.py", line 1332, in _send_request
privategpt-private-gpt-1  |     self.endheaders(body, encode_chunked=encode_chunked)
privategpt-private-gpt-1  |   File "/usr/local/lib/python3.11/http/client.py", line 1281, in endheaders
privategpt-private-gpt-1  |     self._send_output(message_body, encode_chunked=encode_chunked)
privategpt-private-gpt-1  |   File "/usr/local/lib/python3.11/http/client.py", line 1041, in _send_output
privategpt-private-gpt-1  |     self.send(msg)
privategpt-private-gpt-1  |   File "/usr/local/lib/python3.11/http/client.py", line 979, in send
privategpt-private-gpt-1  |     self.connect()
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 205, in connect
privategpt-private-gpt-1  |     conn = self._new_conn()
privategpt-private-gpt-1  |            ^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
privategpt-private-gpt-1  |     raise NewConnectionError(
privategpt-private-gpt-1  | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0xffff609ddf10>: Failed to establish a new connection: [Errno 111] Connection refused
privategpt-private-gpt-1  |
privategpt-private-gpt-1  | During handling of the above exception, another exception occurred:
privategpt-private-gpt-1  |
privategpt-private-gpt-1  | Traceback (most recent call last):
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
privategpt-private-gpt-1  |     resp = conn.urlopen(
privategpt-private-gpt-1  |            ^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 799, in urlopen
privategpt-private-gpt-1  |     retries = retries.increment(
privategpt-private-gpt-1  |               ^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
privategpt-private-gpt-1  |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
privategpt-private-gpt-1  | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff609ddf10>: Failed to establish a new connection: [Errno 111] Connection refused'))
privategpt-private-gpt-1  |
privategpt-private-gpt-1  | During handling of the above exception, another exception occurred:
privategpt-private-gpt-1  |
privategpt-private-gpt-1  | Traceback (most recent call last):
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/queueing.py", line 495, in call_prediction
privategpt-private-gpt-1  |     output = await route_utils.call_process_api(
privategpt-private-gpt-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/route_utils.py", line 235, in call_process_api
privategpt-private-gpt-1  |     output = await app.get_blocks().process_api(
privategpt-private-gpt-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1627, in process_api
privategpt-private-gpt-1  |     result = await self.call_function(
privategpt-private-gpt-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1173, in call_function
privategpt-private-gpt-1  |     prediction = await anyio.to_thread.run_sync(
privategpt-private-gpt-1  |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/to_thread.py", line 33, in run_sync
privategpt-private-gpt-1  |     return await get_asynclib().run_sync_in_worker_thread(
privategpt-private-gpt-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
privategpt-private-gpt-1  |     return await future
privategpt-private-gpt-1  |            ^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 807, in run
privategpt-private-gpt-1  |     result = context.run(func, *args)
privategpt-private-gpt-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 690, in wrapper
privategpt-private-gpt-1  |     response = f(*args, **kwargs)
privategpt-private-gpt-1  |                ^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/private_gpt/ui/ui.py", line 252, in _upload_file
privategpt-private-gpt-1  |     self._ingest_service.bulk_ingest([(str(path.name), path) for path in paths])
privategpt-private-gpt-1  |   File "/home/worker/app/private_gpt/server/ingest/ingest_service.py", line 87, in bulk_ingest
privategpt-private-gpt-1  |     documents = self.ingest_component.bulk_ingest(files)
privategpt-private-gpt-1  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/private_gpt/components/ingest/ingest_component.py", line 135, in bulk_ingest
privategpt-private-gpt-1  |     saved_documents.extend(self._save_docs(documents))
privategpt-private-gpt-1  |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/private_gpt/components/ingest/ingest_component.py", line 142, in _save_docs
privategpt-private-gpt-1  |     self._index.insert(document, show_progress=True)
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/indices/base.py", line 225, in insert
privategpt-private-gpt-1  |     nodes = run_transformations(
privategpt-private-gpt-1  |             ^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/ingestion/pipeline.py", line 127, in run_transformations
privategpt-private-gpt-1  |     nodes = transform(nodes, **kwargs)
privategpt-private-gpt-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 334, in __call__
privategpt-private-gpt-1  |     embeddings = self.get_text_embedding_batch(
privategpt-private-gpt-1  |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 255, in get_text_embedding_batch
privategpt-private-gpt-1  |     embeddings = self._get_text_embeddings(cur_batch)
privategpt-private-gpt-1  |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/embeddings/ollama/base.py", line 64, in _get_text_embeddings
privategpt-private-gpt-1  |     embeddings = self.get_general_text_embedding(text)
privategpt-private-gpt-1  |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/embeddings/ollama/base.py", line 89, in get_general_text_embedding
privategpt-private-gpt-1  |     response = requests.post(
privategpt-private-gpt-1  |                ^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/api.py", line 115, in post
privategpt-private-gpt-1  |     return request("post", url, data=data, json=json, **kwargs)
privategpt-private-gpt-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/api.py", line 59, in request
privategpt-private-gpt-1  |     return session.request(method=method, url=url, **kwargs)
privategpt-private-gpt-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
privategpt-private-gpt-1  |     resp = self.send(prep, **send_kwargs)
privategpt-private-gpt-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
privategpt-private-gpt-1  |     r = adapter.send(request, **kwargs)
privategpt-private-gpt-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
privategpt-private-gpt-1  |     raise ConnectionError(e, request=request)
privategpt-private-gpt-1  | requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff609ddf10>: Failed to establish a new connection: [Errno 111] Connection refused'))
@drupol
Copy link

drupol commented Apr 21, 2024

I'm pretty sure this has nothing to do with private-gpt but with networking shenanigans related to containers.

@BenBatsir
Copy link
Author

I've pretty much tried to dockerize it according to the docs

@drupol
Copy link

drupol commented Apr 21, 2024

Is Ollama running from inside the container too? (localhost)

@link-kut
Copy link

link-kut commented Apr 21, 2024 via email

@BenBatsir
Copy link
Author

Is Ollama running from inside the container too? (localhost)

no

@drupol
Copy link

drupol commented Apr 21, 2024

That's the issue. From the container localhost IS the container, therefore, it will never reach the Ollama instance.

@BenBatsir
Copy link
Author

That's the issue. From the container localhost IS the container, therefore, it will never reach the Ollama instance.

it makes sense, but then again I'm surprised how come I need to change anything given I'm simply following the documentation.

Maybe I need to expose port 11434 to this container ?

@drupol
Copy link

drupol commented Apr 21, 2024 •
edited
Loading

Maybe I need to expose port 11434 to this container ?

No, that is pointless.

What you need to do instead is to replace occurrences of localhost:11434 with <host-ip>:11434. This way, the container will be able to connect to it.

@BenBatsir
Copy link
Author

Maybe I need to expose port 11434 to this container ?

No, that is pointless.

What you need to do instead is to replace occurrences of localhost:11434 with <host-ip>:11434. This way, the container will be able to connect to it.

Not sure I understand where exactly.
In the code itself?

@drupol
Copy link

drupol commented Apr 21, 2024 •
edited
Loading

private-gpt uses YAML files for its configuration.
The ollama address to connect to is hardcoded in these configuration files:

private-gpt/settings-ollama.yaml

Line 16 in c1802e7

api_base: http://localhost:11434

What you need to do is to create your custom YAML file and use it in Docker (via a directory binding?).

AFAIK and since there's no environment variable to change that Ollama URL, you don't have other choice.

@BenBatsir
Copy link
Author

private-gpt uses YAML files for its configuration. The ollama address to connect to is hardcoded in these configuration files:

private-gpt/settings-ollama.yaml

Line 16 in c1802e7

api_base: http://localhost:11434

What you need to do is to create your custom YAML file and use it in Docker (via a directory binding?).

AFAIK and since there's no environment variable to change that Ollama URL, you don't have other choice.

ok... so localhost should be change to what? 127.0.0.1 ? something else?

@drupol
Copy link

drupol commented Apr 21, 2024 •
edited
Loading

I guess your machine has a local IP address ? Then use it.

@BenBatsir
Copy link
Author

I guess your machine has a local IP address ? Then use it.

I ended up changing it to the container name:

ollama:
  llm_model: mistral
  embedding_model: nomic-embed-text
  api_base: http://privategpt-ollama-1:11434
  embedding_api_base: http://privategpt-ollama-1:11434

Now I'm having a different issue:

ValueError: Ollama call failed with status code 404. Details: model 'nomic-embed-text' not found, try pulling it first

@drupol
Copy link

drupol commented Apr 21, 2024

That's very good news. That means you succeeded to connect to Ollama.

Now you need to read and hopefully tackle the next issue.

To fix it, just do: ollama pull nomic-embed-text

@BenBatsir
Copy link
Author

That's very good news. That means you succeeded to connect to Ollama.

Now you need to read and hopefully tackle the next issue.

To fix it, just do: ollama pull nomic-embed-text

it worked!
so should I add it in Dockerfile.external as a step ?

@drupol
Copy link

drupol commented Apr 21, 2024

I have no knowledge in Dockerfile.external, but very curious to learn how you fixed the issue. It would be nice if you could document it here too for future reference.

@BenBatsir
Copy link
Author

I have no knowledge in Dockerfile.external, but very curious to learn how you fixed the issue. It would be nice if you could document it here too for future reference.

actually this docker file belongs to the private-gpt image, so I'll need to figure this out somehow, but I will document it once I'll find a suitable solution.

Thanks a lot for your help

@jaluma
Copy link
Collaborator

jaluma commented Jul 10, 2024

@BenBatsir You can't add this line to Dockerfile.external, as it is something you need to run on the ollama container.

In this situation, I have three ideas on how to fix it:

  1. Modify the command in docker-compose and replace it with something like:
    ollama pull nomic-embed-text && ollama pull mistral && ollama serve.
  2. Open a bash terminal on the container and pull both models.
  3. Create a custom version of Ollama with the downloaded model.

Anyway, we will talk internally about this problem and try to fix a solution to prevent further related problems.

@jaluma jaluma closed this as completed Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@drupol @jaluma @link-kut @BenBatsir