issue: web search is broken

[EntropyYue]

Check Existing Issues

• I have searched the existing issues and discussions.
• I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.6.10

Ollama Version (if applicable)

No response

Operating System

Windows 10

Browser (if applicable)

No response

Confirmation

• I have read and followed all instructions in README.md.
• I am using the latest version of both Open WebUI and Ollama.
• I have included the browser console logs.
• I have included the Docker container logs.
• I have listed steps to reproduce the bug in detail.

Expected Behavior

The citation should be displayed normally and the model should correctly cite the context found in the search

Actual Behavior

The model cannot see the search results and the citations are not displayed

Steps to Reproduce

Conduct web search

Logs & Screenshots

Additional Information

The citations in the text upload works normally, and the model can also see the context

[Ithanil]

Works for me (tried only with full-context mode):

If you are also using full-context, are you really sure your model can handle the context size from 29 web pages?

[EntropyYue]

I did not use full context mode, and after testing, I found that my web search also does not work properly in full context mode

The citations are all empty, but they exist

are you really sure your model can handle the context size from 29 web pages?

Additionally, my model could handle 29 web pages under the rag in version 0.6.9

[liuyiwei3366]

I encountered the same problem as you. This happened in version 0.6.10

[Ithanil]

Check if you find any error messages in the logs regarding content extraction of the sites.

[tjbck]

Unable to reproduce from our end, what web search engine are you using?

[EntropyYue]

I use searxng

[EntropyYue]

This seems to happen in the default loader, firecrawl can work normally

[ExDragine]

Same problem with searxng search engine. Searxny is latest version and no error logs. All deploy in Docker with same network.
OpenWebUI logs have a few clues about the problem

WARNING | langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching ...

[robertcontois]

I'm getting rate limit errors with all search engines as well (brave + duckduckgo). This occurred on 0.6.9 and 0.6.10

[Benthejunebug]

same here rate limit errors. I use searxng

[tth37]

Rate limit is an known issue for brave and DuckDuckGo search engines. However in your case "langchain_community.document_loaders.web_base:_fetch_with_rate_limit:271 - Error fetching ..." does not indicate an rate limit issue, simply it's your default web loader not working (urls were blocked, internet connection error, etc.). You can try different web loaders

[EntropyYue]

Additional information: The browser and backend logs are all normal

[tth37]

#14073, try setting Web Search - Loader - Verify SSL Certificate to False for now!

[ExDragine]

When I change ENABLE_WEB_LOADER_SSL_VERIFICATION to True, the problem seems to be solved!

[tth37]

Great, we might have found the root cause 🎉