Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: dogpile engine #202

Closed
dalf opened this issue Jul 13, 2021 · 1 comment · Fixed by #203
Closed

Bug: dogpile engine #202

dalf opened this issue Jul 13, 2021 · 1 comment · Fixed by #203
Labels
bug Something isn't working

Comments

@dalf
Copy link
Member

dalf commented Jul 13, 2021

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

How did you install SearXNG?
N/A

What happened?
image

How To Reproduce
See above.

Expected behavior
To get the result or disable the engine.

Additional context

According to the logs it seems the engine redirects to http://omblockedips.com (not https). It shows the Recaptcha challenge.
So I suggest to add omblockedips.com to

searxng/searx/network/raise_for_httperror.py

Line 54 in 4f09c7b

def raise_for_captcha(resp):

Technical report

Error

  • Error: httpx.UnsupportedProtocol
  • Percentage: 100
  • Parameters: (None, None, 'omblockedips.com')
  • File name: searx/search/processors/online.py:99
  • Function: _send_http_request
  • Code: response = req(params['url'], **request_args)

Checker

  • simple: httpx.UnsupportedProtocol ,
  • paging: httpx.UnsupportedProtocol ,
@dalf dalf added the bug Something isn't working label Jul 13, 2021
return42 added a commit to return42/searxng that referenced this issue Jul 13, 2021
@return42
[fix] drop engine dogpile
2767046 
The engine was added in commit a4b0746 but now it shows new issues [1].

In the 90'th of the last century, dogpile had its own WEB index, but nowadays it
is a meta-search engine [2]

  Powered by technology, Dogpile returns all the best results from leading
  search engines including Google and Yahoo!

Using dogpile as an engine in SearXNG needs more investigation, a XPath solution
like we have is not enough.   It is questionable whether it still makes sense to
investigate more into a meta-search engine with a ReCAPTCHA in front.

With this patch the dogpile engine is removed

[1] searxng#202
[2] https://www.dogpile.com/support/aboutus

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
@return42 return42 mentioned this issue Jul 13, 2021
Merged
@return42
Copy link
Member

To get the result or disable the engine.

@dalf I decided to remove the dogpile engine / about details see commit message in #203

If you think it is OK to remove, you can merge #203.

@dalf dalf closed this as completed in #203 Jul 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[fix] drop engine dogpile return42/searxng
2 participants
@return42 @dalf