Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] new engine: wikispecies #2589

Merged
merged 3 commits into from
Aug 4, 2023
Merged

[feat] new engine: wikispecies #2589

merged 3 commits into from
Aug 4, 2023

Conversation

Bnyro
Copy link
Member

@Bnyro Bnyro commented Aug 3, 2023

What does this PR do?

  • adds wikispecies as an engine
  • wikispecies is just a standard wikimedia api

Why is this change important?

  • it contains articles about different animals/species and other forms of life.

How to test this PR locally?

  • disable all engines except and enable wikispecies

Related issues

closes #1222

@@ -86,7 +86,7 @@ def response(resp):
)

# append result
results.append({'url': url, 'title': result['title'], 'content': ''})
results.append({'url': url, 'title': result['title'], 'content': result.get('snippet', '')})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow this is rocket cool, it (BTW) fixes the content field of wikibooks, wikinews, wikiquote .. and all the other mediawiki engines 👍

In my review I will strip the HTML tags from the snippet .. coming soon.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The Wikimedia wikis [1] engines provide good answers and have short response
times --> no reason to disable these enhgines by default.  BTW: this patch adds
a (sub-) category ``wikimedia`` for the engines [1].

[1] https://meta.wikimedia.org/wiki/Wikimedia_wikis

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
@return42
Copy link
Member

return42 commented Aug 4, 2023

@Bnyro I added two commits on top of your branch .. can you please have a look / thanks 👍

  1. in 5e58ccf I added some docs and some additional config options for generic wikimedia engines
  2. in a96b6f3 I enabled the engine from the wikimedia foundation

@Bnyro, @unixfox what do you think about the 2nd patch ... is it OK or does it pollute the result list a bit too much?

@Bnyro
Copy link
Member Author

Bnyro commented Aug 4, 2023

That looks awesome in my opinion.
The wikimedia engines provide great results and therefore having them enabled by default is a good decision in my opinion.

@return42 return42 merged commit dad16c0 into searxng:master Aug 4, 2023
8 checks passed
@Bnyro Bnyro deleted the wikispecies branch August 4, 2023 17:07
@return42
Copy link
Member

return42 commented Aug 9, 2023

@Bnyro, @unixfox what do you think about the 2nd patch ... is it OK or does it pollute the result list a bit too much?

After using this setting for a while I'm no longer sure if it was a good decision to activate theses engines by default (a96b6f3) / in the general category I often get to many results with a ranking on top of the result list.

By example: when I query for markdown I got to many results I didn't was looking for.

Should we better deactivate these engines by default?


Or should we rank (weight) them lower?

@Bnyro
Copy link
Member Author

Bnyro commented Aug 9, 2023

I think it's better to weight them lower, since they do in fact provide good results.

return42 added a commit to return42/searxng that referenced this pull request Aug 9, 2023
The search engines deliver hits for many search terms [1], but these are usually
not the focus of the user. In order to arrange these hits further down in the
list, their weighting is reduced.

[1] searxng#2589 (comment)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
return42 added a commit to return42/searxng that referenced this pull request Aug 18, 2023
… wikiversity

The search engines deliver hits for many search terms [1], but these are usually
not the focus of the user. In order to arrange these hits further down in the
list, their weighting is reduced.

[1] searxng#2589 (comment)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
@unixfox
Copy link
Member

unixfox commented Aug 18, 2023

@Bnyro, @unixfox what do you think about the 2nd patch ... is it OK or does it pollute the result list a bit too much?

After using this setting for a while I'm no longer sure if it was a good decision to activate theses engines by default (a96b6f3) / in the general category I often get to many results with a ranking on top of the result list.

By example: when I query for markdown I got to many results I didn't was looking for.

Should we better deactivate these engines by default?

I would prefer to deactivate them, I never really got any meaningful results from them. In fact, I have disabled them on searx.be

I think we should be very careful with what engines we add as enabled by default for the General category, some people will not like if we add too many new kind of engines (4 new engines enabled by default in a row is a lot).

return42 added a commit to return42/searxng that referenced this pull request Aug 18, 2023
… wikiversity

The search engines deliver hits for many search terms [1], but these are usually
not the focus of the user. In order to arrange these hits further down in the
list, their weighting is reduced.

[1] searxng#2589 (comment)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
return42 added a commit to return42/searxng that referenced this pull request Aug 18, 2023
… wikiversity

The search engines deliver hits for many search terms [1], but these are usually
not the focus of the user. In order to arrange these hits further down in the
list, their weighting is reduced.

[1] searxng#2589 (comment)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
return42 added a commit that referenced this pull request Aug 21, 2023
… wikiversity

The search engines deliver hits for many search terms [1], but these are usually
not the focus of the user. In order to arrange these hits further down in the
list, their weighting is reduced.

[1] #2589 (comment)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Wikispecies
3 participants