New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wikidata spice #1991

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
6 participants
@Benestar

Benestar commented Jul 3, 2015

This is a first version of a spice to automatically show results based on Wikidata. More information below or in the introduction.

This proposal currently uses an api wrapper of the Wikidata api to avoid having several requests. To have a stable api, we can provide this wrapper directly on Wikidata. I will create a patch for such an api module if this is going to be merged.

What is Wikidata?

Wikidata is a free linked database that can be read and edited by both humans and machines.

Wikidata acts as central storage for the structured data of its Wikimedia sister projects including Wikipedia, Wikivoyage, Wikisource, and others.

Wikidata also provides support to many other sites and services beyond just Wikimedia projects! The content of Wikidata is available under a free license, exported using standard formats, and can be interlinked to other open data sets on the linked data web.

From the main page (CC-BY-SA 3.0)

Wikidata logo

Pinging @moollaza as he was interested in this wikidata integration.

Pull request template

What does your Instant Answer do?

Parse simple questions/phrases in the form "foo of bar" and search for values on Wikidata.

What problem does your Instant Answer solve (Why is it better than organic links)?

Instead of having to read eg. the Wikipedia article you can directly see the result in a nice format.

What is the data source for your Instant Answer? (Provide a link if possible)

Wikidata

Why did you choose this data source?

It's the new great project for open data, it's free and it's created and maintained by the Wikipedia community

Are there any other alternative (better) data sources?

There may be better sources for some specific data, but no sources have such a broad coverage as Wikidata

What are some example queries that trigger this Instant Answer?

"president of the United States", "population of Berlin", "what are the children of Barack Obama"

Which communities will this Instant Answer be especially useful for? (gamers, book lovers, etc)

very broad topics covered (like Wikipedia and even more)

Is this Instant Answer connected to a DuckDuckHack Instant Answer idea?

something like this was suggested in https://duck.co/ideas/idea/343/improve-0-click-info-with-wikidata-entries

Which existing Instant Answers will this one supersede/overlap with?

it may overlap with some IAs about population data

Are you having any problems? Do you need our help with anything?

yes, merge this :D

What are the terms of use for the API? Will DuckDuckGo need specific authorization (e.g. an API key)? Are there any costs associated with API usage?

Wikidata is licenced under CC0 and no special authorization should be required

What does the Instant Answer look like? (Provide a screenshot for new or updated Instant Answers)

hgzsp

Checklist

Please place an 'X' where appropriate.

[x] Added metadata and attribution information
[x] Wrote test file and added to t/ directory
[x] Verified that instant answer adheres to design guidelines (https://duck.co/duckduckhack/design_styleguide)
[x] Verified that instant answer adheres to code styleguide (https://duck.co/duckduckhack/code_styleguide)
[] Tested cross-browser compatibility

    Please let us know which browsers/devices you've tested on:
    - Windows 8
        [x] Google Chrome
        [x] Firefox
        [] Opera
        [] IE 10

    - Windows 7
        [] Google Chrome
        [] Firefox
        [] Opera
        [] IE 8
        [] IE 9
        [] IE 10

    - Windows XP
        [] IE 8

    - Mac OSX
        [] Google Chrome
        [] Firefox
        [] Opera
        [] Safari

    - iOS (iPhone)
        [] Safari Mobile
        [] Google Chrome
        [] Opera

    - iOS (iPad)
        [] Safari Mobile
        [] Google Chrome
        [] Opera

    - Android
        [] Firefox
        [] Native Browser
        [] Google Chrome
        [] Opera

IA Page: https://duck.co/ia/view/wikidata

@moollaza

This comment has been minimized.

Show comment
Hide comment
@moollaza

moollaza Jul 3, 2015

Member

@Benestar thanks for your contribution!

Can you please fill out the pull request template and edit your initial comment. Once completed, we'll be sure to review your PR and give you some feedback.

Thanks!

Member

moollaza commented Jul 3, 2015

@Benestar thanks for your contribution!

Can you please fill out the pull request template and edit your initial comment. Once completed, we'll be sure to review your PR and give you some feedback.

Thanks!

@Benestar

This comment has been minimized.

Show comment
Hide comment
@Benestar

Benestar Jul 3, 2015

@moollaza thanks for the hint, I've added the template

Benestar commented Jul 3, 2015

@moollaza thanks for the hint, I've added the template

@airon90

This comment has been minimized.

Show comment
Hide comment
@airon90

airon90 Aug 8, 2015

Three questions:

  • is this 0-click answer multilingual (e.g.: if I write "presidente degli Stati Uniti d'America" (Italian language for "president of the United States") it returns "Barack Obama")? If not, how can I contribute without coding (I'm not a coder)?
  • is the answer returned with a link (e.g.: "Barack Obama" is not just text but also a link)? If so, do you use DuckDuckGo, Wikipedia or Wikidata?
  • What are you waiting for? Why don't you merge the code? :)

airon90 commented Aug 8, 2015

Three questions:

  • is this 0-click answer multilingual (e.g.: if I write "presidente degli Stati Uniti d'America" (Italian language for "president of the United States") it returns "Barack Obama")? If not, how can I contribute without coding (I'm not a coder)?
  • is the answer returned with a link (e.g.: "Barack Obama" is not just text but also a link)? If so, do you use DuckDuckGo, Wikipedia or Wikidata?
  • What are you waiting for? Why don't you merge the code? :)
@Benestar

This comment has been minimized.

Show comment
Hide comment
@Benestar

Benestar Aug 9, 2015

Hi @airon90

is this 0-click answer multilingual (e.g.: if I write "presidente degli Stati Uniti d'America" (Italian language for "president of the United States") it returns "Barack Obama")? If not, how can I contribute without coding (I'm not a coder)?

Wikidata is multilingual "per definition" and you can contribute to it by translating the labels of items there (see https://www.wikidata.org/wiki/Wikidata:Contribute). However, I didn't find a way yet to get the language of the user. If that is possible somehow, it should be very easy to use other languages as well.

is the answer returned with a link (e.g.: "Barack Obama" is not just text but also a link)? If so, do you use DuckDuckGo, Wikipedia or Wikidata?

The answers are currently linked to Wikidata though we can change this and link either do DDG or to the connected Wikipedia article.

What are you waiting for? Why don't you merge the code? :)

I didn't have enough time the last weeks to fix the comments by @moollaza but will hopefully address them this week.

Benestar commented Aug 9, 2015

Hi @airon90

is this 0-click answer multilingual (e.g.: if I write "presidente degli Stati Uniti d'America" (Italian language for "president of the United States") it returns "Barack Obama")? If not, how can I contribute without coding (I'm not a coder)?

Wikidata is multilingual "per definition" and you can contribute to it by translating the labels of items there (see https://www.wikidata.org/wiki/Wikidata:Contribute). However, I didn't find a way yet to get the language of the user. If that is possible somehow, it should be very easy to use other languages as well.

is the answer returned with a link (e.g.: "Barack Obama" is not just text but also a link)? If so, do you use DuckDuckGo, Wikipedia or Wikidata?

The answers are currently linked to Wikidata though we can change this and link either do DDG or to the connected Wikipedia article.

What are you waiting for? Why don't you merge the code? :)

I didn't have enough time the last weeks to fix the comments by @moollaza but will hopefully address them this week.

@moollaza moollaza self-assigned this Aug 11, 2015

@moollaza

This comment has been minimized.

Show comment
Hide comment
@moollaza

moollaza Aug 31, 2015

Member

@abeyang this one is ready for an initial design review. It's using our Info, and Media templates. It should look very similar to our Wikipedia answers. It's using Wikidata, but has more flexible queries.

Currently live on: https://beta.duckduckgo.com/?q=president+of+the+United+States&ia=answer

@Benestar I see a couple bugs:

Member

moollaza commented Aug 31, 2015

@abeyang this one is ready for an initial design review. It's using our Info, and Media templates. It should look very similar to our Wikipedia answers. It's using Wikidata, but has more flexible queries.

Currently live on: https://beta.duckduckgo.com/?q=president+of+the+United+States&ia=answer

@Benestar I see a couple bugs:

@Benestar

This comment has been minimized.

Show comment
Hide comment
@Benestar

Benestar Aug 31, 2015

"population of Berlin" isn't currently working because I only implemented the answers for the item datatype (ie. values which contain other items). I will however also implement views for quantities like the population, times like date of creation or birth date and locations. You can see a list of datatypes here.

The images are actually tall enough but they get displayed in a not so good way. You can see the full image when clicking on the item. However, we will always encounter this problem of cropped images as some images may be horizontal and some vertical.

Benestar commented Aug 31, 2015

"population of Berlin" isn't currently working because I only implemented the answers for the item datatype (ie. values which contain other items). I will however also implement views for quantities like the population, times like date of creation or birth date and locations. You can see a list of datatypes here.

The images are actually tall enough but they get displayed in a not so good way. You can see the full image when clicking on the item. However, we will always encounter this problem of cropped images as some images may be horizontal and some vertical.

@abeyang

This comment has been minimized.

Show comment
Hide comment
@abeyang

abeyang Sep 1, 2015

Contributor

@Benestar , I'm not familiar with wikidata, but is their data complete? I just did a few quick sample queries, and a search for Steve Job's children produces 1 result with wikidata, but 4 results with That Other Search Engine...

Contributor

abeyang commented Sep 1, 2015

@Benestar , I'm not familiar with wikidata, but is their data complete? I just did a few quick sample queries, and a search for Steve Job's children produces 1 result with wikidata, but 4 results with That Other Search Engine...

@lucaswerkmeister

This comment has been minimized.

Show comment
Hide comment
@lucaswerkmeister

lucaswerkmeister Sep 1, 2015

@abeyang most of the currently existing items were initially created by bots from existing Wikipedia articles, or a few other datasets (administrative districts). In this case, it seems the three other children of Steve Jobs weren’t deemed significant for inclusion in any Wikipedia language edition. I’ve added them now, since AFAIK Wikidata’s threshold of significance is lower than that of Wikipedia.

lucaswerkmeister commented Sep 1, 2015

@abeyang most of the currently existing items were initially created by bots from existing Wikipedia articles, or a few other datasets (administrative districts). In this case, it seems the three other children of Steve Jobs weren’t deemed significant for inclusion in any Wikipedia language edition. I’ve added them now, since AFAIK Wikidata’s threshold of significance is lower than that of Wikipedia.

@airon90

This comment has been minimized.

Show comment
Hide comment
@airon90

airon90 Sep 2, 2015

https://beta.duckduckgo.com/?q=kuussa+tuulee&ia=iatesting
https://beta.duckduckgo.com/?q=data+about+kuussa+tuulee&ia=iatesting

I'm expecting a table like on Wikipedia 0-click info with all data about Q20872523 because sometimes I need a overview of a topic.
Do I need to create another bug report to implement this? :)
I create a BR #2148 to adapt Wikipedias 0-click info to add info from Wikidata
Thank you very much!

airon90 commented Sep 2, 2015

https://beta.duckduckgo.com/?q=kuussa+tuulee&ia=iatesting
https://beta.duckduckgo.com/?q=data+about+kuussa+tuulee&ia=iatesting

I'm expecting a table like on Wikipedia 0-click info with all data about Q20872523 because sometimes I need a overview of a topic.
Do I need to create another bug report to implement this? :)
I create a BR #2148 to adapt Wikipedias 0-click info to add info from Wikidata
Thank you very much!

@Benestar

This comment has been minimized.

Show comment
Hide comment
@Benestar

Benestar Sep 2, 2015

Thanks @lucaswerkmeister for adding that data. This is exactly how Wikidata works @abeyang. Driven by a community the data gets better when it is used in more places. If more users figure out they can actually edit and fix the data the coverage and quality will increase as well.

@airon90 yes that is a different task but should also definitely be done. Wikidata is per definition machine readable and much better in displaying such data tables than Wikipedia infoboxes.

Benestar commented Sep 2, 2015

Thanks @lucaswerkmeister for adding that data. This is exactly how Wikidata works @abeyang. Driven by a community the data gets better when it is used in more places. If more users figure out they can actually edit and fix the data the coverage and quality will increase as well.

@airon90 yes that is a different task but should also definitely be done. Wikidata is per definition machine readable and much better in displaying such data tables than Wikipedia infoboxes.

@airon90

This comment has been minimized.

Show comment
Hide comment
@Benestar

This comment has been minimized.

Show comment
Hide comment
@Benestar

Benestar Sep 2, 2015

@airon90 for it shows an answer. Note you have to select the "Answer" tab under the search box. https://beta.duckduckgo.com/?q=product+certification+of+Roma-Bangkok&ia=answer

Benestar commented Sep 2, 2015

@airon90 for it shows an answer. Note you have to select the "Answer" tab under the search box. https://beta.duckduckgo.com/?q=product+certification+of+Roma-Bangkok&ia=answer

@airon90

This comment has been minimized.

Show comment
Hide comment
@airon90

airon90 Sep 2, 2015

https://beta.duckduckgo.com/?q=product+certification+of+Roma-Bangkok&ia=answer

schermata 2015-09-02 alle 14 55 15

I know that I have to select "Answer" but no answer tab is shown

airon90 commented Sep 2, 2015

https://beta.duckduckgo.com/?q=product+certification+of+Roma-Bangkok&ia=answer

schermata 2015-09-02 alle 14 55 15

I know that I have to select "Answer" but no answer tab is shown

@Benestar

This comment has been minimized.

Show comment
Hide comment
@Benestar

Benestar Sep 2, 2015

Uhm now that is really strange. For me, an answer section appears with the relevant information.

bildschirmfoto am 2015-09-02 um 15 34 22

Benestar commented Sep 2, 2015

Uhm now that is really strange. For me, an answer section appears with the relevant information.

bildschirmfoto am 2015-09-02 um 15 34 22

@airon90

This comment has been minimized.

Show comment
Hide comment
@airon90

airon90 Sep 3, 2015

Now it works for me too. Strange!

airon90 commented Sep 3, 2015

Now it works for me too. Strange!

@moollaza

This comment has been minimized.

Show comment
Hide comment
@moollaza

moollaza Nov 12, 2015

Member

@Benestar sorry for the delay -- I forget where we left off on this. What's the status?

Member

moollaza commented Nov 12, 2015

@Benestar sorry for the delay -- I forget where we left off on this. What's the status?

@Benestar

This comment has been minimized.

Show comment
Hide comment
@Benestar

Benestar Dec 2, 2015

Hey @moollaza, the status is that this needs more feedback :-) Perhaps this needs a rebase and some updates for the Wikibase api. We now have quantities with units and a lot more useful data so we need a nice way to display this inside of this spice. However, I'd like the simple version to get ready first.

Benestar commented Dec 2, 2015

Hey @moollaza, the status is that this needs more feedback :-) Perhaps this needs a rebase and some updates for the Wikibase api. We now have quantities with units and a lot more useful data so we need a nice way to display this inside of this spice. However, I'd like the simple version to get ready first.

@daxtheduck daxtheduck deployed to beta.duckduckgo.com Dec 7, 2015 Active

@moollaza

This comment has been minimized.

Show comment
Hide comment
@moollaza

moollaza Dec 7, 2015

Member

@Benestar I've re-deployed this to beta. I'm have a hard time getting it to produce any results though.
Most of the time our Wiki Fathead has the correct answer.

Is there a particularly property/attribute that you know is well defined across the data? If so, I'd like to use a list of the top...20? 50? as triggers to really reign in the triggering of this IA because right now it's going to trigger too much.

For "daughters of barack obama" we need to use tile variants and/or larger images to make things look better.

  1. Do you have wider images available? Ideally they should fill the width of the tile so there's no white border
  2. If not, use tile: 'narrow' to force a narrow tile width
  3. Are most titles 1 line? 2? 3? Let's use a tileTitle: '2line' or similar variant to allow for the titles to display clearly
  4. "daughter of President Obama" should be capitalized. A leading lower-case letter looks odd.

I'm still not sure what other kind of info this IA is capable of producing. Can you give me a list of queries that produce different results so I can understand how the IA looks for different results?

Tile Variants Info: https://talsraviv.gitbooks.io/duckduckhackdocs/content/duckduckhack/frontend-reference/variants-reference.html

Member

moollaza commented Dec 7, 2015

@Benestar I've re-deployed this to beta. I'm have a hard time getting it to produce any results though.
Most of the time our Wiki Fathead has the correct answer.

Is there a particularly property/attribute that you know is well defined across the data? If so, I'd like to use a list of the top...20? 50? as triggers to really reign in the triggering of this IA because right now it's going to trigger too much.

For "daughters of barack obama" we need to use tile variants and/or larger images to make things look better.

  1. Do you have wider images available? Ideally they should fill the width of the tile so there's no white border
  2. If not, use tile: 'narrow' to force a narrow tile width
  3. Are most titles 1 line? 2? 3? Let's use a tileTitle: '2line' or similar variant to allow for the titles to display clearly
  4. "daughter of President Obama" should be capitalized. A leading lower-case letter looks odd.

I'm still not sure what other kind of info this IA is capable of producing. Can you give me a list of queries that produce different results so I can understand how the IA looks for different results?

Tile Variants Info: https://talsraviv.gitbooks.io/duckduckhackdocs/content/duckduckhack/frontend-reference/variants-reference.html

@moollaza

This comment has been minimized.

Show comment
Hide comment
@moollaza

moollaza Apr 12, 2016

Member

@Benestar I'm closing this PR for now. If you're still interested in finishing, please let me know and I'll gladly re-open it for you!

Member

moollaza commented Apr 12, 2016

@Benestar I'm closing this PR for now. If you're still interested in finishing, please let me know and I'll gladly re-open it for you!

@moollaza moollaza closed this Apr 12, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment