Get Perseus definitions

[kylepjohnson]

Cross-referencing Issue cltk/cltk_frontend#60 by @lukehollis.

[manu-chroma]

@kylepjohnson I would like to implement the definition endpoint. How to get started on this one ?

[manu-chroma]

I'm assuming the json format to be {‘word’ : ‘definition’}
Proposed query URLhttps://api.cltk.org/<lang>/<define>?q=<word>
This is what I had in mind.

[lukehollis]

Great--looks good. Here is a proposed data format I have been working with for the definitions so far: https://github.com/cltk/cltk_frontend/blob/master/client/views/reading/DefinitionsPanel.jsx#L18

[manu-chroma]

Thanks for the link. I'll post a more detailed overview, considering your implementation on the front end.

[manu-chroma]

@lukehollis Can you please give me the link to definitions in the corpus ? Thanks!

[lukehollis]

We will need to look these up programmatically based on an input string. I'm indifferent on whether that input string should be a single word or a sentence/line of poetry.

[lukehollis]

Here's an example query for the first word of the Aeneid: http://www.perseus.tufts.edu/hopper/morph?l=Arma&la=la

[manu-chroma]

@lukehollis @kylepjohnson If I am not wrong, whenever a user queries for a definition, we want to extract data from the following source: http://www.perseus.tufts.edu/hopper/morph?l=Arma&la=la

1. I am not able to find a exposed API for search results on the given website, thus we might have to scrape which is not bad as it has bare minimum html code.
   There is a XML option which has incomplete info. http://www.perseus.tufts.edu/hopper/xmlmorph?lang=la&lookup=virum
2. The query URL for the site is very intuitive and can easily be worked out to provide different options in our API
3. Should this not be included in the core CLTK and then exposed using the API directly from the core ?

Give me a heads up and I'll implement this feature right on.
Thanks.

[lukehollis]

That's a good question, @manu-chroma--and thanks for tracking down the xmlmorph version. I can see both sides and don't have a very strong opinion one way or the other, but I think that since this is specifically an issue dealing with hitting an external web API, it should exist in the cltk_api instead of core. @kylepjohnson will have a better sense of the direction for the core package though.

[manu-chroma]

@lukehollis I get your point. By the way, the xmlmorph version looks incomplete to me. It doesn't contain the definition. I think scraping is the way to go. What do you think ?

[kylepjohnson]

Hey guys, An alternative to using the Perseus API would be to use the same lexicon they're using.

Somehow accidentally deleted the relevant files for Latin (needs to be re-added to cltk/latin_lexica_perseus), here's Greek:

• Repo: https://github.com/cltk/greek_lexica_perseus
• Map of headword-inflected word: https://raw.githubusercontent.com/cltk/greek_lexica_perseus/master/greek-lemmata.txt (large file)
• Greek-English dictionary: https://raw.githubusercontent.com/cltk/greek_lexica_perseus/master/greek-analyses_1.txt (large file)
• See also greek-analyses_2.txt. I had to break the original greek-analyses.txt into two because it was over GitHub's 100MB limit.

The Latin works the same. @lukehollis does the Perseus API offer more than what's in greek-lemmata.txt, other than parsing it into XML for us?

Note: I'm going to open a new ticket that the Latin lemmata files get re-added to latin_lexica_perseus.

[kylepjohnson]

Referencing Issue #30. @manu-chroma would you like to do this? It's related and I bet will answer your question about whether to use the API or parse the files ourselves.

[lukehollis]

Not that I know of, and if we can use our own servers it seems like it'd be nicer to their systems not to be hitting their API all the time (not sure of the usage restraints there). Working from the cltk lexica repos here seems like a better solution if possible. All for it. 👍

[lukehollis]

Hey @manu-chroma, any updates here? Would it be possible to push any progress that you've made?

[manu-chroma]

Hey @lukehollis, I've mailed you regarding the same. Please check.

[lukehollis]

Thanks so much, @manu-chroma--and good luck with the switch!