Text Snowball

Text snowball

Goal. To create a list of terms (words and phrases) which are effective at searching the literature for a desired topic.

Why not arxiv?

Method

(We

• set relevancy metric (e.g. Caption contains "cyclic voltammogram" or other precise terms) image pixel characteristics.
• small precise rapid
• query pygetpapers with a relevant query (AND OR NOT) (maybe fairly general) and TERMS (OR'ed cyclic voltammetry) NOT generator
• download small corpus (<= 100 papers)
• rapidly inspect these for frequent relevant terms

• RAKE / YAKE (adjust phrase length)
• human eyeballs
• SPacy

• triage the list
• create/append to list of terms
• repeat until we get enough, or give up

now we have a list of terms

• create dictionary from terms

• section the paper (e.g. methods and figures)

• retrieve images

• GUI / browser display

• resurrect old examples of extracting curves from diagrams

(emphasize generality - plants + terpenes + genes - everything is reusable) - climate ,

demo

Set up on Google Collab

Choose voltammetry

Use EPMC

pygetpapers -q "cyclic voltammetry" -n
INFO: Final query is cyclic voltammetry
INFO: Total number of hits for the query are 30078

refine

pygetpapers -q "(cyclic voltammetry) AND (lithium ion)" -n
INFO: Final query is (cyclic voltammetry) AND (lithium ion)
INFO: Total number of hits for the query are 2675

Now snowball

• download 100 papers and examine for additional precision terms (OR)