Skip to content

hfthair/emerald_crawler

main
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Download public paper from emerald

Install requirements

pip install -r requirements.txt

Get cookies

  1. Login account and visit a emerald paper link, make sure you have access to the full paper.
  2. Open developer tool of the browser: Application -> Cookies
  3. Copy all Key:Value pairs to cookies.py

Download papers with cookie

python download.py --save_dir . --auth_by_cookie True

Download open access papers (cookie not required)

python download.py --save_dir .

convert to jsonl

python csv2jsonl.py --csv_dir . --jsonl_filename emerald.jsonl

For BARTFacet Finetuning

For fine tune code and model output, please visit this repository Finetuning_BART_for_FACET_Summarization

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published