Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support JSTOR papers #15

Open
2 of 3 tasks
Nelson-Gon opened this issue Jul 6, 2021 · 4 comments
Open
2 of 3 tasks

Support JSTOR papers #15

Nelson-Gon opened this issue Jul 6, 2021 · 4 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed priority:high

Comments

@Nelson-Gon
Copy link
Owner

Nelson-Gon commented Jul 6, 2021 •
edited

Description

I would like to cite papers from JSTOR

Similar Features

Similar to NCBI and other sources.

Feature Details

Full Harvard citation as required.

Proposed Implementation

  • Access dynamically loaded elements e.g. journal volume, journal pages. Use selenium/webdriver
  • Create citation as required.
  • Fix 403s in jstor tests
@Nelson-Gon Nelson-Gon added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers priority:high labels Jul 6, 2021
Nelson-Gon added a commit that referenced this issue Jul 6, 2021
@Nelson-Gon
Extend jstor elements, todo: #15
890dce1
Nelson-Gon added a commit that referenced this issue Jul 8, 2021
@Nelson-Gon
Extract jstor vol, issue, pages #15
b56aa82
Nelson-Gon added a commit that referenced this issue Jul 8, 2021
@Nelson-Gon
Partially full jstor citation #15, todo: clean up authors, clean up v…
21332ca 
…ol-issues-pages
Nelson-Gon added a commit that referenced this issue Jul 12, 2021
@Nelson-Gon
full jstor citations #15
29633c3
Nelson-Gon added a commit that referenced this issue Jul 12, 2021
@Nelson-Gon
Update docs #15
8a5d9d1
Nelson-Gon added a commit that referenced this issue Jul 20, 2021
@Nelson-Gon
Fix single author truncation #15
d35162e
Nelson-Gon added a commit that referenced this issue Jul 21, 2021
@Nelson-Gon
#15 JSTOR 403s another take
252b825
@Nelson-Gon
Copy link
Owner Author

Nelson-Gon commented Jul 21, 2021 •
edited

This works with curl -I "https://www.jstor.org/stable/26469531" --user-agent "Mozilla/5.0" but not python requests. No idea why for now.
This works locally but not on remote (Github actions).

@Nelson-Gon Nelson-Gon reopened this Jul 21, 2021
@Nelson-Gon Nelson-Gon pinned this issue Jul 21, 2021
Nelson-Gon added a commit that referenced this issue Jul 22, 2021
@Nelson-Gon
Automate method selection, DRY DRY DRY. Todo: Fix remote script #15 t…
b6bec31 
…hrowing jstor 403s
Nelson-Gon added a commit that referenced this issue Jul 23, 2021
@Nelson-Gon
Rethink tests, skip jstor tests for now #15
5099153
Nelson-Gon added a commit that referenced this issue Jul 23, 2021
@Nelson-Gon
Separate jstor tests #15
58315d5
Nelson-Gon added a commit that referenced this issue Jul 23, 2021
@Nelson-Gon
Raise codecov threshold, skip jstor tests unless interactive #15
98766cc
@Nelson-Gon
Copy link
Owner Author

To test that jstor works (for now), remove the skipUnless condition and run tests locally. Tests fail on Github probably because Github scripts are blocked by JSTOR. See quanteda/quanteda#1593

Nelson-Gon added a commit that referenced this issue Jul 23, 2021
@Nelson-Gon
Only run jstor if not via github #15
aa507fa
@Nelson-Gon
Copy link
Owner Author

Nothing to fix, at least locally.

@Nelson-Gon
Copy link
Owner Author

As of 5th November 2021, JSTOR uses dynamic JS with div attributes in the form data v--- which require a bit more work to work with. Plans are underway to (request?) access (to) JSTOR's API instead.

@Nelson-Gon Nelson-Gon reopened this Nov 5, 2021
Nelson-Gon added a commit that referenced this issue Nov 5, 2021
@Nelson-Gon
Rethink tests, drop JSTOR support #15
1ec3c9d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed priority:high
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Nelson-Gon