New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NYTimes data? #3

Closed

ghdi6758 opened this issue Oct 25, 2018 · 1 comment

ghdi6758 commented Oct 25, 2018

Dear Maciej,

Thanks a lot for making this amazing dataset available! :D I have one quick question and a comment.

I found this dataset includes 1.5M NYTimes articles, can you elaborate little more how you collect them?
I'd love to use this dataset for research. But the lack of details on data collection procedure (e.g., when the collection started and ended, what is the time range of collected news articles) makes it really hard to use this data for academic purposes. If you can describe how you collected this data, it would be gratefully helpful!

Thanks,
Jisun

Owner

several27 commented Nov 21, 2018

NYTimes articles were scraped directly from their website. The URLs that were scrapped were found through their developer API: https://developer.nytimes.com/
This project was a part of my dissertation, this report contains most of the information, but if it's missing any details feel free to ask: https://drive.google.com/file/d/1SPeBXjvbes9EmItDjR5PyRPkV54QsvPa/view?usp=sharing

several27 closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment