Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Pushshift Data API is only accessible to moderators #96

Open
DenverCoder1 opened this issue May 29, 2023 · 3 comments
Open

The Pushshift Data API is only accessible to moderators #96

DenverCoder1 opened this issue May 29, 2023 · 3 comments
Labels
announcement Announcement of changes or outages

Comments

@DenverCoder1
Copy link
Owner

DenverCoder1 commented May 29, 2023

The Pushshift Data API is only accessible to approved moderators as announced here.

Since Unedit and Undelete for Reddit relies on Pushshift to work, non-moderators can not use this extension unless an alternative data provider is found.

@DenverCoder1 DenverCoder1 added the announcement Announcement of changes or outages label May 29, 2023
@DenverCoder1 DenverCoder1 pinned this issue May 29, 2023
@RebootedDuck
Copy link
Contributor

I know you've probably already seen this but RedArc was recently created as an API interface to access historical Pushshift dumps, I could look into how difficult it would be to change the API requests in the extension if you'd like?

@DenverCoder1
Copy link
Owner Author

DenverCoder1 commented May 31, 2023

I know you've probably already seen this but RedArc was recently created as an API interface to access historical Pushshift dumps, I could look into how difficult it would be to change the API requests in the extension if you'd like?

Thanks, I heard about that, but as far as I know they aren't hosting an indexed API of all historical data.

All of the data is about 30 terabytes uncompressed which is not so simple to host or make accessible. I'd be happy to use an alternate API if someone has created one. I was able to find http://redarc.basedbin.org/ which has just an archive of just r/DataHoarder accessible via web requests, but not all subreddits.

@DenverCoder1 DenverCoder1 changed the title ⚠️ The Pushshift Data API is no longer able to ingest new data ⚠️ The Pushshift Data API is no longer available for use May 31, 2023
@RebootedDuck
Copy link
Contributor

RebootedDuck commented Jun 4, 2023

Oh ok sorry I misunderstood then, judging from the recent announcement on r/pushshift and the MoU that they won't let us see it doesn't look good for pushshift ever coming back, also the site imploding on itself because of 3rd party apps

@DenverCoder1 DenverCoder1 changed the title ⚠️ The Pushshift Data API is no longer available for use The Pushshift Data API is only accessible to moderators Jun 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
announcement Announcement of changes or outages
Projects
None yet
Development

No branches or pull requests

2 participants