Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Successful Rate 0% when batch loading comments by ids #23

Closed
dukesun99 opened this issue Sep 4, 2021 · 1 comment
Closed

Successful Rate 0% when batch loading comments by ids #23

dukesun99 opened this issue Sep 4, 2021 · 1 comment

Comments

@dukesun99
Copy link

I found a strange bug, basically when calling search comments, large max_ids_per_request will result in 0% successful rate and forever retry. Maybe the result is too long?

To reproduce:

comments = api.search_submission_comment_ids(ids="haucpf")    
comment_list1 = [comment for comment in comments]    
comments = api.search_comments(ids=comment_list1, max_ids_per_request=1000)    
comment_list_full = [comment for comment in comments]    

The above code works if we change to max_ids_per_request=500.

@mattpodolak
Copy link
Owner

mattpodolak commented Sep 6, 2021

Hi @dukesun99 it looks like Pushshift has started returning 520s for requests with 1000 ids, I'll include an update in the upcoming v2 release to set the maximum value to 500, thanks for reporting this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants