max-items is just used for issues, but not for others

[xiao623]

grimoirelab-perceval/perceval/backends/core/github.py

Line 731 in 4a65ece

self.PPER_PAGE: self.max_items,

The value of max-items is only used for issues method, so what is the point of setting this value?

For example:

grimoirelab-perceval/perceval/backends/core/github.py

Line 711 in 4a65ece

self.PPER_PAGE: self.VPER_PAGE,

when we called https://api.github.com/repos/apache/spark/pulls/27058/comments?per_page=100&direction=asc&sort=updated, we got Server Error.
But if per_page is 50, the response is 200.

It should also be used in others methods so that we can set per_page what we want.

[vchrombie]

Thanks for opening the issue @xiao623.

From what I understand is, GitHub treats pull_requests as issues too (but not vice versa). So, I guess the backend implements the logic accordingly. It fetches the issues and then filters the pull requests later.

grimoirelab-perceval/perceval/backends/core/github.py

Line 752 in 4a65ece

issues_groups = self.issues(from_date=from_date)

when we called https://api.github.com/repos/apache/spark/pulls/27058/comments?per_page=100&direction=asc&sort=updated, we got Server Error.
But if per_page is 50, the response is 200.

I think this is unrelated to perceval and I'm afraid if the problem is from the github side.

apache/spark#27058 has 197 comments

$ curl -sSD/dev/stderr https://api.github.com/repos/apache/spark/pulls/27058/comments\?per_page=100 > /dev/null
HTTP/2 502 
server: GitHub.com
...

apache/spark#5694 has 406 comments

$ curl -sSD/dev/stderr https://api.github.com/repos/apache/spark/pulls/5694/comments\?per_page=100 > /dev/null
HTTP/2 200 
server: GitHub.com
...

I would like to hear more comments from the other community members as well.
Thanks.

[zhquan]

@xiao623 - max-items is used when the category is issue or pull_request but only in the first level and not in its sublevels as comments, reactions, etc. That means it will set the per_page to max_items only to fetch issues or pull requests but for each issue/pull to fetch comments the per_page is always the same VPER_PAGE = 100.

Maybe the value max-items should extend to all requests and not only for the first level.

when we called https://api.github.com/repos/apache/spark/pulls/27058/comments?per_page=100&direction=asc&sort=updated, we got Server Error.
But if per_page is 50, the response is 200.

I agree with @vchrombie that the problem is from the GitHub side

[xiao623]

Yes, I agree that the problem is from the Github side, and I ask for help in GitHub Support Community : got-server-error-when-called-api

Before they fix the problem, we need to modify per_page in issue_comments method so that the program can continue to run.