[Bug][github_graphql] 502 when collecting pull requests

[likyh]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

sometimes PrCollector in graphql will raise 502.

What do you expect to happen

collect success.

How to reproduce

1. use graphql_github to collect https://github.com/apache/jackrabbit-oak
2. it will raise 502.

Anything else

The reason is too much data on each page. Now 30 pr, 30 * most 100 commits will bind to one graphql query. But I think it will be too slow if set to less than 30 pr per page. In past, the number is 100 pr per page.

Maybe we need a dynamic page size. It could request 100 per page and reduce less if 502 got.

Version

main

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[yumengwang03]

@likyh Do we want to put this bug in v0.15? cc @klesh

[likyh]

@likyh Do we want to put this bug in v0.15? cc @klesh

No, I don't think so.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.