You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The api client does not request more than the first page for some resources from projects hosted at GitLab.com.
The pagination approach relies on the key x-total or x-total-pages being present in the request response headers.
The GitLab API doc's state the following concerning x-total and x-total-pages:
For performance reasons, if a query returns more than 10,000 records, GitLab doesn’t return the following headers:
x-total.
x-total-pages.
rel="last" link.
If both keys are missing from response headers, gitlab2prov naivly assumes that there is only one page of the requested resource.
Not all GitLab.com projects are affected. Updates will follow.
See also this section in the official GitLab documentation.
The text was updated successfully, but these errors were encountered:
This issue requires a rework of the gitlab2prov.api.request_handler class. We can no longer rely on the x-total pages key and can therefore not craft all request coroutines 'a priori'. A request coroutine could create new ones if needed and add them to a centralized queue if there are more pages left.
In the future we should move to KeySet-based pagination, as GitLab is slowly converting their API endpoints to support it. This should speed up response times for requests for bigger projects.
Rewrite of RequestHandler:
Requests are now queued on demand.
Added pagination fallback to the key 'x-next-page' to not solely rely on 'x-total-pages'.
Add initial exception system, see TODO notes. Exceptions are supposed to be handled in the GitlabClient methods.
The api client does not request more than the first page for some resources from projects hosted at GitLab.com.
The pagination approach relies on the key
x-total
orx-total-pages
being present in the request response headers.The GitLab API doc's state the following concerning
x-total
andx-total-pages
:If both keys are missing from response headers,
gitlab2prov
naivly assumes that there is only one page of the requested resource.Not all GitLab.com projects are affected. Updates will follow.
See also this section in the official GitLab documentation.
The text was updated successfully, but these errors were encountered: