Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: cursor based list api. #436

Merged

Conversation

sgotti
Copy link
Member

@sgotti sgotti commented Sep 25, 2023

This commit adds code required to implement cursor based list apis.
It also changes the first list api (GetOrgMembers).
These api changes are breaking changes, so a related change in the agola
web repository is required and should be merged together.

Some implementation notes:

  • Only the gateway list APIs use opaque cursors.
  • The cursor is provided as a custom X-Agola-Cursor response header.
  • The provided cursor will be for the next batch of result
    using the same query options of the first call (limit excluded).
  • When a cursor is provided all the other query options cannot be
    provided with the exception of limit.
  • A default limit is forced if not provided.
  • The underlying services (configstore, runservice) will continue using
    explicit list options (start, sort etc...) based api instead of an
    opaque cursor.
  • The underlying services (configstore, runservice) will continue to not
    force any default limit. If a limit isn't provided then all the output
    results will be provided. This is required since their api must
    sometimes provide an atomicity in the results (i.e. GetSecrets tree).
    So doing multiple calls will break this atomicity since they'll
    execute different db transactions.

This commit adds code required to implement cursor based list apis.
It also changes the first list api (GetOrgMembers).
These api changes are breaking changes, so a related change in the agola
web repository is required and should be merged together.

Some implementation notes:

* Only the gateway list APIs use opaque cursors.
* The cursor is provided as a custom X-Agola-Cursor response header.
* The provided cursor will be for the next batch of result
  using the same query options of the first call (limit excluded).
* When a cursor is provided all the other query options cannot be
  provided with the exception of limit.
* A default limit is forced if not provided.
* The underlying services (configstore, runservice) will continue using
  explicit list options (start, sort etc...) based api instead of an
  opaque cursor.
* The underlying services (configstore, runservice) will continue to not
  force any default limit. If a limit isn't provided then all the output
  results will be provided. This is required since their api must
  sometimes provide an atomicity in the results (i.e. GetSecrets tree).
  So doing multiple calls will break this atomicity since they'll
  execute different db transactions.
Wrap http.Response in custom client Response type so, in future,
additional fields (based on http response headers or other http response
data) could be added to the response.
Add ListOptions to request list limit and sort direction

Add HasMore to Response fetched from an http header.
Add ListOptions to request list limit, sort direction and provide a
cursor to start the fetch from.

Add Cursor to Response fetched from an http header.
@sgotti sgotti force-pushed the initial_cursor_based_lists_get_org_members branch from 611143c to e77148b Compare September 25, 2023 13:05
@sgotti sgotti marked this pull request as ready for review September 26, 2023 07:23
@sgotti sgotti merged commit ee2d6f7 into agola-io:master Sep 26, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant