Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pagination to the REST API v3 #2060

Closed
Zejnilovic opened this issue May 4, 2022 · 1 comment · Fixed by #2086
Closed

Add pagination to the REST API v3 #2060

Zejnilovic opened this issue May 4, 2022 · 1 comment · Fixed by #2086
Assignees
@dk1844
Labels
feature New feature priority: high Critical to the health of the project under discussion Requires consideration before a decision is made whether/how to implement

Comments

@Zejnilovic
Copy link
Contributor

Background

There are calls which pull a lot of data. /datasets and /runs. These calls should be paginated, maybe others as well.

Feature

Add pagination to the REST API v3
@Zejnilovic Zejnilovic added feature New feature priority: undecided Undecided priority to be assigned after discussion under discussion Requires consideration before a decision is made whether/how to implement labels May 4, 2022
@dk1844 dk1844 added priority: high Critical to the health of the project and removed priority: undecided Undecided priority to be assigned after discussion labels Jun 7, 2022
@dk1844 dk1844 self-assigned this Jun 7, 2022
dk1844 added a commit that referenced this issue Jun 9, 2022
@dk1844
#2060 very basic pagination for Datasets (untyped List, no extra fields)
fdce098
dk1844 added a commit that referenced this issue Jun 9, 2022
@dk1844
#2060 pagination for Datasets via Paginated wrapper (features `offs…
859006b 
…et`: Int, `limit`: Int, `truncated`: Boolean) + IT coverage

 - test coverage uses `TestPaginated` (array-based) to correctly asses wrapped types while deserialization. Custom matcher added to check conforming TestPaginated objects
dk1844 added a commit that referenced this issue Jun 9, 2022
@dk1844
#2060 pagination for Schema, PDs, MTs via Paginated IT coverage
8dd17ee
dk1844 added a commit that referenced this issue Jun 9, 2022
@dk1844
#2060 pagination for RunSummary for /runs + IT
fe7ca82
dk1844 added a commit that referenced this issue Jun 10, 2022
@dk1844
#2060 pagination for RunSummary for /runs/{datasetName} + IT
c53c078
dk1844 added a commit that referenced this issue Jun 10, 2022
@dk1844
#2060 pagination for RunSummary for /runs/{datasetName}/{datasetVersi…
bf9a9c5 
…on} + IT
@dk1844 dk1844 mentioned this issue Jun 10, 2022
Merged
dk1844 added a commit that referenced this issue Jun 13, 2022
@dk1844
#2060 common pagination functionality for controllers moved to `Contr…
bbf27ac 
…ollerPagination` object
dk1844 added a commit that referenced this issue Jul 18, 2022
@dk1844
#2060 pagination: PR review updates
ec323a6
dk1844 added a commit that referenced this issue Jul 18, 2022
@dk1844 @benedeki @Zejnilovic
Feature/2060 pagination (#2086)
75de414 
* #2060
 - pagination for Datasets, Schema, PDs, MTs, RunSummary (/runs, /runs/{datasetName},/runs/{datasetName}/{datasetVersion})  via `Paginated` wrapper (features `offset`: Int, `limit`: Int, `truncated`: Boolean)
 - common pagination functionality for controllers moved to `ControllerPagination` object
 - pagination params sanitization -> invalid value (non-int), incorrect value (negative)
 - IT coverage
 - test coverage uses `TestPaginated` (array-based) to correctly assesses wrapped types while deserialization. Custom matcher added to check conforming TestPaginated objects


Co-authored-by: David Benedeki <benedeki@volny.cz>
Co-authored-by: Saša Zejnilović <zejnils@gmail.com>
@dk1844
Copy link
Contributor

dk1844 commented Jul 18, 2022

Release notes

Some potentionally large container-listing endpoints are now paginated, with offset (default: 0) and limit (default: 20).
Specifically, pagination is added to: GET of /api-v3/{datasets|schemas|mapping-tables|property-definitions/datasets} and /api-v3/runs[/{datasetName}[/{datasetVersion}]].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dk1844 dk1844

Labels
feature New feature priority: high Critical to the health of the project under discussion Requires consideration before a decision is made whether/how to implement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Feature/2060 pagination AbsaOSS/enceladus
2 participants
@dk1844 @Zejnilovic