-
Notifications
You must be signed in to change notification settings - Fork 677
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for keyset-based scrolling #2151
Comments
Junho, Kim commented Is there any update? |
wow, I found this 6 years old article asking for this kind of feature and stating that it is not very well known in the field, guess it is true |
@terence-sk same here... this link was referenced today in a hackernews thread :) It would be amazing to have this in spring-data. About a year ago in our company project, we imported a lot of data into our database, and then all paginated queries were suffering. Going through pages was extremely slow, and after some page, requests were timing-out at 30sec. We had to move paginated queries to elasticsearch in order to avoid offset paging that is used in spring-data (we duplicate most of our sql data in elasticsearch anyway for full-text-search support). |
Any thoughts on when keyset-based pagination will become available? |
@mp911de I know that you are pretty busy with other stuff, but any plans to implement that soon and integrated in spring-data-jdbc? |
I have no idea. We're currently busy with other topics that are required for Spring Data 3.0. If someone is interested in picking up this theme and crafting a pull request along with an implementation for JPA, Mongo, or Spring Data JDBC, we'd be happy to provide additional guidance. |
@mp911de I would like to help with this, but:
Is this ok for you? |
Any kind of progress is appreciated. You might want to explore existing keyset pagination implementations to collect some ideas. |
@mp911de Could you please point me to specific code. Any other guidance will be highly appreciated. |
Hi! I'd also be interested in collaborating, but have no real clue how to! Any kind of support would be really helpful! |
Keyset pagination is not pagination in the sense of page number and page size but rather an approach to resume cursor consumption where we've left off the last time. Taking a step back, keyset cursoring introduces the concept of consumption windows of data. We should make this concept explicit instead of squeezing cursoring into pagination. Another aspect of this is that we can provide means for index/offset-based cursoring and we can retrofit |
The intend of WindowIterator is to support users who need to iterate multiple windows. It keeps track of the position and loads the next window if needed so that the user does not have to interact with the position at all. Also remove the Window methods to get the first/last position and enforce the index based variant. Update the documentation to make use of the newly introduced API. See: #2151 Original Pull Request: #2787
closed by: #2787 |
Discussed spring-projects/spring-data-commons#2151 and implemented in spring-projects/spring-data-commons#2787 we can build on top and provide basic support for both imperative and reactive repositories. The support will be available only on the repository level in the first iteration, think ```java import java.util.UUID; import org.springframework.data.domain.ScrollPosition; import org.springframework.data.domain.Sort; import org.springframework.data.domain.Window; import org.springframework.data.neo4j.repository.Neo4jRepository; public interface ScrollingRepository extends Neo4jRepository<ScrollingEntity, UUID> { Window<ScrollingEntity> findTop4By(Sort sort, ScrollPosition position); } ``` and other derived finder methods that have a limit and a stable sort. If requested, further support can be added to the templates, too. Closes #2691.
Discussed in spring-projects/spring-data-commons#2151 and implemented in spring-projects/spring-data-commons#2787 we can build on top and provide basic support for both imperative and reactive repositories. The support will be available only on the repository level in the first iteration, think ```java import java.util.UUID; import org.springframework.data.domain.ScrollPosition; import org.springframework.data.domain.Sort; import org.springframework.data.domain.Window; import org.springframework.data.neo4j.repository.Neo4jRepository; public interface ScrollingRepository extends Neo4jRepository<ScrollingEntity, UUID> { Window<ScrollingEntity> findTop4By(Sort sort, ScrollPosition position); } ``` and other derived finder methods that have a limit and a stable sort. If requested, further support can be added to the templates, too. Closes #2691.
Closes #2878 Original pull request #2885 See spring-projects/spring-data-commons#2151
Closes #2878 Original pull request #2885 See spring-projects/spring-data-commons#2151
Jens Schauder opened DATACMNS-1729 and commented
Value based pagination basically uses queries of the form: select n rows after the one with id value :id
On relational databases these kind of queries show better performance than the typical: select n rows after the first m especially when m is large.
It is likely that other stores benefit as well
No further details from DATACMNS-1729
The text was updated successfully, but these errors were encountered: