Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[5.6] Add builder/grammar support for row values in where condition #22446

Merged
merged 1 commit into from
Dec 29, 2017

Conversation

mfn
Copy link
Contributor

@mfn mfn commented Dec 15, 2017

Support for "row values" is mostly useful when implementing pagination with keysets aka seek method.

Well known pagination with offset/limit has almost linearly bad performance characteristics with every "page" turned. They can't make use of an index to efficiently jump to the result page and thus have to always read from the beginning.

An alternative way is to define predicates to jump to the desired page which can make use of indices. Characteristics of such paginations are:

  • no indicator how many pages; best used with "endless scrolling/loading" approaches
  • usually stable pages (depending on predicates). Insertion of data on previous pages will not alter the result of the current page

This PR implements the low-level building block. Example:

  • instead of ->where(…)->orderBy(…)->offset(100)->limit(10)
  • you could use ->where(…)->whereRowValues(['last_update', 'ticket_numbner'], '<', ['2017-12-15 23:00:00', 123])->orderBy(…)->limit(10)
    (iff the business logic of your application allows it)

The benefits/performance characteristics as well as in-depth examples how and why this works are well described at:

Note

$builder->whereRaw('(last_update, ticket_number) < (?, ?)', […, …]);
  • The naming RowValues may not be obvious to related to this "keyset pagination" or "seek method". But OTOH this seems to be the term describing this particular syntax (I'm not an expert on this topic), that's why I choose it instead of ->whereSeek or ->whereKeysetPagination
    This may be left open for the future to directly support this via something like ->paginateKeyset() (just thinking out loud)
  • Not every database out there supports this syntax. PostgresSQL supports it since a long time; MySQL does but with caveats, etc. There's some information at http://use-the-index-luke.com/sql/partial-results/fetch-next-page about it (not sure if it is accurate/up2date).
  • There are ways to emulate this behaviour for databases not supporting this syntax (again, see http://use-the-index-luke.com/sql/partial-results/fetch-next-page), but this is out of the scope of this PR:
SELECT *
  FROM ( SELECT *
           FROM tickets
          WHERE last_update <= ?
            AND NOT (last_update = ? AND number >= ?)
          ORDER BY last_update DESC, number DESC
       )
 WHERE rownum <= 10

@mfn mfn changed the title [QUERY] Add builder/grammar support for row values in where condition [5.6][QUERY] Add builder/grammar support for row values in where condition Dec 15, 2017
@sisve
Copy link
Contributor

sisve commented Dec 16, 2017

So this implements a partially supported syntax for something that is already fully supported in the "full" syntax?

@tillkruss tillkruss changed the title [5.6][QUERY] Add builder/grammar support for row values in where condition [5.6] Add builder/grammar support for row values in where condition Dec 16, 2017
@mfn
Copy link
Contributor Author

mfn commented Dec 16, 2017

Thanks for the feedback!

Guess it the depends on POV and expectations. Basically, using DB::raw() you can write almost any SQL statement without the need for any of the existing where…-shortcuts 🤷‍♀️ ?

OTOH (although I'm not trying to convince/evangelize anyone here), when I implemented this recently I wished I had a ready-made approach for, because why not; makes code more readable etc.
I think it may help raise awareness also a bit for developers who may not know there are alternative ways to offset/limit which have better performance characteristics (iff the business logic supports it).

But maybe you're right because not every database out there might support that syntax? Not sure how this is properly dealt with in the Builder though. Feedback for that welcome!

@mfn
Copy link
Contributor Author

mfn commented Dec 18, 2017

I could also imagining this as a building block for a future paginator extension which makes use of this technique instead of offset/limit

@mpyw
Copy link
Contributor

mpyw commented Jun 18, 2021

According to SQL Feature Comparison, SQLServer does not support Tuple Comparison syntax. So

(a, b, c) > (1, 2, 3)

should be rewritten to

a=1 and b=2 and c>3
or
a=1 and b>2
or
a>1

If you use SQLServer, still lampager/lampager-laravel: Rapid pagination for Laravel may help implementing cursor pagination.

@driesvints
Copy link
Member

@mpyw probably best to attempt PR's. Merged PR's are never revisited here.

@mpyw
Copy link
Contributor

mpyw commented Jun 18, 2021

@driesvints Yes I know, I actually agree with the PR method. Just writing reference information for minor users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants