Enforce pagination (or at least limits) for arbitrary custom SQL

[simonw]

It's way too easy to accidentally trigger a page that returns 100,000 rows at the moment. I need to use the LIMIT clause on views and custom SQL - I can support pagination "next" links using offset as well.

[simonw]

By default I'll allow LIMIT and OFFSET up to a maximum of X (where X is let's say 50,000 to start with, but can be custom configured to a larger number or set to None for no limit).

[simonw]

I think the only safe way to do this is using SQLite .fetchmany(1000) - I can't guarantee that the user has not entered SQL that will outfox a limit in some way. So instead of attempting to edit their SQL, I'll always return 1001 records and let them know if they went over 1000 or not.

[simonw]

I'm going with a page size of 100 and a max limit of 1000

[simonw]