?_sort=rowid with _next= returns error

[eigenfoo]

For example:

• Go to https://cryptics.eigenfoo.xyz/clues/clues?_next=100 (this is the second page of results in a Datasette site)
• Search anything using the FTS search bar. For example, searching for hello will take you to https://cryptics.eigenfoo.xyz/clues/clues?_search=hello&_sort=rowid&_next=100
• A 500 Error: list index out of range is raised.

This is because the search URL includes the &_next=100 UTM parameter, carried over from where the FTS search was run. However, there isn't a second page in the search results, so a list index out of range error is raised. You can confirm that removing this UTM parameter from the URL returns the appropriate search results.

The FTS search request should strip any _next UTM parameter.

---

datasette, version 0.58.1
sqlite-utils, version 3.17

[simonw]

Here's the exception:

-> params[f"p{len(params)}"] = components[0]
(Pdb) list
603  	
604  	                # Figure out the SQL for next-based-on-primary-key first
605  	                next_by_pk_clauses = []
606  	                if use_rowid:
607  	                    next_by_pk_clauses.append(f"rowid > :p{len(params)}")
608  ->	                    params[f"p{len(params)}"] = components[0]
609  	                else:
610  	                    # Apply the tie-breaker based on primary keys
611  	                    if len(components) == len(pks):
612  	                        param_len = len(params)
613  	                        next_by_pk_clauses.append(

Debugger shows that components is an empty array, so components[0] cannot be resolved:

-> params[f"p{len(params)}"] = components[0]
(Pdb) params
{'search': 'hello'}
(Pdb) components
[]

So the bug is in this code:

datasette/datasette/views/table.py

Lines 604 to 617 in adb5b70

	# Figure out the SQL for next-based-on-primary-key first
	next_by_pk_clauses = []
	if use_rowid:
	next_by_pk_clauses.append(f"rowid > :p{len(params)}")
	params[f"p{len(params)}"] = components[0]
	else:
	# Apply the tie-breaker based on primary keys
	if len(components) == len(pks):
	param_len = len(params)
	next_by_pk_clauses.append(
	compound_keys_after_sql(pks, param_len)
	)
	for i, pk_value in enumerate(components):
	params[f"p{param_len + i}"] = pk_value

[simonw]

On further investigation this isn't related to _search at all - it happens when you explicitly sort by _sort=rowid and apply a _next

• https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200 works without an error (currently)
• https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200&_sort=rowid shows that error

[simonw]

I'll open a separate issue for removing _next= when running a search.

[simonw]

https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200&_sort=rowid is fixed now.