?_searchmode=raw option for running FTS searches without escaping characters

[tunguyenatwork]

After the version 0.34. I am not able to use the wildchar in the _search option( or the full text search). It will not return any result unless I specify the whole word for text search.

If I use 'match :search || "*" ' in the sql statement then it will work as expected.

[simonw]

I'm afraid this is by design: supporting wildcards in that mode was more-or-less accidental and often resulted in unexpected user-facing syntax errors when users accidentally entered search text that included characters with special meaning to SQLite.

You can still execute wildcard searches but you have to do so using custom SQL statements.

I'm open to suggestions for more friendly ways to support this functionality. Would an alternative _search_wildcard= URL parameter be worth supporting I wonder? What's your specific use-case for wildcards?

[tunguyenatwork]

Thanks Simon,
My use case is using Datasette for full text search type ahead. That was working pretty well. The _search_wildcard= option will be awesome. Thanks

[simonw]

I'm going to call it ?_search_raw= because it will pass through the value as a raw (unprocessed) argument to SQLite FTS, which means it will expose features other than just wildcards: https://www.sqlite.org/fts5.html#full_text_query_syntax

[simonw]

?_search_raw= doesn't work because it clashes with an existing feature - you can use ?_search_colname= to search just within a specific column, and there could possibly be a column called raw.

I could go with ?_searchraw= - or maybe have an additional "mode" selection like ?_search=foo*&_searchmode=raw

[simonw]

Nice thing about the "search mode" concept is that I can optionally reflect it in the HTML UI in some way - maybe with a checkbox for "raw" mode that only shows up if the user hits the page with ?_searchmode=raw in the querystring.

It also hints at potentially adding further search modes in the future. Not sure if I'd do that but it's an interesting potential avenue to explore.

[simonw]

You can try this right now like so:

pip install https://github.com/simonw/datasette/archive/search-raw.zip

Then use the following:

?_search=foo*&_searchmode=raw`

[simonw]

Still needs documentation before I can land it in master.

[tunguyenatwork]

Awesome, thank you so much. I’ll try it out and let you know.

…

On Sun, Feb 23, 2020 at 1:44 PM Simon Willison ***@***.***> wrote: You can try this right now like so: pip install https://github.com/simonw/datasette/archive/search-raw.zip Then use the following: ?_search=foo*&_searchmode=raw` — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#676?email_source=notifications&email_token=AN3FXEFS6B22U2NOT6M5FULRELNY7A5CNFSM4KYIOIB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMWGYTI#issuecomment-590113869>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN3FXEANDJ6AIHGU4ADK4D3RELNY7ANCNFSM4KYIOIBQ> .

[simonw]

OK, this is in master now. You can install master using:

pip install https://github.com/simonw/datasette/archive/master.zip

Documentation here: https://datasette.readthedocs.io/en/latest/json_api.html#special-table-arguments