Sanitizing selector values for the advanced text search operator (#=)

[teppokoivula]

I'm having trouble figuring out a way to sanitize user-provided query string that is used in context of the advanced text search operator. The general guideline is that $sanitizer->selectorValue() should be used (and it does, or there's a risk that the query may contain something potentially harmful, or at least "surprising"), yet I haven't found a clean way to make a query such as +"foo bar" or -"foo bar"survive this.

Is this an issue, something that hasn't been implemented (yet?), or perhaps a special case that should be handled in some other way?

I can kind of get this to work if I call selectorValue() with options 'useQuotes' => false, 'version' => 1 and wrap the query string in double quotes, but it seems potentially problematic to rely on V1 (there's no info if this is a permanent option or something that may be deprecated later on), and I'd prefer to avoid custom quote handling.

[teppokoivula]

Very loosely related: is there a typo in the selector operators docs page, or does "foo bar" really mean that this phrase MUST NOT appear?

-(foo bar) or "foo bar" indicates the phrase "foo bar" MUST NOT appear

[teppokoivula]

One more addition: according to the docs +(foo bar) and +"foo bar" should be equal, but the results I'm getting seem to signal otherwise. Similarly if I try these in the test tool included in the 3.0.160 blog post, I get different set of results for those (although the title also seems to indicate that the sanitizing issue might be causing problems here):

[ryancramerdesign]

@teppokoivula That's a typo in the docs, thanks—fixed. The reason for the different grouping support (parenthesis vs quotes) is so that we can accommodate different API usage cases, since both parenthesis and quotes have other meanings in selectors. So when you are using them, you'd need to quote your query to indicate you are using those things not as part of the selector. For instance, if using parenthesis, then you'd want to quote the value, i.e. title#="+(foo bar) ..." or if using double quotes, then you'd want to single quote the value, i.e title#='+"foo bar" ...'.

I'm not sure the advanced search is an ideal one to take directly from user input, but the $sanitizer->selectorValue() will at least need to know what operator you are using in this case, since it would otherwise filter out some of the things that the advanced text search allows. So you'd want to do this:

$clean = $sanitizer->selectorValue($dirty, [ 'operator' => '#=' ]); 

[teppokoivula]

Thanks, @ryancramerdesign! I'll do some additional testing with selectorValue(). For some "advanced" search forms the advanced text search would be awesome, as it provides almost Google type logic.

By the way, I'm not sure if this was clear from my comments above, but the reason I wondered if the two versions (parenthesis/quotes) were really equal is that the test tool you set up earlier gives different results for those. I assume that's due to sanitizing issues as well though.

[ryancramerdesign]

@teppokoivula That search demo in the blog post was setup for the first iteration of the new operators, but things have progressed beyond it, so I should probably remove that search demo, and maybe update it and move it into the Selectors part of the documentation.

Sanitizing an advanced search query is actually going to be simpler than others. For a selector where you want to allow the user to use parenthesis for grouping, I think it may be adequate just to sanitize with $sanitizer->text(), and remove double "quotes" and commas from the input value. Maybe something like this: $q = str_replace([ '"', ',' ], ' ', $input->get->text('q'));

Following that, you'd want also want to surround it double quotes in your selector, i.e. $pages->find('body#="' . $q . '"');. I think all of this is similar to what we have selectorValue() doing with you specify the operator is #=.

[teppokoivula]

[...] and maybe update it and move it into the Selectors part of the documentation.

Makes sense. I think this tool is quite helpful for actually understanding how different selector operators work :)

For a selector where you want to allow the user to use parenthesis for grouping [...]

That's a good point. Just for the record, I've been going with quote-version by default. This is mostly because that's pretty much the way search engines have always worked, while parenthesis are less well known. Quotes are something that I find users trying to do anyway, parenthesis not so much.

Anyway, thanks again for chiming in; I think this issue can be closed, as there's no real "issue" here as far as I can tell :)

[adrianbj]

Not sure just how relevant it is, but can I mention this: https://processwire.com/talk/topic/23873-pw-30160-%E2%80%93%C2%A0new-search-abilities/?tab=comments#comment-213732