queryparser whitespace escaping and documentation? [LUCENE-1122]

[asfimport]

as noted in this solr thread...

http://www.nabble.com/PhraseQuery-and-WildcardQuery-to14503609.html#a14503609

...it's possible to escape a whitespace character in the value of a term or prefix query by using a backslash so that the QueryParser will not treat it as "special" (ie: won't split on it when dividing the input into chunks for analysis).

at a minimum, this should be noted here...

http://lucene.apache.org/java/docs/queryparsersyntax.html#Escaping%20Special%20Characters

...but it got me wondering...

• is this a side effect of something else, or will QueryParser really respect this everywhere? even in field names? (i haven't tested) ... i think this is a result of QueryParser allowing you to escape any character even if it isn't "special" to the syntax
• shouldn't a space be considered "special" since it does trigger certain behavior? ... so shouldn't QueryParser.escape(String) escape spaces as well as the other special characters?

---

Migrated from LUCENE-1122 by Chris M. Hostetter (@hossman), resolved Mar 10 2013
Linked issues:

• queryparsersyntax.html escaping section needs beefed up [LUCENE-1000] #2076

[asfimport]

Steven Rowe (@sarowe) (migrated from JIRA)

shouldn't a space be considered "special" since it does trigger certain behavior? ... so shouldn't QueryParser.escape(String) escape spaces as well as the other special characters?

If escape() were changed to escape spaces, a previously supported use case (escaping an entire query, consisting exclusively of terms in the default field) would no longer be supported. In this case, the Javadocs should be changed to be very explicit about the supported use case(s).

[asfimport]

Mark Miller (@markrmiller) (migrated from JIRA)

Its because a term is madeup of TERM_START_CHAR + TERM_CHAR*, both of which can contain an ESCAPED_CHAR, which is an escape, '', followed by anything. So backslash space matches ESCAPED_CHAR, and the space is eaten as part of the term rather than being used as a whitespace break.

We could change it, or it sounds like its fine to leave it. If we leave it, I guess we can doc it and close this issue. Give me your preference and I'll be happy to do it.

[asfimport]

Erick Erickson (@ErickErickson) (migrated from JIRA)

SPRING_CLEANING_2013 We can reopen if necessary.