Refactoring of RegexpQuery #11896

alexksikes · 2015-06-26T18:44:39Z

Relates to #10217

This PR is against the query-refactoring branch.

javanna · 2015-07-01T16:59:31Z

heya @alexksikes sorry may I ask you to rebase please? #11974 changed things quite a bit, for the better!

alexksikes · 2015-07-02T00:44:57Z

@javanna OK you can go ahead, it's rebased. Thank you.

javanna · 2015-07-02T07:42:10Z

core/src/test/java/org/elasticsearch/index/query/RegexpQueryBuilderTest.java

+                    flags.add(flag);
+                }
+            }
+            query.flags(randomFrom(flags));


this way you always a single flag. I think I would do something similar to what we do here: https://github.com/elastic/elasticsearch/blob/feature/query-refactoring/core/src/test/java/org/elasticsearch/index/query/SimpleQueryStringBuilderTest.java#L65

I think what I wanted to do is generate arrays of unique flags. I'll fix this.

javanna · 2015-07-02T07:47:31Z

I did a first round of review, left a few comments

javanna · 2015-07-02T16:01:52Z

core/src/test/java/org/elasticsearch/index/query/RegexpQueryBuilderTest.java

+
+    @Test
+    public void testValidate() {
+        RegexpQueryBuilder commonTermsQueryBuilder = new RegexpQueryBuilder("", "regex");


rename variable to regexpQueryBuilder?

javanna · 2015-07-06T08:16:29Z

left a few comments

… Parser Relates to elastic#11896

alexksikes · 2015-07-07T08:08:46Z

@javanna thanks for the review, I updated the PR.

javanna · 2015-07-07T08:39:57Z

core/src/test/java/org/elasticsearch/index/query/BaseQueryTestCase.java

@@ -131,6 +132,7 @@ protected void configure() {
            String type = randomAsciiOfLengthBetween(1, 10);
            mapperService.merge(type, new CompressedXContent(PutMappingRequest.buildFromSimplifiedDef(type,
                    DATE_FIELD_NAME, "type=date",
+                    DATE_FIELD_NAME_REGEX, "type=date,format=YYYY\\-MM",


couldn't we re-use the existing date field that we already have?

i did try that but got failures when it gets to handle regex on dates. The accepted date format need all special regex parameters to be escaped, so the regex (here just a plain date) could be valid.

javanna · 2015-07-07T08:41:44Z

I did another round of review, maybe @cbuescher wants to have a quick look too?

cbuescher · 2015-07-07T09:29:43Z

core/src/main/java/org/elasticsearch/index/query/RegexpQueryBuilder.java

    /**
     * Sets the regexp maxDeterminizedStates.
     */
    public RegexpQueryBuilder maxDeterminizedStates(int value) {
        this.maxDeterminizedStates = value;
-        this.maxDetermizedStatesSet = true;


This was a bug previously? Seems so, just trying to understand.

oh boy I didn't spot this :) we should definitely backport this to master

while refactoring some of the queries I did find quite a few bugs just like that. Should we always backport them to master? What's our overall policy on this matter?

I think you can answer for yourself at least for this one: would you be comfortable with 2.0 released with still this bug in it? Mayeb this fix should even be backported to 1.x, it's a bug and should be treated as such

If you ask me I'd be tempted to say that a bug is a bug and should definitely be backported. However, there are bugs which are only relevant in master, and so I thought that the query refactoring branch would be merged to master before 2.0 release. For this bug, I'll submit a fix for sure.

good! as for when we merge the branch...well we will do it when it's ready, most likely not before 2.0 but we don't know yet. One other thing about backporting fixes is that the branch is already big enough with the changes that we are making. If we can isolate non related fixes we simplify things a lot and clarify what happened when for the future.

didn't you send a PR against master to fix this?

it wasn't a bug: #12083

cbuescher · 2015-07-07T09:49:17Z

Did a quick round of review as well. Beeing late to the party, I first found the change from allowed regex in Builder from String to general Object confusing, also introducing much complexity that wasn't there before. I tried to follow the argument in the comments, especially seeing that value was Object and also parsed as that in the current parser code. However I feel that using numbers/date values as regex is a little bit strange, would be interested in the reason why this is allowed in the parser (it apparently was introduced long ago with Commit 22077d1).
Looking at how value = parser.objectBytes(); currenty cannot return Date object, I think this doesn't have to be supported since any value will be converted using toString() later anyway.

The parser takes an Object value, so should the builder. Relates to elastic#11896

javanna · 2015-07-07T10:04:43Z

I share your concerns @cbuescher around supporting Object rather than String, but thought that we already support objects everywhere (parser + mappers) so this really seemed like a leftover in the java api and it makes sense to fix it. let's ask for a second opinion from @jpountz ? :)

Value was improperly set to `true`. Relates to elastic#11896

Value was improperly set to true. Relates to #11896 Closes #12083

… Parser Relates to #11896 Closes #12067

alexksikes · 2015-07-13T07:48:56Z

@javanna Thanks for the review. I updated the PR accordingly.

javanna · 2015-07-13T08:00:30Z

core/src/main/java/org/elasticsearch/index/query/RegexpQueryBuilder.java

+
+    /**
+     *  Returns the value used in this query.
+     *  If necessary, converts internal {@link org.apache.lucene.util.BytesRef} representation back to string.


this comment is outdated

javanna · 2015-07-13T08:03:53Z

did another round, left a few comments

alexksikes · 2015-07-13T08:25:30Z

@javanna I updated the PR accordingly. Thank you.

javanna · 2015-07-13T08:29:43Z

I think this is ready (besides my last small comment above), but it needs some rebasing before it can get in

alexksikes · 2015-07-13T09:18:20Z

@javanna It's rebased

javanna · 2015-07-13T09:21:48Z

I think we may need to rebase again after #12200 I would wait for that one.

Relates elastic#11896 Closes elastic#12200

Relates #11896 Closes #12200

alexksikes · 2015-07-23T15:08:48Z

@javanna It's rebased you can take a look now.

javanna · 2015-07-23T15:15:20Z

core/src/test/java/org/elasticsearch/index/query/RegexpQueryBuilderTest.java

+
+    @Override
+    protected Query doCreateExpectedQuery(RegexpQueryBuilder queryBuilder, QueryParseContext context) throws IOException {
+        // NO COMMIT: fix to be removed to avoid NPE on unmapped fields


you can replace with //norelease so we don't forget but at least you can get this in while we fix this problem in master.

sure i'll replace that

javanna · 2015-07-23T15:16:17Z

left two minor comments, LGTM though

Relates to #10217 Closes #11896 This PR is against the query-refactoring branch.

Relates to elastic#10217 Closes elastic#11896 This PR is against the query-refactoring branch.

Value was improperly set to true. Relates to elastic#11896 Closes elastic#12083

alexksikes added review labels Jun 26, 2015

alexksikes force-pushed the feature/query-refactoring-regexp branch from aa9bdfa to ba70a71 Compare July 2, 2015 00:40

javanna reviewed Jul 2, 2015
View reviewed changes

clintongormley mentioned this pull request Jul 6, 2015

Refactor parsing of queries/filters, aggs, suggester APIs #10217

Closed

alexksikes added a commit to alexksikes/elasticsearch that referenced this pull request Jul 6, 2015

Properly fix the default regex flag to ALL for RegexpQueryBuilder and…

b9cf5cc

… Parser Relates to elastic#11896

alexksikes mentioned this pull request Jul 6, 2015

Properly fix the default regex flag to ALL for RegexpQueryParser and Builder #12067

Closed

javanna reviewed Jul 7, 2015
View reviewed changes

cbuescher reviewed Jul 7, 2015
View reviewed changes

alexksikes added a commit to alexksikes/elasticsearch that referenced this pull request Jul 7, 2015

Fix RegexpQueryBuilder to support an Object value

690184e

The parser takes an Object value, so should the builder. Relates to elastic#11896

alexksikes mentioned this pull request Jul 7, 2015

Fix RegexpQueryBuilder to support an Object value #12076

Closed

alexksikes added a commit to alexksikes/elasticsearch that referenced this pull request Jul 7, 2015

fix RegexpQueryBuilder#maxDeterminizedStates

b7e26fa

Value was improperly set to `true`. Relates to elastic#11896

alexksikes added a commit that referenced this pull request Jul 7, 2015

fix RegexpQueryBuilder#maxDeterminizedStates

eeac8a5

Value was improperly set to true. Relates to #11896 Closes #12083

alexksikes added a commit that referenced this pull request Jul 7, 2015

Properly fix the default regex flag to ALL for RegexpQueryBuilder and…

5e02384

… Parser Relates to #11896 Closes #12067

javanna reviewed Jul 13, 2015
View reviewed changes

alexksikes mentioned this pull request Jul 13, 2015

RegexpQueryParser takes a String as value like its Builder #12200

Closed

alexksikes force-pushed the feature/query-refactoring-regexp branch from c7fbd35 to 5b9d5cc Compare July 13, 2015 09:17

alexksikes added a commit to alexksikes/elasticsearch that referenced this pull request Jul 13, 2015

RegexpQueryParser takes a String as value like its Builder

ad6a922

Relates elastic#11896 Closes elastic#12200

alexksikes added a commit that referenced this pull request Jul 16, 2015

RegexpQueryParser takes a String as value like its Builder

d0256da

Relates #11896 Closes #12200

alexksikes force-pushed the feature/query-refactoring-regexp branch from 5b9d5cc to 05d3fe6 Compare July 23, 2015 15:08

javanna reviewed Jul 23, 2015
View reviewed changes

alexksikes added a commit that referenced this pull request Jul 23, 2015

Refactoring of RegexpQuery

93674b8

Relates to #10217 Closes #11896 This PR is against the query-refactoring branch.

alexksikes closed this Jul 23, 2015

alexksikes force-pushed the feature/query-refactoring-regexp branch from e0356dd to 93674b8 Compare July 23, 2015 15:48

alexksikes deleted the feature/query-refactoring-regexp branch July 23, 2015 15:48

mute pushed a commit to mute/elasticsearch that referenced this pull request Jul 29, 2015

Refactoring of RegexpQuery

0f3f8a0

Relates to elastic#10217 Closes elastic#11896 This PR is against the query-refactoring branch.

mute pushed a commit to mute/elasticsearch that referenced this pull request Jul 29, 2015

fix RegexpQueryBuilder#maxDeterminizedStates

f8cb5fe

Value was improperly set to true. Relates to elastic#11896 Closes elastic#12083

clintongormley removed the review label Aug 7, 2015

clintongormley added :Search/Search Search-related issues that do not fall into other categories and removed :Query Refactoring labels Feb 14, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactoring of RegexpQuery #11896

Refactoring of RegexpQuery #11896

alexksikes commented Jun 26, 2015

javanna commented Jul 1, 2015

alexksikes commented Jul 2, 2015

javanna Jul 2, 2015

alexksikes Jul 2, 2015

javanna commented Jul 2, 2015

javanna Jul 2, 2015

javanna commented Jul 6, 2015

alexksikes commented Jul 7, 2015

javanna Jul 7, 2015

alexksikes Jul 7, 2015

javanna commented Jul 7, 2015

cbuescher Jul 7, 2015

javanna Jul 7, 2015

alexksikes Jul 7, 2015

javanna Jul 7, 2015

alexksikes Jul 7, 2015

javanna Jul 7, 2015

javanna Jul 23, 2015

alexksikes Jul 23, 2015

cbuescher commented Jul 7, 2015

javanna commented Jul 7, 2015

alexksikes commented Jul 13, 2015

javanna Jul 13, 2015

javanna commented Jul 13, 2015

alexksikes commented Jul 13, 2015

javanna commented Jul 13, 2015

alexksikes commented Jul 13, 2015

javanna commented Jul 13, 2015

alexksikes commented Jul 23, 2015

javanna Jul 23, 2015

alexksikes Jul 23, 2015

javanna commented Jul 23, 2015

Refactoring of RegexpQuery #11896

Refactoring of RegexpQuery #11896

Conversation

alexksikes commented Jun 26, 2015

javanna commented Jul 1, 2015

alexksikes commented Jul 2, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

javanna commented Jul 2, 2015

Choose a reason for hiding this comment

javanna commented Jul 6, 2015

alexksikes commented Jul 7, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

javanna commented Jul 7, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cbuescher commented Jul 7, 2015

javanna commented Jul 7, 2015

alexksikes commented Jul 13, 2015

Choose a reason for hiding this comment

javanna commented Jul 13, 2015

alexksikes commented Jul 13, 2015

javanna commented Jul 13, 2015

alexksikes commented Jul 13, 2015

javanna commented Jul 13, 2015

alexksikes commented Jul 23, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

javanna commented Jul 23, 2015