-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make fetch sub phases pluggable #12400
Conversation
@clintongormley could you take a look and let me know what you think? |
I haven't looked at the pull request yet, but for more context: I was first on the fence about adding more entry points to plugins, until Britta brought up the selling point that it would allow to move some current fetch sub phases, like fielddata fields, to plugins, which I like as it would help keep the core small and focused on major features (see #10368). |
+1 to make sub fetch phases pluggable. If sub fetch phase implementations are better isolated, then this will automatically improve the design and code of these sub fetch phase implementations. |
for (FetchSubPhase phase : fetchSubPhases) { | ||
this.fetchSubPhases[counter] = phase; | ||
counter++; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or just fetchSubPhases.toArray(this.fetchSubPhases)
?
It looks good to me overall. Maybe @martijnvg and @dakrone should have a look since they worked on inner hits (which is handled differently in your PR) and fielddata fields. One thing I'm wondering when looking at the changes is whether we actually need hasFetchSubPhaseContext. It looks to me that we always do something like: |
|
||
package org.elasticsearch.search.fetch; | ||
|
||
public interface FetchSubPhaseContext { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you document this class?
left some comments, I agree with Adrien's comment about potentially getting rid of |
Thanks a lot for the feedback! I removed To proceed: Do we want to merge this after review cycles are done and then later convert each of the other sub phases in separate pull requests? Or should I do all at once? |
I'm all for doing things incrementally. Less long lived branches the better. |
@Override | ||
public void parse(XContentParser parser, SearchContext context) throws Exception { | ||
protected void innerParse(XContentParser parser, FetchSubPhaseContext fetchSubPhaseContext) throws Exception { | ||
FieldDataFieldsContext fieldDataFieldsContext = (FieldDataFieldsContext) fetchSubPhaseContext; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe FetchSubPhaseParseElement should be parameterized?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm +1 if it fits nicely, but otherwise I'd like us to be cautious with generics. Our code base probably makes too much use of generics today, to a point where they don't help avoid casting anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll make it parameterized and then we can decide.
+1 this looks good. I'm good with inner hits being handled differently than all the other sub fetch phases. |
839f473
to
d113b59
Compare
7ec0174
to
1397ff4
Compare
1397ff4
to
b86d07f
Compare
Thanks again for the feedback! I did my best to parameterize all places where I had casts before. It works but I am unsure if this is the right way. Can you take another critical look? |
public Map<String, ? extends SearchParseElement> parseElements() { | ||
ImmutableMap.Builder<String, SearchParseElement> parseElements = ImmutableMap.builder(); | ||
parseElements.put("term_vectors_fetch", new TermVectorsFetchParseElement()); | ||
return parseElements.build(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd write this ImmutableMap.of("term_vectors_fetch", new TermVectorsFetchParseElement());
. Not that it matters either way.
@nik9000 thanks again for the comments! updated pr and happily awaiting next review round |
this.fetchSubPhases = new FetchSubPhase[]{scriptFieldsPhase, matchedQueriesPhase, explainPhase, highlightPhase, | ||
fetchSourceSubPhase, versionPhase, fieldDataFieldsFetchSubPhase, innerHitsFetchSubPhase}; | ||
this.fetchSubPhases = fetchSubPhases.toArray(new FetchSubPhase[fetchSubPhases.size() + 1]); | ||
this.fetchSubPhases[fetchSubPhases.size()] = innerHitsFetchSubPhase; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you want to add sorting here? I forget what the outcome of that discussion was. Right now the order is random and that might be wrong for some use cases. Or maybe we just wait until we see such use cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could not think of any use case and I believe the order was random before. Tests pass so I guess that is OK?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Find by me.
Ok - I left one comment but it can wait. I feel good about this. |
LGTM |
1 similar comment
LGTM |
Uh. forgot to squash before merging. sorry...will remember next time. |
This is wip, only opening this pull request so that it is easier to discuss.
I made the fetch sub phases pluggable to see if this adds too much complexity. With this change new or existing fetch sub phases such as fielddata fields or script fields can be plugged in.
As a proof of concept I changed fielddata fields to use the plugin mechanism and added an example for
a plugged in fetch phase in the test.
While this adds a little more guice stuff, it also removes the need for implementing methods like
hasFieldDataFields()
andfieldDataFields()
in every subclass of SearchContext.In addition we could easily add features like #10729 and also move features like fielddatafields from core to a plugin immediately. Overall my feeling is that it would be a win to make sub phases pluggable.
It is work in progress because converting inner hits to use the plugin mechanism will be a little more
tricky. Also not all fetch phases are structured the same so converting all sub phases might need more work as well.