Add list intersection processor #269
Conversation
| Iterable<Object> listB = doc.getField(sourceFieldB); | ||
|
|
||
| Set<Object> intersection = new HashSet<>(intersection(listA, listB)); | ||
| if (isNotEmpty(intersection)) { |
There was a problem hiding this comment.
Why not create the field and leave the value empty?
There was a problem hiding this comment.
Will be easier for querying later (using exists), but can be changed (or even be configurable) if we want
There was a problem hiding this comment.
Seems counter-intuitive to me. But I am not sure what is the common behavior in other processors.
There was a problem hiding this comment.
I agree, it makes more sense to add an empty list. I just hope this doesn't produce more complexity later when querying it.
There was a problem hiding this comment.
LGTM.
Note that the automatic build currently does not run all unit tests - this is something @alexpalchuk started working on. So make sure to run all unit tests locally.
Similarly, the automatic upload to the maven repository is also broken...
There was a problem hiding this comment.
Yes I have already run all tests locally using mvn
| @ProcessorProvider(type = "listIntersect", factory = ListIntersectProcessor.Factory.class) | ||
| public class ListIntersectProcessor implements Processor { |
There was a problem hiding this comment.
I think it's better to call it arraysIntersect and ArraysIntersectProcessor, this way it's aligned with ElasticSearch type names. (there is no list in ES / OS).
plural form is probably better because we need 2 arrays to do the intersect
There was a problem hiding this comment.
Shit. I missed this comment @matvey-mtn.
This PR introduces a new processor that allows intersecting two list fields.
The result of the intersection is saved to a chosen target field.
If the intersection is empty, we just add an empty list to that field.