Issue: Multiple fields with multiple filters loosing filters in DocumentState

[aj-swipejobs]

so when we have multiple filters for the same property we are loosing the first filter in the map. I have tracked it down to the class "DocumentState" method "andTogether".

eg. is let's say we have following filters.
"attrib_A": "$exists"
"attrib_A": "$ne: ''"
"attrib_B": "$exists"
"attrib_B": "$ne: ''"

Document state would loose "attrib_B": "$exists" in the "finished" map.

`
private Document andTogether(Document doc, String key, @nullable Object additional) {

    if (additional != null) {

        Document newSubdoc = new Document(key, additional);

        var extant = doc.remove(key);

        List<Document> and = (List<Document>) doc.get("$and");

        if (and != null) {

            and.add(newSubdoc);

        } else {

            and = new ArrayList<>();

            and.addAll(of(new Document(key, extant), newSubdoc));

            doc.put("$and", and);

            return newSubdoc;

        }

    }

    return doc;

}`

in this code.
first filter
finished = {"attrib_A" : "$exists"}
second filter
finished = {"$and": ["attrib_A" : "$exists", "attrib_A" : "$ne: ''"]}
third filter
finished = {"$and": ["attrib_A" : "$exists", "attrib_A" : "$ne: ''"], "attrib_B": "$exists"}
fourth filter
'var extant = doc.remove(key);' this would remove "attrib_B": "$exists" and "if (and != null)" would be true so
finished = {"$and": ["attrib_A" : "$exists", "attrib_A" : "$ne: ''", "attrib_B": "$ne: ''"],}

Is this a bug or how it's suppose to work?

[evanchooly]

I would need to see your actual code but this looks like a big fixed in 2.4.11. If you could try that version to be sure the problem persists, we can dig a bit deeper.

[aj-swipejobs]

Hi, @evanchooly it doesn't throw any errors, code just works but it's missing the eq filter when it runs the query.

[gnewitt]

To be fair Evan, my example above is perhaps a bit nonsensical and that may be why it fails.
The query does an eq() filter and then a ne(null) filter. Now I guess being logical, for a given field eq() will be true for a given result and any other filters should be unncessary. If that is the upshot here, then fair enough.

I guess if I rearrange the filters, I will get my answer on that score.

[gnewitt]

Having done this:

` void test_multiplefields_multiplefilters() {
List filters = of(
eq("aString", TEST_STRING_2),
ne("aString", null),
gt("number", NUMBER_1),
lt("number", NUMBER_5)
);
List results = testRepository.getListBy(filters);

    assertThat(results, is(notNullValue()));
    assertThat(results.isEmpty(), is(false));
    assertThat(results.size(), is(2));
}

`

I see this issue still manifested, such that I believe the gt() filter is ignored. This is using 2.4.11

Cheers,

Greg

[evanchooly]

this is all good to know. within reason, morphia should replicate your query structure regardless of how nonsensical it may be. let the server handle that but morphia should just transcribe what it's given. I'll try to tackle this tomorrow night (i'm presenting at a JUG tonight so I won't be home until late).

[gnewitt]

Delving deeper, the issue appears to be when there is an eq() filter for a particular field and then another filter for the same field is added. So as in the example above, if I set up an eq() filter and then follow it with another filter (in this case ne()), the eq() filter seems to be disregarded.