Query cache and use setParameters not working properly

[ffyliu]

🚨 First, please check:

• existing issues,
• Docs https://docs.objectbox.io/
• Troubleshooting page https://docs.objectbox.io/troubleshooting
• FAQ page https://docs.objectbox.io/faq

Describe the bug
When I query data, I cache the query object and do the query, the result is normal. Then the next same method query uses setParameters method to modify parameters, and the result is not right.

Basic info (please complete the following information):

• ObjectBox version 2.8.0
• Reproducibility: always
• Device: huawei mate 9
• OS: Android 9

attachments are corresponding demo
1.compile and install apk
2.start the app and click on the database test in the upper right corner
3.click the bottom button and check the log.

demo explanation:
Entity:Message
method getMsgListWithType use setParameters(Property<?> property, int[] values) not work
method getMsgListWithType2 use setParameters(String alias, int[] values) not work
method getMsgList use setParameter work right.

TestAll.zip

[greenrobot]

It might speedup response time if you could show the relevant code parts directly in the ticket.

[ffyliu]

Thank you very much for your reply. The main code is as follows：

@Entity
public class Message {

    private static Map<String, Query<Message>> queryMap = new ConcurrentHashMap<>(5);

    public static List<Message> getMsgList(String sid, boolean isDesc) {
        try {
            String keyName = "getMsgList".concat(isDesc ? "1" : "2");
            Query<Message> query = queryMap.get(keyName);
            if (query == null) {
                QueryBuilder<Message> queryBuilder = 
                        StoreManager.getBoxStore().boxFor(Message.class).query().equal(Message_.sid, sid);
                query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
                queryMap.put(keyName, query);
            } else {
                query.setParameter(Message_.sid, sid);
            }
            return query.find();
        } catch (Exception e) {
            return new LinkedList<>();
        }
    }

    public static List<Message> getMsgListWithType(String sid, int[] types, boolean isDesc) {
        try {
            String keyName = "getMsgListWithType".concat(isDesc ? "1" : "2");
            Query<Message> query = queryMap.get(keyName);
            if (query == null) {
                QueryBuilder<Message> queryBuilder = StoreManager.getBoxStore().boxFor(Message.class).query()
                        .equal(Message_.sid, sid)
                        .in(Message_.tp, types);
                query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
                queryMap.put(keyName, query);
            } else {
                query.setParameter(Message_.sid, sid).setParameters(Message_.tp, types);
            }
            return query.find();
        } catch (Exception e) {
            return new LinkedList<>();
        }
    }

    public static List<Message> getMsgListWithType2(String sid, int[] types, boolean isDesc) {
        try {
            String keyName = "getMsgListWithType2".concat(isDesc ? "1" : "2");
            Query<Message> query = queryMap.get(keyName);
            if (query == null) {
                QueryBuilder<Message> queryBuilder = StoreManager.getBoxStore().boxFor(Message.class).query()
                        .equal(Message_.sid, sid)
                        .in(Message_.tp, types).parameterAlias("1");
                query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
                queryMap.put(keyName, query);
            } else {
                query.setParameter(Message_.sid, sid).setParameters("1", types);
            }
            return query.find();
        } catch (Exception e) {
            return new LinkedList<>();
        }
    }
    @Id
    public long id;
    @Index
    public long serverId; // 消息id
    @Index
    public String fr; // 发送者用户名
    public int tp; // 消息类型
    public int s; // 发送平台
    public String m;
    public String fn; // 发送者名字
    public String nn; // 发送者昵称,在昵称发生变化时,需要进行对应消息的更新.
    public long ts; // 发送时间
    public String oz; // 扩展字段
    public String at; // 是否有@别人
    @Index
    public String sid; // topicId
    public Message(int tp, String sid, String m) {
        this.tp = tp;
        this.sid = sid;
        this.m = m;
    }
}

method getMsgListWithType use setParameters(Property<?> property, int[] values) not work
method getMsgListWithType2 use setParameters(String alias, int[] values) not work
method getMsgList use setParameter work right.

[greenrobot]

I didn't see anything obvious with your code. 🤔

Maybe this helps: this unit test ensures that setParamters() with int[] works.

PS.: is queryMap the same static instance? That would explain it. You need to use different keys for different queries.

[ffyliu]

sorry,the testCase code is below:

public static final String[] SIDS = {"adfadfasdfqweqweadfasd", "vczvcvzcdfgafgwaasdasd", "qfhgsdfgsasdasdasdfas", "khjkkjhdafhadjfhasd", "qwyeqwyudnadhfahfadf"};

@DebugLog
@OnClick(R.id.tv_start_test)
public void startQueryTest() {
    Log.d(TAG, "startQueryTest start");
    checkAndAddData();
    int[] types = new int[]{1, 3};
    for (String sid : SIDS) {
        Log.d(TAG, "startQueryTest sid=" + sid + " start");
        List<Message> messages = Message.getMsgListWithType(sid, types, false);
        Log.d(TAG, "startQueryTest sid=" + sid + ",method1MsgSize=" + messages.size());
        messages = Message.getMsgListWithType2(sid, types, false);
        Log.d(TAG, "startQueryTest sid=" + sid + ",method2MsgSize=" + messages.size());
        messages = Message.getMsgList(sid, false);
        Log.d(TAG, "startQueryTest sid=" + sid + ",method3MsgSize=" + messages.size());
    }
    Log.d(TAG, "startQueryTest end");
}

@DebugLog
public void checkAndAddData() {
    long count = Message.getMsgNum();
    Log.d(TAG, "checkAndAddData count=" + count);
    if (count >= 2000) {
        Log.d(TAG, "checkAndAddData data is OK");
        return;
    }
    count = Message.clearMessages();
    Log.d(TAG, "checkAndAddData clearCount=" + count);
    List<Message> messageList = new LinkedList<>();
    int num = 300;
    for (String sid : SIDS) {
        for (int i = 0; i < num; i++) {
            messageList.add(new Message(i / 100 + 1, sid, "随机测试大数据发哈加快速度华发商都后方可哈师大" + i));
        }
        num += 100;
    }
    Message.addOrUpdateMessage(messageList);
    Log.d(TAG, "checkAndAddData success and nowCount=" + Message.getMsgNum());
}

the testcase log is below:

2020-11-18 10:14:05.009 14438-14438/com.sogou.test V/QueryCacheFragment: ⇢ startQueryTest()
2020-11-18 10:14:05.009 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest start
2020-11-18 10:14:05.010 14438-14438/com.sogou.test V/QueryCacheFragment: ⇢ checkAndAddData()
2020-11-18 10:14:05.142 14438-14438/com.sogou.test D/QueryCacheFragment: checkAndAddData count=2500
2020-11-18 10:14:05.142 14438-14438/com.sogou.test D/QueryCacheFragment: checkAndAddData data is OK
2020-11-18 10:14:05.142 14438-14438/com.sogou.test V/QueryCacheFragment: ⇠ checkAndAddData [132ms]
2020-11-18 10:14:05.142 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd start
2020-11-18 10:14:05.149 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method1MsgSize=200
2020-11-18 10:14:05.154 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method2MsgSize=200
2020-11-18 10:14:05.160 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method3MsgSize=300
2020-11-18 10:14:05.160 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd start
2020-11-18 10:14:05.161 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd,method1MsgSize=0
2020-11-18 10:14:05.162 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd,method2MsgSize=0
2020-11-18 10:14:05.169 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=vczvcvzcdfgafgwaasdasd,method3MsgSize=400
2020-11-18 10:14:05.169 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas start
2020-11-18 10:14:05.169 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas,method1MsgSize=0
2020-11-18 10:14:05.170 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas,method2MsgSize=0
2020-11-18 10:14:05.180 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qfhgsdfgsasdasdasdfas,method3MsgSize=500
2020-11-18 10:14:05.180 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd start
2020-11-18 10:14:05.181 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd,method1MsgSize=0
2020-11-18 10:14:05.182 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd,method2MsgSize=0
2020-11-18 10:14:05.192 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=khjkkjhdafhadjfhasd,method3MsgSize=600
2020-11-18 10:14:05.192 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf start
2020-11-18 10:14:05.193 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf,method1MsgSize=0
2020-11-18 10:14:05.194 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf,method2MsgSize=0
2020-11-18 10:14:05.204 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest sid=qwyeqwyudnadhfahfadf,method3MsgSize=700
2020-11-18 10:14:05.204 14438-14438/com.sogou.test D/QueryCacheFragment: startQueryTest end
2020-11-18 10:14:05.205 14438-14438/com.sogou.test V/QueryCacheFragment: ⇠ startQueryTest [195ms]

the question is:
1. method getMsgList use only setParameter to modify the sid param,the result is always right
2.method getMsgListWithType and getMsgListWithType2 use setParameter and setParameters to modify the sid and tp param,the result is first with create right,next with cache is always wrong.

[ffyliu]

I didn't see anything obvious with your code. 🤔

Maybe this helps: this unit test ensures that setParamters() with int[] works.

PS.: is queryMap the same static instance? That would explain it. You need to use different keys for different queries.

the queryMap is static for the three methods and the query cache keys of the three methods are all different。

[greenrobot-team]

Played with the given example and indeed it looks like setParameter is broken. E.g. replacing with:

Query<Message> query = queryMap.get(keyName);
if (query == null) {
    QueryBuilder<Message> queryBuilder = StoreManager.getBoxStore().boxFor(Message.class).query()
            .equal(Message_.sid, "")
            .in(Message_.tp, new int[]{});
    query = (isDesc ? queryBuilder.orderDesc(Message_.serverId) : queryBuilder.order(Message_.serverId)).build();
    queryMap.put(keyName, query);
}
query.setParameter(Message_.sid, sid).setParameters(Message_.tp, types);
return query.find();

so setParameter is always used: on the first try 0 results are returned as well.

However, I can't reproduce this (it's not about multiple conditions, not about String and Integer, not about count, not about desktop vs Android). It works when switching the in to an equal condition, so not an issue with the example data either.

I'll try to create a minimal example that copies what the given example does, and maybe I'll find something.

Edit: query parameters also look fine:

2020-11-23 16:06:53.998 10131-10131/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd start
2020-11-23 16:06:54.001 10131-10131/com.sogou.test I/Box: Parameters for query #2:
    (sid ==(i) "adfadfasdfqweqweadfasd"
     AND tp in [1|3])
2020-11-23 16:06:54.001 10131-10131/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method1MsgSize=0
2020-11-23 16:06:54.002 10131-10131/com.sogou.test I/Box: Parameters for query #3:
    (sid ==(i) "adfadfasdfqweqweadfasd"
     AND tp in [1|3])
2020-11-23 16:06:54.004 10131-10131/com.sogou.test D/QueryCacheFragment: startQueryTest sid=adfadfasdfqweqweadfasd,method2MsgSize=100
2020-11-23 16:06:54.004 10131-10131/com.sogou.test I/Box: Parameters for query #4:
    sid ==(i) "adfadfasdfqweqweadfasd"

[ffyliu]

Thank you very much for your reply.
Looking forward to good news.😊

[greenrobot-team]

@ffyliu You found a bug, thank you!

It appears this only affects the Android 32-bit libraries (so x86 and armeabi-v7) running on 64-bit devices. The 64-bit libraries (x86_x64, arm64-v8a), which most devices probably use are not affected.

Update: running the Android 32-bit libraries on 32-bit devices also do not have this issue.

Your app/build.gradle excludes the 64-bit libraries, hence the issue always occurs in your example app:

ndk { abiFilters "armeabi", "armeabi-v7a", "x86", "mips" }

As a workaround you can use a combination of equal + or/and to achieve the same result as an in condition.

You should update (or even remove) your ABI filters to include the 64-bit libraries. ObjectBox only supports x86, x86_x64, armeabi-v7, arm64-v8a.

[ffyliu]

OK，I will remove my app ndk filters and retry。
Thank you very much。

@greenrobot-team I change the ndk filters to below
ndk { abiFilters "armeabi", "armeabi-v7a", "arm64-v8a", "x86", "x86_64", "mips" }
or remove it，the demo run right，Thanks.👍

[greenrobot-team]

Update: this issue only occurs when running the Android 32-bit libraries (x86 and armeabi-v7) on 64-bit capable hardware. In the above case the use of ABI filters excluded the 64-bit libraries.

Removing/updating the ABI filters so the 64-bit libraries are included resolves this issue.

Projects that do not exclude the 64-bit libraries are not affected. Put differently: the 32-bit Android libraries (x86 and armeabi-v7) work fine on 32-bit hardware.

Regardless, a fix to support running the 32-bit lib on 64-bit hardware exists internally and will be released in an upcoming release.

Updated my message above in that regard.

Side note: since August 2019 Play Store requires new apps and app updates to ship 64-bit versions of native libraries.