between() seems not to use an index when used with a bitwise AND (&)

[s-yata]

Description

When between() is used with a bitwise AND &, it seems not to use an index.

The following shows a list of commands to create a table.

table_create --name Data --flags TABLE_NO_KEY
column_create --table Data --name Value --type Int32
column_create --table Data --name Flag --type UInt32
load --table Data '[{"Value":860,"Flag":4}]'
...  // Value: [0, 1024), Flag: 1, 2, ..., or 512, #rows: 1M.
load --table Data '[{"Value":75,"Flag":2}]'
table_create --name DataValue --flags TABLE_PAT_KEY --key_type Int32
column_create --table DataValue --name Index --flags COLUMN_INDEX --type Data --source Value

Then, the first select is too slow compared with the other two selects.

> select --table Data --filter 'between(Value, 0, "include", 100, "include") && ((Flag & 1) != 0)'
[[0,1414418631.38877,5.48378705978394],[[[10283],...

> select --table Data --filter 'between(Value, 0, "include", 100, "include")'
[[0,1414418638.27391,0.0533609390258789],[[[103719],...

> select --table Data --filter 'between(Value, 0, "include", 100, "include") && (Flag == 1)'
[[0,1414418654.59622,0.0734946727752686],[[[10283],...

Also, if the index is disabled, the result is as follows:

> select --table Data --filter 'between(Value, 0, "include", 100, "include") && ((Flag & 1) != 0)'
[[0,1414417393.46272,5.49005770683289],[[[10283],...

> select --table Data --filter 'between(Value, 0, "include", 100, "include")'
[[0,1414417404.07405,4.96082067489624],[[[103719],...

> select --table Data --filter 'between(Value, 0, "include", 100, "include") && (Flag == 1)'
[[0,1414417416.61478,4.91911029815674],[[[10283],...

[kou]

少し調べたんですが、1箇所でもビット演算子（&とか）または算術演算子（+とか）を使ってしまうと、全体で1つもインデックスを使わなくなってしまうようでした。

クエリーオプティマイザー（scan_info_build()）ががんばらないといけないところなんですが、Cで書くのはツライので、mrubyを有効にしたときだけインデックスを使えるようになる、というのはありですかねぃ。。。

[kou]

Rubyで書いてみたらCでも簡単に書けるやつだったのでmrubyなしでもがんばるようにしました。

で、たぶん、masterだと直っていると思うので確認してもらえませんか？

[s-yata]

確認してみたところ，索引ありの状態で以下に示す結果となりました．
一つ目の & を含むクエリでも索引が効いているようです．
検索結果にも問題ありません．

> select --table Data --filter 'between(Value, 0, "include", 100, "include") && ((Flag & 1) != 0)'
[[0,1415062629.88284,0.089080810546875],[[[10283],...

> select --table Data --filter 'between(Value, 0, "include", 100, "include")'
[[0,1415062633.69047,0.0181918144226074],[[[103719],...

> select --table Data --filter 'between(Value, 0, "include", 100, "include") && (Flag == 1)'
[[0,1415062636.64962,0.0799901485443115],[[[10283],...

Bravo!

[kou]

よかったです！