Log a warning when doing iter_by_col_range without an index

[gefjon]

Description of Changes

Adds a warning to iter_by_col_range when there is no applicable index and the table has >1000 rows. Should make it easier for us to debug BitCraft performance. This is an obviously suboptimal implementation which is intended as a stopgap until we can properly implement this warning.

Concerns:

• Warnings go to the host logs, not the module logs. This will clearly need to change before 1.0.

Possible alternatives:

• Simply don't generate filter_by_{col} methods unless there's an index.
  • This won't stop unindexed queries via the query! macro, but will catch the vast majority of cases.
  • This will not allow unindexed filters on small tables.
• Generate the above methods, but mark them #[deprecated].
  • The point would not be to actually remove them, but because that's the only mechanism Rust exposes (AFAIK) for warning at a method's callsites.
  • This will warn on unindexed filters on small tables.
• Generate the above methods, but with an ugly name, like slow_unindexed_filter_by_{col}.

Example

Using a modified version of the spacetimedb-quickstart module with a find_named reducer which does Person::filter_by_name. Note that I've snipped some calls; the table count is in fact accurate.

phoebe@phoebe-framework:~/clockworklabs/SpacetimeDB/modules/spacetimedb-quickstart]$ for i in {1..1000}; do spacetime call demo add "person-$i" 20; done

[phoebe@phoebe-framework:~/clockworklabs/SpacetimeDB/modules/spacetimedb-quickstart]$ spacetime call demo find_named person-1000
2024-03-14T13:56:59.291994Z  WARN iter_by_col_eq:iter_by_col_eq: crates/core/src/db/datastore/locking_tx_datastore/mut_tx.rs:1068: iter_by_col_range without index: table Person has 1021 rows; scanning columns ["name"]    

TODO:

• Gate this behind a compile-time feature.
  • Should be enabled by default, but possible to disable so we can deploy the alpha without the logs.

API and ABI breaking changes

N/a.
[phoebe@phoebe-framework:~/clockworklabs/SpacetimeDB/modules/spacetimedb-quickstart]$ spacetime call demo list_over_age 20

Expected complexity level and risk

2 - social concern that we merge this and then never come back and write the better version.

[cloutiertyler]

The one modification I would make is that we should have a threshold.

It's totally valid to do this if you have less than ~1000 elements.

So it really should be a "large table iter_by_col_range" warning.

[kazimuth]

LGTM.

[mamcx]

Possible alternatives:

• Simply don't generate filter_by_{col} methods unless there's an index.

  • This won't stop unindexed queries via the query! macro, but will catch the vast majority of cases.
  • This will not allow unindexed filters on small tables.

• Generate the above methods, but mark them #[deprecated].

Or better:

• Indexes columns => filter_by_{col}, else => scan_by_{col}

[cloutiertyler]

This looks great to me