v3.0.2
Warning
The module in this release fails to load with "The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception."
Use v3.0.3.
Fixes a performance problem that would show up on any sizeable library.
What was wrong
SQLite does not maintain query statistics on its own. Until ANALYZE has run there is no sqlite_stat1, and the planner does not merely pick a worse index — it stops using indexes at all.
Measured on a 500,000-file library (5 million similar pairs, 974 MB):
| Query | Without statistics | With statistics |
|---|---|---|
| Single-file lookup | 55 ms, full table scan | under 1 ms, index seek |
| Path prefix search | 126 ms, full scan | under 1 ms, index |
| Exact path match | 83 ms, full scan | under 1 ms, index |
| Load all files | 468 ms | 271 ms |
| Similar pairs by degree | 655 ms | 461 ms |
Sync-ImageStoreFolder performs one single-file lookup per file. At 55 ms each, a 500,000-file sync would take hours instead of minutes.
Sql Server maintained these statistics automatically, so nothing in the move to SQLite made the gap visible.
The fix
Statistics are now maintained in three places:
- The migration tool analyses after loading the data.
- Opening a database analyses if statistics are absent entirely — about 1.5 seconds, once, on a library of that size. Subsequent opens cost nothing.
Compress-ImageStoreDatabaserefreshes them, since compacting normally follows a large deletion.
PRAGMA optimize on close already handled incremental drift.
If you already migrated with v3.0.1
Nothing is wrong with your database — it simply has no statistics yet. Opening it with this release fixes that automatically on the first open. Running ANALYZE by hand would have had the same effect.