Skip to content

v3.0.2

Choose a tag to compare

@github-actions github-actions released this 15 Aug 14:46
· 3 commits to master since this release

Warning

The module in this release fails to load with "The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception."
Use v3.0.3.

Fixes a performance problem that would show up on any sizeable library.

What was wrong

SQLite does not maintain query statistics on its own. Until ANALYZE has run there is no sqlite_stat1, and the planner does not merely pick a worse index — it stops using indexes at all.

Measured on a 500,000-file library (5 million similar pairs, 974 MB):

Query Without statistics With statistics
Single-file lookup 55 ms, full table scan under 1 ms, index seek
Path prefix search 126 ms, full scan under 1 ms, index
Exact path match 83 ms, full scan under 1 ms, index
Load all files 468 ms 271 ms
Similar pairs by degree 655 ms 461 ms

Sync-ImageStoreFolder performs one single-file lookup per file. At 55 ms each, a 500,000-file sync would take hours instead of minutes.

Sql Server maintained these statistics automatically, so nothing in the move to SQLite made the gap visible.

The fix

Statistics are now maintained in three places:

  • The migration tool analyses after loading the data.
  • Opening a database analyses if statistics are absent entirely — about 1.5 seconds, once, on a library of that size. Subsequent opens cost nothing.
  • Compress-ImageStoreDatabase refreshes them, since compacting normally follows a large deletion.

PRAGMA optimize on close already handled incremental drift.

If you already migrated with v3.0.1

Nothing is wrong with your database — it simply has no statistics yet. Opening it with this release fixes that automatically on the first open. Running ANALYZE by hand would have had the same effect.