Image Search

Image Search (visual similarity)

A second, separate search: instead of matching the words an image produced, it matches what the image looks like. Type "sunset over a lake" and you get sunsets over lakes — no caption, no filename, no vision server involved.

It is powered by CLIP ViT-B/32, which encodes images and text into the same 512-dimensional space. Your query is embedded as text; the nearest neighbours are images.

How it differs from normal search

	Semantic search	Image search
Index	chunks table, embedding.dimensions	images table, 512-d CLIP
Built by	The indexing pipeline, automatically	On demand, from the Image Search panel
Model	Your embedding model	CLIP ViT-B/32, local, always
Matches	Captions, qualifications, extracted text	Visual content itself
Needs a vision server	For captions, yes	No — CLIP runs locally
Retrieval	Hybrid + RRF + reranking	Plain cosine nearest neighbours

They are complementary. Vision captions find "the invoice photographed on my desk" because the model read the invoice; CLIP finds "a photo of a desk" because it looks like one. Neither replaces the other.

Using it

1. Open the Image search panel.
2. Click Index images the first time. This walks your indexed roots, embeds every image found, and stores the vectors. On first run it also downloads the CLIP weights.
3. Type a visual description and search. Results come back as thumbnails, ranked by similarity; click one to open it.

Re-run Index images after adding a batch of photos — unlike document indexing, the visual index is not kept in sync automatically.

Scope and limits

• Formats: png, jpg, jpeg, gif, bmp, webp.
• 5000 images maximum per indexing run — a guard against a pathological tree.
• Batches of 16 images per CLIP call; unreadable files are skipped silently.
• One vector per image (no tiling), so a small detail in a large picture may not surface.
• Results are checked against disk before being shown; thumbnails are generated as data URLs and capped at 8 MB per image.
• Default 24 results, up to 100.

Practical notes

• Describe the picture, not the subject's name. CLIP has no idea who "Marie" is; it does know "a woman in a red coat on a beach".
• English tends to work better than French — CLIP ViT-B/32 was trained predominantly on English captions.
• It is entirely local, needs no configuration and no server, and is unaffected by your embedding-model choice. Changing the embedding model does not invalidate the visual index.
• The visual index lives in its own LanceDB table, alongside the document vectors in %APPDATA%\com.virgi.sensetree.

Related commands

Command	Role
index_images	Build/refresh the CLIP index over a scope (or all roots). Returns the count indexed.
image_search	Text query → visually similar images.
image_data_url	Thumbnail for the results grid.