v1.22.50
Missing SOLR content is now rebuilt everywhere it is read
v1.22.49 rebuilt a missing term_info document on demand. It turned out four hydration paths had the same silent skip, and one of them was returning nothing at all.
The connectivity queries were returning empty answers. Not fewer rows — count 0, for any neuron class whose upstream/downstream_connectivity_query index had not been built:
FBbt_00110929 alpha/beta core Kenyon cell 0 rows → 236
FBbt_00048273 wing bristle mechanosensory neuron 0 rows → 604
152 of 4,000 sampled connectivity classes were in that state.
Four paths, one cause — records that reach the PDB after the bulk indexer last completed have nothing in vfb_json:
| path | reads | did |
|---|---|---|
_owlery_query_to_results |
anat_query, anat_image_query |
skipped the id — rows and count short |
_fetch_connectivity_entries |
up/downstream_connectivity_query |
skipped the class — empty result |
_bulk_fetch_per_instance_connectivity |
its own result cache | skipped, calling counts "a slight underestimate" |
get_hierarchy |
term_info |
skipped — and lost the whole branch beneath the term |
All four now collect what SOLR returned, work out what is missing, rebuild it and carry on. The index rebuilds borrow the bulk indexer's own queries, batched at its own REQUEST_BATCH_SIZE — one round trip rather than per-id work (500 ids in 1.49 s) — and a set larger than one batch is looped, never truncated. Uncached per-instance connectivity is simply computed, which also fills the cache. Deliberately unbounded: an individual call fixes itself even if that call is slow, rather than return a number that is wrong without saying so.
v1.22.49's term_info fallback was serving but never indexing. solr_client builds its update URL from SOLRserver/SOLRcollection, and a VFBquery deployment set neither, so the write returned False without a request and every miss paid a fresh Neo4j round trip. Both are now seeded from VFBquery's own SOLR client.
Two tests now assert contract rather than content — the exact type count on Ito2013 (DataSets gained stage labels) and the strict-decrease demand on the default connectome excludes.
Measured coverage against each indexer's own population: anat_image_query 0.5% missing, connectivity ~2.1%, cluster_expression 0%. anat_query has 279 more documents than its population — stale entries a fallback cannot fix.
What's Changed
- Rebuild missing SOLR content everywhere it is read, not just in term_info by @Robbie1977 in #110
Full Changelog: v1.22.49...v1.22.50
