Summary
`vault_stats` reports `communities_coarse=12` and `communities_fine=8`. Textbook Hopfield dynamics (low β = broad basins, few clusters; high β = narrow basins, many clusters) says fine should be greater than coarse. On a 458-note vault, the fine partition has 2 giant (≥100-note) clusters and zero small ones, while the coarse partition has 1 giant and 2 tiny ones.
| level |
count |
min |
max |
mean |
giants ≥100 |
| 0 β=0.5 ('coarse') |
12 |
3 |
121 |
38 |
1 |
| 1 β=2.0 ('fine') |
8 |
11 |
123 |
57 |
2 |
Root cause
Three compounding issues in `src/neurostack/attractor.py`:
- Shared top-K sparsification (`TOP_K_NEIGHBORS=50`, line 61; applied at L237-238). At high β, softmax concentrates on the largest neighbour, and identical top-K neighbour sets funnel notes into a handful of hub-attractors. Raising β counter-intuitively helps hubs monopolise basins.
- Singleton merging (`_assign_communities`, L296-315) merges every 1-note basin into its nearest non-singleton. At β=2.0 many notes converge to their own attractor and then get absorbed — fine level has 0 singletons pre-merge.
- Only β differs between levels. Same similarity matrix, same initial state (`np.eye`, L244), same sparsification.
`communities_summarized: 20` is not a third count — it's `12+8` concatenated (rows from both levels live in the same table). See `tools/search_tools.py:400-402` and `community.py:183-190`.
Proposed fix
- Per-level top-K. Add `TOP_K_COARSE=80` and `TOP_K_FINE=20` in `attractor.py:61`; thread through `_attractor_convergence`.
- Gate singleton-merge at fine level. Add `merge_singletons: bool = True` arg to `_assign_communities`; call with `False` at β=2.0 or only merge if similarity to a non-singleton exceeds a threshold.
- Report sizes + modularity in stats. `search_tools.py:394-402` — add `min/max/mean_size` and modularity per level so regressions are visible.
- Modularity sanity check after assignment in `attractor.py` — warn if `Q(fine) <= Q(coarse)`.
- Naming. If tuning can't reliably deliver `n_fine > n_coarse` on all vaults, rename `coarse`/`fine` → `broad`/`narrow` (describes attractor width, not partition-size). Update `BETA_COARSE`/`BETA_FINE` constants, stats keys, and `vault_communities` docstring.
Expected effect
Fine partition retains narrow basins. Hierarchy is either truly hierarchical or honestly named. Stats surface the size distribution so anomalies are obvious.
Key files
- `src/neurostack/attractor.py:42-50, 61, 237-238, 273-325, 417-441`
- `src/neurostack/tools/search_tools.py:394-402`
- `src/neurostack/community.py:183-190`
Summary
`vault_stats` reports `communities_coarse=12` and `communities_fine=8`. Textbook Hopfield dynamics (low β = broad basins, few clusters; high β = narrow basins, many clusters) says fine should be greater than coarse. On a 458-note vault, the fine partition has 2 giant (≥100-note) clusters and zero small ones, while the coarse partition has 1 giant and 2 tiny ones.
Root cause
Three compounding issues in `src/neurostack/attractor.py`:
`communities_summarized: 20` is not a third count — it's `12+8` concatenated (rows from both levels live in the same table). See `tools/search_tools.py:400-402` and `community.py:183-190`.
Proposed fix
Expected effect
Fine partition retains narrow basins. Hierarchy is either truly hierarchical or honestly named. Stats surface the size distribution so anomalies are obvious.
Key files