Summary
RepoMapBuilder.Build() (src/CodeIndex/Database/RepoMapBuilder.cs:113-174) calls GetFileStats() once, then performs 8 separate .ToList() materializations on the same dataset for Languages, Modules, TopFiles, LargestFiles, SymbolRichFiles, ReferenceRichFiles, and Entrypoints. Each materialization re-evaluates the LINQ chain (GroupBy / OrderBy / Take / Select), so for repos with 100K+ files the same dataset is fully traversed 8 times. There is no shared cache of the grouped/sorted intermediates.
Where
src/CodeIndex/Database/RepoMapBuilder.cs:99 (fileStats retrieved once)src/CodeIndex/Database/RepoMapBuilder.cs:125, 141, 150, 157, 165, 172 (repeated .ToList() chains)
Suggested approach
- Materialize
fileStats once into List<RepoFileStat> and pass it explicitly into per-section builders.
- Pre-compute a
Lookup<language, List<file>> and a Lookup<module, List<file>> so module/language sections share work.
- Move
.Take(...) to the final assignment rather than mid-chain so intermediate results aren't discarded.
- Extract each section into a small named method that takes the materialized inputs to make hotspots obvious.
- Add a micro-benchmark targeting a 100K-file repo and assert GC allocations drop by ≥30%.
- Add a regression test that fixes the section ordering / counts so the refactor cannot subtly change output.
Summary
RepoMapBuilder.Build()(src/CodeIndex/Database/RepoMapBuilder.cs:113-174) callsGetFileStats()once, then performs 8 separate.ToList()materializations on the same dataset for Languages, Modules, TopFiles, LargestFiles, SymbolRichFiles, ReferenceRichFiles, and Entrypoints. Each materialization re-evaluates the LINQ chain (GroupBy / OrderBy / Take / Select), so for repos with 100K+ files the same dataset is fully traversed 8 times. There is no shared cache of the grouped/sorted intermediates.Where
src/CodeIndex/Database/RepoMapBuilder.cs:99(fileStats retrieved once)src/CodeIndex/Database/RepoMapBuilder.cs:125, 141, 150, 157, 165, 172(repeated.ToList()chains)Suggested approach
fileStatsonce intoList<RepoFileStat>and pass it explicitly into per-section builders.Lookup<language, List<file>>and aLookup<module, List<file>>so module/language sections share work..Take(...)to the final assignment rather than mid-chain so intermediate results aren't discarded.