perf: harden and accelerate Django indexing

[forhappy]

Summary

• fixes duplicate Python function-symbol IDs with portable deterministic AST identities and collision-safe Program merging
• reports explicit process wall-clock totals for compass init, compass extract, and compass update
• accelerates cache-valid initialization and unchanged updates
• reduces forced full Django indexing from 21.08 seconds to below five seconds in the recorded qualification set
• parallelizes canonical Program serialization while preserving exact canonical bytes
• removes redundant graph copies, uses faster normalized-ID and lookup paths, and reduces cache/publication syscall pressure
• excludes generated graphify-out content and prunes files removed from configured scope

Root cause

The original Django run failed because two Python functions could receive the same provider symbol ID. After fixing that collision, sampling showed repeated AST ID rewriting, full graph inventory clones, sequential canonical serialization, Unicode normalization of already-normalized IDs, and thousands of cache/output syscalls.

Django qualification

Release binary against /Users/haipingfu/Github/django, using the saved all-project scope:

4310 matched files: 3096 code, 1128 documents, 3 papers, 83 images
3105 indexed files
50944 nodes, 190401 edges, 1747 communities
Program analysis: 3038 modules, 32046 summaries, 0 conflicts

The canonical program.json bytes, sorted node-ID set, and complete sorted edge-identity set match the established Django output exactly. All runs completed without the duplicate function-symbol error.

Current user-facing commands

command                                      Compass wall   process wall
compass init . --yes --force --timing          3.92s          3.98s
compass update . --timing                       0.95s          0.96s

The init run reused 3105 valid extraction entries and the immediate update reused the complete sealed generation.

Forced full rebuild qualification

After one discarded forced warm-up, three recorded isolated runs used a new temporary output directory each time:

/usr/bin/time -p compass update /Users/haipingfu/Github/django --force --out <new-temp-dir> --timing

sample   Compass wall   process wall   extracted   cached
1          4.91s          4.97s          3105        0
2          4.91s          4.96s          3105        0
3          4.62s          4.68s          3105        0

Every recorded forced build reparsed all indexed sources, rebuilt Program IR, clustering, reports, graph output, caches, and completion seals before exit.

Checks

• cargo fmt --all --check
• cargo test -p compass-analysis -p compass-core -p compass-files -p compass-graph -p compass-ir -p compass-languages -p compass-output -p compass-resolve
• cargo test -p compass-cli --tests -- --skip native_hooks_are_self_contained_safe_and_preserve_user_content
  • the skipped pre-existing hook assertion is path-sensitive because this worktree path contains python
• exact canonical Program byte comparison
• complete Django node-ID and edge-identity comparison
• repeated real /usr/bin/time -p init, update, and forced-update measurements
• git diff --check

Per repository-owner direction, graphify update . was not run.

Related: #46