Skip to content

History

Revisions

  • Replace resolution UCB bandit with bandwidth sweep Resolution is already pinned by calibrate_resolution() (always finds 5.0). Bandwidth is the true free parameter: w_new = pow(w_orig, bw_orig/bw_new). evaluate_candidate() now takes raw edges + bandwidth, re-weights per trial, and applies marker penalization. run_adaptive_restarts() sweeps log-uniform bandwidth arms [0.05, 0.5] instead of resolution. Also pass config.bandwidth to qcfg.original_bandwidth in bin2.cpp.

    @genomewalker genomewalker committed Feb 19, 2026
    be3fef8
  • Fix marker name→accession translation for correct CheckM colocation scoring HMM profiles use human-readable NAMEs (e.g. Ribosomal_L39) but bacteria.ms uses Pfam/TIGRFAM accessions (PF00832, TIGR00967). Only 20 of 104 bacteria.ms markers were matching — the 20 TIGR ones whose NAME == accession. The 84 Pfam markers were silently failing, reducing colocation quality signal by 80%. Fix: parse NAME->ACC from the HMM file in load_hmm_accessions(), then translate hit.marker to normalized accessions when building hits_by_contig_. Seed selection (select_seeds) uses hits directly and is unaffected.

    @genomewalker genomewalker committed Feb 19, 2026
    4130e4c
  • Fix colocation pre-assign to use only bacteria.ms valid markers marker_hits_by_contig contains hits for all 206 HMM profiles, but colocation scoring should only use the single-copy markers defined in bacteria.ms (same set used by MarkerIndex/CheckM). The discrepancy caused 1070 pre-assignments instead of the expected ~269 (MarkerIndex reports). Fix: load bacteria.ms marker sets before the candidate loop and filter each contig's markers to valid_markers only. Contigs with zero valid markers are skipped.

    @genomewalker genomewalker committed Feb 19, 2026
    df24ce1
  • Filter pre-assignment candidates to >=1001bp (matches seed generator length filter) marker_hits_by_contig contains raw hmmer hits including short contigs that the seed generator filtered out. Pre-assigning short contigs adds noise — their embeddings are less reliable and they weren't part of the quality-filtered set.

    @genomewalker genomewalker committed Feb 19, 2026
    68811bc
  • Fix const operator[] compile error in pre-assignment seed marker init

    @genomewalker genomewalker committed Feb 19, 2026
    f2398ca
  • Fix colocation pre-assignment: marker_hits_by_contig is contig→markers not marker→contigs The variable name was misleading — SeedGenerator::get_hits_by_contig() returns contig→markers. The pre-assignment code treated it as marker→contigs, building a completely wrong reverse map, so all contig name lookups failed → 0 candidates.

    @genomewalker genomewalker committed Feb 19, 2026
    fdf4ad3
  • Use CheckM colocation to pre-assign all marker contigs before Leiden Theoretical max HQ = seed count (median marker copy count = 17 for this sample). We recover only 9-11 due to marker-contig fragmentation and contamination. Fix: for each non-seed marker-bearing contig, greedily assign it to the closest compatible seed genome (colocation blocks: no duplicate of a single-copy marker; embedding ranks: closest L2 seed wins). Process most-constrained contigs first to anchor unambiguous assignments. The result is stored as a warm-start initial partition for libleidenalg: seeds + pre-assigned contigs share a cluster (0..16), all other contigs start as unique singletons. libleidenalg now respects this partition instead of always using iota. This converts all ~269 marker contigs from free-floating singletons into genome-anchored starting points, constraining Leiden to only optimise the ~89k non-marker contigs.

    @genomewalker genomewalker committed Feb 18, 2026
    827f1f2
  • Add partition-level quality guard to Phase 1b Phase 1b (map equation refinement) moves individual contigs, but most contigs have no markers and get moved freely by the map equation. This cascades into large cluster merges (e.g. 282 → 143) without triggering quality penalties. Fix: snapshot scg_score_labels() before/after Phase 1b and revert if score decreased. The revert is rare when Phase 1b genuinely improves the partition, but prevents the common case where map equation merging destroys HQ bin boundaries.

    @genomewalker genomewalker committed Feb 18, 2026
    90d71f1
  • Fix restart scoring: use sum not mean for soft tiebreaker Normalising by n_viable rewarded fewer, larger clusters (higher mean completeness) over more, smaller clusters — causing the search to pick res=2.32 (177 clusters, 6 HQ) instead of res=5.0 (282 clusters, 10 HQ). Using the sum directly rewards having more viable bins at the right resolution.

    @genomewalker genomewalker committed Feb 18, 2026
    665e426
  • Fix sweep bandwidth double-transform and embeddings format mismatch Bug 1 (high): sweep loop called compute_comebin_edges() which already applies exp(-dist/bw), then re-applied exp(-w²/bw²) treating the already-transformed weight as a distance. Fixed by switching to compute_comebin_edges_distances() and applying exp(-dist/bw) once. Bug 2 (medium): embeddings.tsv writer emitted contig\tbin\tdim_0... but load_embeddings_tsv() parses all post-name tokens as floats, making bin the first embedding dimension when fed back via --embeddings. Fixed by removing the bin column from embeddings.tsv (assignments are in the output .fa files).

    @genomewalker genomewalker committed Feb 18, 2026
    cfac666
  • Fix Phase 2 objective consistency and restart tie-breaking Three bugs found by GPT-5.3-codex review: 1. Broken dup_excess+2 guard in CheckM mode: dup_excess can be 0 while CheckM contamination is >5%, making the guard allow <=2 children but acceptance requires new_dup<0 (impossible) -> splits always rejected. 2. Mixed objectives: trigger on CheckM cont>5% but accept on dup_excess decrease -> valid splits silently rejected. 3. Hard HQ count in scg_score_labels causes many ties -> weak restart ranking. Fixes: - Phase 2 uses fully separate CheckM and proxy branches (trigger + guard + accept all consistent within each branch). - CheckM fragmentation guard: max_children = min(12, 2+parent_bp/min_viable_bp) + spill_frac <= 25% + viable_children >= 2. - CheckM acceptance: HQ gain (primary) OR wcont drops >=1% with wcomp loss <=2%. - scg_score_labels: composite lexicographic score 1e6*905 + 1e4*705 + 1e2*505 + soft(avg comp-5*cont / 100), preserving strict priority ordering.

    @genomewalker genomewalker committed Feb 18, 2026
    9a59932
  • Use CheckM colocation scoring in restart search and Phase 2 scg_score_labels(): when CheckMQualityEstimator is available, return proper HQ bin count (completeness>=90%, contamination<=5% via colocation formula) instead of the soft Σ[present - 5*dup_excess] proxy. Phase 2 split criterion: replace dup_excess>0 (too aggressive, caused 320-cluster fragmentation) with CheckM contamination>5%. This should fix quality_leiden over-fragmentation from 300 → 861-1380 clusters. bin2.cpp: build CheckMQualityEstimator alongside MarkerIndex when bacteria.ms exists; pass to QualityLeidenBackend via set_checkm_estimator(). checkm_markers.h: add has_marker_sets() for clean availability check.

    @genomewalker genomewalker committed Feb 18, 2026
    97d4a6e
  • Fix scg_score_labels: only count viable clusters (>= min_bin_size bp) Without the size filter, SCG score monotonically increases with resolution: more clusters = less dup_excess per cluster, even when 95% of clusters are sub-200kb fragments that will never become bins. UCB bandit correctly optimised the (broken) objective, landing on res=50 with 1200 clusters. Fix: track per-cluster bp, skip clusters below restart_min_viable_bp (200kb).

    @genomewalker genomewalker committed Feb 18, 2026
    56d1c8d
  • Add config 8: best stack with --leiden-restarts 25

    @genomewalker genomewalker committed Feb 18, 2026
    68d5118
  • Implement best-of-K adaptive restart search for Leiden Joint (resolution x seed) search using 3-stage adaptive racing: - Stage 1: 7 log-uniform resolutions x 1 seed, Phase 1 SCG score - Stage 2: top 3 arms, UCB bandit (mean + beta*std), Phase 2 for contenders - Stage 3: best arm, up to 6 runs with early stop (patience=4), always Phase 2 Scores by scg_quality_score() not modularity. Hybrid fidelity: Phase 1 SCG for exploration, Phase 2 SCG for contenders/final selection. Caches best post-Phase-2 partition to avoid redundant re-splitting. Also extracts Phase 2 contamination splitting into run_phase2() helper. CLI: --leiden-restarts N (implies --quality-leiden, default N=1 = off)

    @genomewalker genomewalker committed Feb 18, 2026
    1a1017c
  • Add configs 6+7 to ablation: damage+map_equation and +CGR

    @genomewalker genomewalker committed Feb 18, 2026
    2599f49
  • Fix calibrate_lambda to use map equation delta when use_map_equation=true Was sampling modularity delta for s_R calibration even in map equation mode, giving incorrect λ scaling. Now branches on qconfig_.use_map_equation to sample delta_map_equation() for s_R. Pass comm_internal and q_total to calibrate_lambda.

    @genomewalker genomewalker committed Feb 18, 2026
    5920896
  • Add SCG-based resolution calibration to QualityLeiden calibrate_resolution() tries n_res_trials resolutions (log-uniform over [res_search_min, res_search_max]), scores each with the internal SCG metric (completeness - penalty*contamination), fits a quadratic in log-res space, and returns the argmax. No CheckM2 needed — uses the loaded MarkerIndex. Activate with --quality-leiden + calibrate_resolution=true in QualityLeidenConfig.

    @genomewalker genomewalker committed Feb 18, 2026
    90b6211
  • Add resolution exploration to EnsembleLeiden (log-uniform grid) Wire --ensemble-vary-res / --ensemble-res-min / --ensemble-res-max flags. Run i uses resolution exp(log(res_min) + i/(n-1) * log(res_max/res_min)), covering the scale space without a fixed grid.

    @genomewalker genomewalker committed Feb 18, 2026
    e909dbc
  • Fix: add use_map_equation to CLI Bin2Config struct

    @genomewalker genomewalker committed Feb 18, 2026
    2d65d4f
  • Wire --map-equation CLI flag and add config 5 to ablation script

    @genomewalker genomewalker committed Feb 18, 2026
    2650147
  • Implement marker-guided map equation delta for Phase 1b Replace modularity delta with the map equation delta (Rosvall & Bergstrom, parameter-free MDL objective) in move_nodes_fast_quality_mt. Combined with the marker quality penalty, this gives a proper marker-guided Infomap: L*(M) = h(q_total) - 2·Σh(q_c) + Σh(flow_c) + λ·Σ dup_excess(c) where h(x)=x·ln(x), q_c is the exit rate, flow_c=q_c+p_c is the module visit rate including exit. The map equation has no resolution parameter (MDL self-regulates granularity), so it avoids the over-merging caused by the modularity-based Phase 1b. Phase 1b activates via use_map_equation=true in QualityLeidenConfig.

    @genomewalker genomewalker committed Feb 18, 2026
    443c94d
  • Revert Phase 1b quality refinement: over-merges (14092-node monolith → 0 HQ) The quality-guided local moves reward contamination reduction so strongly that they collapse 322 clusters into a 14092-node super-cluster (50% of contigs). Phase 2 then shatters this into 136 near-singleton fragments, producing 0 HQ bins vs 9.3 with Phase 1b absent. Phase 2 guards (400kb min + dup_excess+2 cap) retained.

    @genomewalker genomewalker committed Feb 18, 2026
    a6e4180
  • Fix Phase 2 over-fragmentation in QualityLeiden Two guards prevent shattering valid bins: 1. Skip clusters < 400kb (too small to yield two viable 200kb bins) 2. Reject sub-clustering if num_clusters > dup_excess+2 (a cluster with 1 duplicate marker should split into ~2 parts, not 304) Previously a 1836-node cluster with dup_excess=1 was split into 304 near-singleton sub-clusters at 3x resolution, discarding a potential HQ bin. Now such over-fragmented results are rejected.

    @genomewalker genomewalker committed Feb 18, 2026
    2eb804e
  • Fix Phase 2 over-fragmentation: scale resolution by 1/sqrt(cluster_size/100) Large clusters (1836 nodes) at 3x resolution fragmented into 320 near-singleton parts — none reaching the 200kb bin threshold. Scale the multiplier down with cluster size so large clusters get conservative splits (~0.7x) while small clusters still get fine splits (~3x).

    @genomewalker genomewalker committed Feb 18, 2026
    d9cf0e7
  • Add Phase 1b: quality-guided local move refinement after initial libleidenalg After the initial libleidenalg clustering, run the combined modularity+quality objective (calibrated lambda) to refine community assignments before Phase 2 splitting. Replaces the COMEBin single-pass approach with quality-aware iterative moves.

    @genomewalker genomewalker committed Feb 18, 2026
    c244e85
  • Fix Phase 2 loop iterating past comm_q bounds n_communities was modified inside the for loop, extending iteration beyond comm_q's allocated size (358 entries). When loop index c exceeded original n_communities, comm_q[c] was out-of-bounds UB, causing memory corruption and std::bad_alloc. Fix: freeze original_n_communities before the loop so only the original clusters (not new split sub-clusters) are processed.

    @genomewalker genomewalker committed Feb 18, 2026
    c63c54e
  • Fix ablation build: use in-place cmake instead of rm -rf build rm -rf build fails on NFS when the previous binary is still mapped by running processes. CMake handles existing build directories fine and only recompiles changed files.

    @genomewalker genomewalker committed Feb 18, 2026
    e2a3df7
  • Fix Phase 2 crash: is_membership_fixed size mismatch in subgraph LibLeidenalgBackend::cluster() was copying the full-graph is_membership_fixed (28162 entries) into a cluster subgraph call (400 nodes), causing igraph to crash with 'Vertex 10410 is not in the graph'. Two fixes: 1. LeidenBackend: guard is_membership_fixed copy with use_fixed_membership AND size check (matches how node_sizes is handled) 2. Phase 2: explicitly clear initial_membership and is_membership_fixed from sub_config to prevent stale full-graph data leaking in

    @genomewalker genomewalker committed Feb 18, 2026
    b09974f
  • Append 9 multi-scale CGR features to clustering space After encoder training, concatenate scale-transition features (entropy/occupancy/lacunarity slopes at 16->32->64 CGR resolutions) to the 148-dim embeddings → 157-dim clustering space. Captures sequence complexity at multiple scales, orthogonal to the 20 aDNA damage features. Zeros for sequences < 500bp.

    @genomewalker genomewalker committed Feb 18, 2026
    2d15aab