Skip to content

Phase D: Multi-model support for AMICAMLXNG - #82

Merged
neuromechanist merged 2 commits into
feature/issue-74-epic-apple-gpufrom
feature/issue-81-phaseD-mlx-multimodel
Jul 8, 2026
Merged

Phase D: Multi-model support for AMICAMLXNG#82
neuromechanist merged 2 commits into
feature/issue-74-epic-apple-gpufrom
feature/issue-81-phaseD-mlx-multimodel

Conversation

@neuromechanist

Copy link
Copy Markdown
Member

Summary

Epic #74 Phase D (final phase): extend the MLX Apple-GPU backend (AMICAMLXNG) to n_models > 1. The Phase B benchmark showed multi-model had no GPU path (MLX was single-model, MPS loses); this closes that gap so the ~7x single-model MLX win extends to multi-model AMICA.

Ports the multi-model machinery from AMICATorchNG:

  • comp_list indirection (params go (n_mix, n_channels) -> (n_mix, n_comps)), per-model W = inv(A[:, comp_list[:,h]]) + slogdet on the CPU stream (hoisted per iteration).
  • E-step per-model loop -> logV (batch, n_models), cross-model responsibilities v = softmax(logV), u = v_h * z.
  • M-step: gm, the per-model exact-EM bias c update, and the gm-weighted A-update scattered through comp_list.

Single-model (#76) stays byte-for-byte unchanged (the loop runs once, gm=1, identity comp_list -> the A-update collapses to the old A - lrate*(dA.T@A)). Component sharing remains a fast-follow.

Validation (real sample EEG; Apple GPU)

  • Multi-model (n_models=2) matches PyTorch float32: one-iteration sufficient stats agree to float32 precision (dWtmp 2.7e-3, dmu_d 6.2e-4, rest <=1e-4); converged LL gap ~1e-5 (MLX -3.38967 vs torch -3.38966). New test_multimodel_matches_torch_float32.
  • Single-model byte-identity + torch-float32 tests still pass (5/5 mlx tests green).

Benchmark

MLX now runs in the multi-model configs (dropped the single-model gate). MLX wins multi-model too: ~38 ms/it (32ch) / 45 ms/it (70ch) -- ~5x over torch-CPU, MPS still loses. Findings updated (.context/issue-77/benchmark_findings.md).

Scope note

The comprehensive native-Fortran + CPU-core-scaling (4/6/8/12) + CUDA cross-platform benchmark is intentionally NOT in this PR -- it is being split into its own epic (native x86 Linux + CUDA host, since Apple's Fortran binary is x86-under-Rosetta and Mac CPU timing is unreliable).

Test plan

uv run pytest pyAMICA/tests/mlx_tests/ (Apple Silicon; self-skips in CI). Full torch suite unaffected (mlx_impl is isolated + optional). ruff clean; mlx_impl omitted from the coverage gate (Apple-only).

Closes #81
Part of epic #74

Epic #74 Phase D: extend the MLX Apple-GPU backend to n_models > 1, porting the
multi-model machinery from AMICATorchNG -- comp_list indirection, per-model
W=inv(A[:,comp_list]) + slogdet (CPU stream), cross-model responsibilities
v=softmax(logV), the per-model exact-EM bias c update, and the gm-weighted
A-update scattered through comp_list. Single-model (#76) stays byte-for-byte
unchanged (the loop runs once, gm=1, identity comp_list). Component sharing
remains a fast-follow.

Validated on real sample EEG: multi-model (n_models=2) one-iteration sufficient
stats match AMICATorchNG float32 to float32 precision, and the converged LL
matches to ~1e-5. New tests/mlx_tests test_multimodel_matches_torch_float32; the
single-model byte-identity and torch-float32 tests still pass.

Benchmark (benchmark_dimsweep.py) now runs MLX in the multi-model configs: MLX
wins multi-model too -- ~38-45 ms/it, ~5x over torch-CPU (MPS still loses),
matching the ~7x single-model win. Findings updated (.context/issue-77).
Silent-failure review:
- Port the dead-model warning to the multi-model c update (a zero-responsibility
  model kept its prior c but was surfaced-free; matches AMICATorchNG).
- Add gm and c to the per-iteration mx.eval and the params_finite guard: they are
  new multi-model state that feeds the next E-step, so a last-iteration blow-up
  must be caught (nan_params) and c's cross-iteration dependency must be
  materialized each iteration rather than growing the lazy graph unbounded.

Comment/doc review:
- Fix two wrong AMICATorchNG citations (c-update 1083-1092, A-update 1231-1247)
  and the stale _lgamma_table shape comment ((n_mix, n_comps)).
- Drop the leftover "MVP"/"v1" framing that contradicted the new multi-model
  scope (module/class docstrings, transform(), the module title now #76/#81).

Test review:
- Expand the multi-model one-iteration stat comparison to every accumulator,
  including dWtmp (the gm-weighted A-update input, transposed to align the
  (n_models,n,n) vs (n,n,n_models) layouts) and the scattered mixture stats.
- Add a direct multi-model c-update check (responsibility-weighted data mean, and
  the two models' c must differ) and a single-model c==0 regression guard.

mlx suite green (5 passed); the port is unchanged (code reviewer verified
single-model byte-identity independently).
@neuromechanist

Copy link
Copy Markdown
Member Author

Review summary (4 Sonnet reviewers, worktree-isolated)

The code reviewer independently verified byte-identity (single-model ll_history exactly identical pre/post-PR) and multi-model correctness -- no blocking bugs; the port is faithful. All findings were quality/coverage/doc improvements, addressed in e8e5d53:

Silent-failure

  • Ported the dead-model warning to the multi-model c update (a collapsed zero-responsibility model kept its prior c but was surfaced-free; torch logs it).
  • Added gm/c to the per-iteration mx.eval + params_finite guard -- they are new multi-model state feeding the next E-step, so a last-iteration blow-up must be caught (nan_params), and c's cross-iteration dependency must materialize each iteration rather than growing the lazy graph unbounded (a real, if narrow, bug).

Tests

  • Expanded the multi-model one-iteration stat comparison to every accumulator, including dWtmp (the gm-weighted A-update input, transposed to align (n_models,n,n) vs (n,n,n_models)) and the scattered mixture stats.
  • Added a direct c-update value check (responsibility-weighted data mean; the two models' c must differ) and a single-model c==0 regression guard.

Comments/docs

  • Fixed two wrong AMICATorchNG citations (c-update 1083-1092, A-update 1231-1247), the stale _lgamma_table shape comment, and dropped the leftover "MVP/v1" framing that contradicted the new multi-model scope.

Not changed (with rationale)

  • Dead-model 0/0 in alpha/A-direction -> NaN (silent-failure Finding 3): pre-existing, bit-for-bit matches AMICATorchNG, and correctly caught by params_finite (nan_params stop + warning), not silent. A "freeze the dead model gracefully like c" refactor is a cross-backend follow-up, not this PR.
  • 3-model / dead-model-branch unit tests (test-analyzer, low priority): the loops are generic over range(n_models); n_models=2 exercises the comp_list arithmetic. Noted as nice-to-haves.

mlx suite green (5 passed).

@neuromechanist
neuromechanist merged commit e4433aa into feature/issue-74-epic-apple-gpu Jul 8, 2026
7 checks passed
@neuromechanist
neuromechanist deleted the feature/issue-81-phaseD-mlx-multimodel branch July 8, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant