CONSENSUS_DIAGNOSTICS_PLACEHOLDER (barplots: component alignment & subject consistency with thresholds)
+ + +--- + +### 2.5 Mathematical cores (rigorous but compact) + +SRM (orthogonal, correlation-maximising form). For subjects $X_i\in\mathbb{R}^{T_i\times d}$ find $W_i\in\mathbb{R}^{d\times k}$ with $W_i^\top W_i=I$ and shared response $S\in\mathbb{R}^{T\times k}$: + +{% raw %} +$$ +\min_{\{W_i\},\, S}\sum_i \lVert X_i W_i - S \rVert_F^2 +\;\Longleftrightarrow\; +\max_{\{W_i\},\, S}\sum_i \operatorname{tr}\!\big(S^\top X_i W_i\big), +\qquad W_i^\top W_i = I. +$$ +{% endraw %} + +Updates: $S\leftarrow \tfrac{1}{n}\sum_i X_i W_i$ (column-wise z-scored), +$W_i \leftarrow U V^\top$ where $U\Sigma V^\top = \operatorname{SVD}(X_i^\top S)$. + +MCCA (SUMCOR-style, whitened). Let $X_i$ be centred and whitened to $\tilde X_i$. Find $a_i\in\mathbb{R}^{d\times k}$ maximising total cross-correlation: + +{% raw %} +$$ +\max_{\{a_i\}}\; \sum_{iSCREE_AND_VARIANCE_PLACEHOLDER (scree, per-component & cumulative variance; consensus variance explained)
+ + +--- + +### 2.7 Outputs (what to expect) + +* Method-specific models (components/loadings), aligned across methods. +* Consensus model (columns = population-universal latents), with weights and stability report. +* Per-subject projections (time × components) ready for binarisation → ELA/PDA. +* QC bundle: metric table, thresholded alignment & consistency plots, dominance index, and retained-component justifications. + +--- + +{% details Click to expand: implementation highlights & safeguards %} + +* RSA-style structure preservation reconstructs regions from components (per-region least squares) to compare correlation matrices before/after, reporting Pearson on vectorised upper triangles, Frobenius/mean diffs, and weighted sign preservation. +* Temporal metrics (trustworthiness/continuity, neighbourhood hit) use downsampled representations for speed without losing neighbourhood signal; autocorr preservation checked at task-relevant lags. +* Robust sign handling: column-wise correlation checks after Procrustes; optional baseline-anchored flipping (percentile-based) prevents biologically implausible polarity swaps. +* Dominance index offers four normalised variants (Gini/CV/TVD/min-max) to ensure balanced consensus. +* Permutation/phase-randomisation thresholds provide statistical guardrails for declaring alignment/consistency “good enough.” + {% enddetails %} + +--- + +### References (core methods and metrics) + +* SRM: Chen, R. S., Chen, P.-H., et al. A Reduced-Dimension fMRI Shared Response Model. NeurIPS, 2015. +* MCCA: Kettenring, J. R. Canonical analysis of several sets of variables. Biometrika, 1971. +* CCA: Hotelling, H. Relations Between Two Sets of Variates. Biometrika, 1936. +* Orthogonal Procrustes: Schönemann, P. H. A generalized solution of the orthogonal Procrustes problem. Psychometrika, 1966. +* Hungarian algorithm: Kuhn, H. W. The Hungarian Method for the Assignment Problem. Naval Research Logistics Quarterly, 1955. +* PCA/Varimax: Jolliffe, I. T. Principal Component Analysis; Kaiser, H. F. The Varimax Criterion. Psychometrika, 1958. +* ICA / FastICA: Hyvärinen, A. and Oja, E. Independent Component Analysis: Algorithms and Applications. Neural Networks, 2000. +* Trustworthiness/Continuity: Venna, J. and Kaski, S. Neighborhood Preservation in Nonlinear Projection Methods. 2001. +* Kruskal stress: Kruskal, J. B. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 1964. +* RSA: Kriegeskorte, N. et al. Representational Similarity Analysis. Frontiers in Systems Neuroscience, 2008. +* RV coefficient: Robert, P. and Escoufier, Y. A Unifying Tool for Linear Multivariate Statistical Methods: The RV-Coefficient. 1976. + +--- + +
+
+::contentReference[oaicite:0]{index=0}
+
+---
+
+
+## 3) Binarisation of latent time series
+
+After alignment and dimensionality reduction, each latent x_i(t) is thresholded per-latent using median or mean and mapped to s_i(t) in the set {-1,+1}.
+
+This respects component-specific baselines, keeps PMEM tractable, and standardises inputs for inter-subject comparability.
+
+---
+
+## 4) Pairwise maximum-entropy (Ising) fitting
+
+We model binary latents with the pairwise maximum-entropy distribution
+
+$$
+P(\mathbf{s}) \propto \exp\left(\sum_i h_i s_i + \sum_{iPIPELINE_SUMMARY_PLACEHOLDER (overview from preprocessing → pick-and-mix methods → alignment → consensus → QC)
+
+
+
+
+---
+
+## Robustness, uncertainty and diagnostics
+
+- Uncertainty via variational-Bayes posteriors for h and J; bootstrap intervals for mu and sigma and for near-criticality; block bootstrap for autocorrelation
+- Convergence checks including MCMC R-hat and effective sample size where applicable, pseudo-likelihood relative-norm stopping, and ELBO improvements for variational Bayes
+- Quality-control gates including permutation or null thresholds for spurious minima, component-stability filters, and sensitivity to number of latents and alignment choice
+- Ablations: pooled versus subgroup reference surfaces; method toggles among SRM, MCCA, and ICA; median versus mean thresholds
+
+---
+
+## Interactive figures
+
+- Phase placements with confidence intervals
+
+
+
+
+- Basin visits over time (stripe plot)
+
+
+
+
+- Barrier distribution and kinetics
+
+
+
+
+---
+
+## Reproducibility and artefacts
+
+We operate config-first with deterministic seeds and machine-parsable outputs.
+
+{% highlight yaml %}
+# configs/pipeline.yaml
+seed: 123
+preprocess:
+ detrend: conservative
+ concat_runs: true
+ impute: short_gaps_only
+alignment:
+ methods: [SRM, MCCA, GroupPCA, GroupICA]
+ consensus: true
+ select_dim: auto
+binarise:
+ threshold: median
+ising:
+ mode: PL # {EXACT|PL|VB}
+ l2_h: 1e-5
+ l2_J: 1e-4
+ pl_tol: 1e-6 # relative-norm stopping
+ vb:
+ prior_prec_h: 6.0
+ prior_prec_J: 30.0
+ela:
+ minima_search: exhaustive
+ kinetics: true
+pda:
+ observables: [m, q, chiSG, chiUni, C]
+ bootstrap: true
+ ci_level: 0.95
+reports:
+ phase_report: true
+ landscape_report: true
+{% endhighlight %}
+
+Repository layout for this post:
+- post file: _posts/2025-10-25-a-shared-coordinate-system.md
+- static images: assets/img/a-shared-coordinate-system/…
+- plotly HTML: assets/plotly/…
+- BibTeX: assets/bibliography/a-shared-coordinate-system.bib
+
+---
+
+## Limitations and scope
+
+- Binarisation coarsens signals, but enables interpretable PMEM fitting and stable cross-subject comparability
+- Latent selection is influential; consensus and metric-guided selection mitigate but semantics remain partly model-dependent
+- The choice of reference surface affects PDA; we quantify sensitivity and expose cost bowls for transparency
+- Designed for resting or task neural time series; extension to other binarisable dynamical systems is often straightforward
+
+---
+
+## Outlook
+
+Population-universal latents combined with physics-grounded descriptors provide a shared language for multi-subject brain dynamics that is portable across modalities, tasks, and species, and a bridge to mechanistic interpretation and clinical translation. Planned extensions include multi-modal fusion, alignment-aware causal probes, and targeted clinical studies.
+
+---
+
+## Appendix: Mathematical details
+
+### PMEM objective via pseudo-likelihood
+
+Maximise pseudo-likelihood with L2 regularisation:
+
+$$
+\mathcal{L}_{\mathrm{PL}}(\mathbf{h},\mathbf{J})
+=
+\sum_{t,i}\log P\left(s_i^{(t)}\mid \mathbf{s}_{\setminus i}^{(t)}\right)
+-
+\frac{\lambda_h}{2}\lVert \mathbf{h}\rVert_2^2
+-
+\frac{\lambda_J}{2}\lVert \mathbf{J}\rVert_F^2.
+$$
+
+### Variational Bayes sketch
+
+Gaussian priors on parameters; update posterior precision via empirical feature covariances; track the evidence lower bound; optional Monte Carlo moment approximation for larger N.
+
+### Observables for PDA
+
+$$
+\begin{aligned}
+m &= \frac{1}{N}\sum_i \langle s_i\rangle, \\
+q &= \frac{1}{N}\sum_i \langle s_i\rangle^2, \\
+\chi_{\mathrm{Uni}} &= \frac{1}{N}\sum_{i\neq j}\mathrm{Cov}(s_i,s_j), \\
+\chi_{\mathrm{SG}} &= \frac{1}{N}\sum_k \lambda_k^2, \\
+C &= \frac{1}{N}\left(\langle E^2\rangle - \langle E\rangle^2\right).
+\end{aligned}
+$$
+
+---
+
+### Acknowledgements
+
+We thank colleagues at PSNC for discussions and infrastructure support.
+
+---
+
+### References
+
+Rendered from the bibliography file.
diff --git a/assets/bibliography/2025-10-25-phase-diagram-playbook.bib b/assets/bibliography/2025-10-25-phase-diagram-playbook.bib
new file mode 100644
index 0000000..e69de29
From bce1fedb5f2cda641fa09e18d6932315bb9d76cf Mon Sep 17 00:00:00 2001
From: juliankedys <96544513+juliankedys@users.noreply.github.com>
Date: Tue, 28 Oct 2025 05:14:34 +0000
Subject: [PATCH 3/8] Exctended the descriptions of the key methods;
appropriate visuals will be included very shortly (same day), in the upcoming
push.
---
_posts/2025-10-25-phase-diagram-playbook.md | 553 +++++++++++++++-----
1 file changed, 429 insertions(+), 124 deletions(-)
diff --git a/_posts/2025-10-25-phase-diagram-playbook.md b/_posts/2025-10-25-phase-diagram-playbook.md
index 87f679a..fc74bb2 100644
--- a/_posts/2025-10-25-phase-diagram-playbook.md
+++ b/_posts/2025-10-25-phase-diagram-playbook.md
@@ -1,63 +1,109 @@
---
layout: distill
-title: "A Shared Coordinate System for Whole-Brain Dynamics"
-tags: [computational neuroscience, shared latent representations, tutorial, subject alignment, neural state-spaces, energy landscapes, intersubject comparability, interpretable descriptors, brain dynamics, statistical physics]
+title: "Shared Coordinates for Cross-Subject Brain Dynamics: Universal Latents and Directly Comparable Phase Diagrams"
+categories: [shared latent representations, tutorial, subject alignment, neural state-spaces, energy landscapes, intersubject comparability, state transitions, interpretable descriptors, brain dynamics]
giscus_comments: true
date: 2025-10-25
featured: false
authors:
- - name: "Julian Kędys"
+ - name: Julian Kędys
url: "https://www.linkedin.com/in/julian-kedys-a332222a6/"
affiliations:
- - name: "Department of Digital Medicine, Poznań Supercomputing and Networking Center (PSNC), Polish Academy of Sciences"
- url: "https://www.psnc.pl"
- - name: "Cezary Mazurek"
+ name: Department of Digital Medicine, Poznań Supercomputing and Networking Center (PSNC), Polish Academy of Sciences
+ - name: Cezary Mazurek
url: "http://pl.linkedin.com/in/cezarymazurek"
- affiliations:
- - name: "Department of Digital Medicine, Poznań Supercomputing and Networking Center (PSNC), Polish Academy of Sciences"
- url: "https://www.psnc.pl"
----
-# Contents:
-### Introduction
-### Motivation and contribution
-### Pipeline overview
- - 1) ELA-secure preprocessing (brief)
- - 2) Population-universal shared latent space
- - 3) Binarisation of latent time series
- - 4) Pairwise maximum-entropy (Ising) fitting
- - 5) Energy-Landscape Analysis (ELA): descriptors and kinetics
- - 6) Phase-Diagram Analysis (PDA): multi-observable placement
+bibliography: 2025-10-25-phase-diagram-playbook.bib
+
+toc:
+ - name: "Overview"
+ - name: "Introduction"
+ subsections:
+ - name: "1. Big‑picture overview - why shared latents + PMEM → ELA → PDA?"
+ - name: "2. Where the methods come from (intuitive recap)"
+ - name: "3. Why shared latents are necessary (and useful)"
+ - name: "Motivation and contribution"
+ - name: "Pipeline overview"
+ subsections:
+ - name: "1) ELA-secure preprocessing (brief)"
+ - name: "2) Population-universal shared latent space"
+ - name: "3) Binarisation of latent time series"
+ - name: "4) Pairwise maximum-entropy (Ising) fitting"
+ - name: "5) Energy-Landscape Analysis (ELA): descriptors and kinetics"
+ - name: "6) Phase-Diagram Analysis (PDA): multi-observable placement"
+ - name: "Robustness, uncertainty and diagnostics"
+ - name: "Limitations and scope"
+ - name: "Outlook"
-### Robustness, uncertainty and diagnostics
-### Limitations and scope
-### Outlook
---
-## Overview
+## Overview (TL;DR)
-We present a modular, modality-agnostic workflow that turns heterogeneous whole-brain time series into cohort-comparable, interpretable coordinates on a shared phase diagram, together with energy-landscape descriptors such as attractors, barriers, and kinetics.
+We present a modular, modality-agnostic workflow that turns heterogeneous whole-brain time series into cohort-comparable, interpretable coordinates on a shared phase diagram, together with energy-landscape descriptors such as attractors, barriers, and kinetics.
Key steps:
-1) population-universal latent spaces (SRM, MCCA, Group PCA or Group ICA with consensus and automated dimensionality selection)
+1) population-universal latent spaces (Shared Response Model - SRM, Multiset Canonical Correlation Analysis - MCCA, Group PCA or Group ICA with consensus and automated dimensionality selection)
2) per-latent binarisation to the +/-1 format
-3) PMEM or Ising fitting (exact for small N, pseudo-likelihood, or variational Bayes
-4) energy-landscape analysis: minima, disconnectivity, barriers, occupancies, kinetics
-5) phase-diagram analysis: multi-observable placement on a shared reference surface with uncertainty
+3) Pairwise Maximum Entropy Model (PMEM) or Ising fitting: exact for small N, pseudo-likelihood, or variational Bayes
+4) energy landscape analysis (ELA): minima, disconnectivity, barriers, occupancies, kinetics
+5) phase diagram analysis (PDA): novel multi-observable placement on a shared reference surface with uncertainty
Outputs include uncertainty, quality control, and interactive visuals. Methods are user-tweakable, reliable, and reproducible.
+
+---
+
+## Introduction
+
+#### 1. Big‑picture overview - why shared latents + PMEM → ELA → PDA?
+Modern whole‑brain recordings are heterogeneous across subjects, sessions, tasks and modalities. If we analyse each participant in their own idiosyncratic space, descriptors of “brain state” are not directly comparable. Our pipeline solves this in two moves:
+1. Population‑universal shared latents. We align subjects into a common, low‑dimensional space (SRM / MCCA / Group PCA‑ICA with consensus). Variables have stable meaning across participants and runs, so everything downstream is comparable and reproducible.
+2. Physics‑grounded descriptors. On the binarised latents we fit a pairwise maximum‑entropy model (PMEM/Ising), then read out two complementary summaries:
+ * Energy‑Landscape Analysis (ELA) - an attractor‑and‑barrier view of the fitted Ising energy. It yields minima/basins, disconnectivity graphs, barrier spectra, and kinetic descriptors (basin dwell times, mean first-passage times (MFPTs), committors, relaxation times). This is the mechanistic, state‑space view.
+ * Phase‑Diagram Analysis (PDA) - a macroscopic view that places each subject on the **($$\mu$$, $$\sigma$$)** plane of a disordered (via parametric perturbations) Ising model (SK‑like). In broad outline, it uses multiple observables at once to locate individuals relative to critical boundaries, providing cohort‑comparable coordinates and uncertainty.
+
+#### 2. Where the methods come from (intuitive recap)
+* **PMEM/Ising:** Among all binary models that match the empirical means and pairwise correlations, PMEM has maximum entropy. It is equivalent to the zero‑temperature Ising family used throughout statistical physics. Minimal assumptions; parameters are interpretable as fields $$h_i$$ and couplings $$J_{ij}$$.
+* **ELA:** Treat the fitted Ising as an energy landscape:
+ $$
+ E(\mathbf{s}) = -\sum_i h_i s_i - \tfrac{1}{2}\sum_{i\neq j} J_{ij} s_i s_j
+ $$,
+
+ over binary states $$\mathbf{s}\in\{-1,+1\}^N$$.
+
+ Local minima = attractors; energy differences = barriers; transition graphs + Markov kinetics = how the brain moves between them.
+* **PDA:** In spin‑glass models, the distribution of couplings matters. If the off‑diagonal $$J_{ij}$$ have mean $$\mu$$ and standard deviation $$\sigma$$ with $$h_i\approx 0$$, the system sits in regions (paramagnetic / spin‑glass / ferromagnetic) that govern ordering, glassiness and susceptibility. PDA maps each subject onto this phase surface so cohorts can be compared at a glance.
+
+#### 3. Why shared latents are necessary (and useful)
+* **Stable semantics:** Each latent represents the same population‑level pattern across participants, which makes ELA basins and PDA coordinates directly comparable.
+* **Tractability:** PMEM scales as $$\mathcal{O}(N^2)$$ parameters; a well‑chosen latent space puts us in the sweet spot between information retention and robust estimation.
+* **Downstream identifiability:** Binarisation → PMEM → ELA/PDA relies on on/off switching. Our alignment preserves this structure and gives us comparable switching rasters across the cohort.
+* **Utility‑forward:** with one aligned space we can publish shared phase diagrams and landscape reports that are re‑usable across datasets and modalities, enabling baseline‑independent comparisons and cross‑study synthesis.
+
+**Select practical advantages of our framework:**
+
+The workflow is largely standalone and implemented locally, from scratch - allowing researchers/analysts to adapt its workings to their exact needs - with numerical-stability and computational-efficiency improvements (relative to analogous implementations in the domain), novel algorithmic extensions (i.a. for multi-subject dimnensionality reduction, comparative phase diagram analysis of heterogenous brain dynamics), informative metrics and rich visualisations, and emphasis on **a)** automated parameter optimisation - not requiring domain expertise or significant prior experience with the pipeline from the user, **b)** data-driven model selection, **c)** data-modality universality and independece of heuristics/meta-knowledge throughout the entire design process, and **d)** availability of alternative methods and hyperparameters for key processing/modelling stages, so as to best fit the needs of the user
+
+**Limitations worth remembering:**
+- Binarisation coarsens signals (whereas more granular discretisation becomes computationally prohibitve almost instantly for real-life data/problems)
+- Results depend on the selection of binarisation thresholds, dimensionality-reduction models and target counts of obtained latent features
+- For exact modelling methods, the set of possible states doubles in size with every additional feature/node/brain region
+- PDA assumes $$h\approx 0$$ and an SK‑like (Sherrington-Kirkpatrick) parametrisation
+- ELA explores single‑spin‑flip pathways, which is a limited and simplified assumption
+
+To counteract the influence of initial choice of (hyper)parameters, we quantify uncertainty in the workflow, track convergence wherever applicable, offer truly data-driven and bias-free optimisation of pipeline parameters, and expose diagnostics so these choices remain transparent and testable.
+
---
## Motivation and contribution
-The UniReps community prioritises representational alignment and comparability across subjects, datasets, and models. This post demonstrates:
+Reflecting the UniReps community's interest in representational alignment and comparability across subjects, datasets, and models, this post demonstrates:
-- a subject-alignment front-end that produces shared latent representations with stable semantics across a population
-- a stitched, physics-grounded back-end (PMEM to ELA to PDA) that yields mechanistically interpretable descriptors and shared coordinates
-- a robustness-first toolkit that includes consensus alignment, automated selection, uncertainty quantification, diagnostics, and review-ready artefacts
+- a subject-alignment front-end that produces shared latent representations with stable semantics across a population, offering several alternative approaches
+- a stitched, physics-grounded back-end (PMEM to ELA to PDA) that yields mechanistically interpretable descriptors and shared phase-diagram coordinates derived with our original methodology
+- a robustness-first toolkit that includes custom-built consensus alignment, automated parameter selection, uncertainty quantification, diagnostics, and review-ready artefacts
---
@@ -69,10 +115,10 @@ The UniReps community prioritises representational alignment and comparability a
1. Preprocess and engineer: ELA-secure detrending, conservative handling of missing data, safe within-subject run concatenation, per-region standardisation.
2. Population-aware alignment and dimensionality reduction: shared latents via SRM, MCCA, or Group PCA or Group ICA with consensus; automatic dimensionality selection.
-3. Binarisation: per-latent threshold (median or mean), yielding +/-1 time series.
-4. PMEM or Ising fitting: exact (small N), safeguarded pseudo-likelihood, or variational Bayes.
-5. Energy-landscape analysis: attractors, barriers, disconnectivity graph, occupancies, kinetics.
-6. Phase-diagram analysis: multi-observable placement on a shared reference surface with confidence intervals and near-criticality indices.
+3. Binarisation: per-latent threshold (usually median or mean, unless domain-expertise justifies, e.g., percentile-based thresholding), yielding +/-1 time series.
+4. PMEM or Ising fitting: exact (small N), safeguarded pseudo-likelihood, or variational Bayes - each enriched with its adequate set of solution-stability and significance/quality assessments.
+5. Energy-landscape analysis: attractors, barriers, disconnectivity graph, occupancies, kinetics, and many more descriptors providing mechanistic, biologically meaningful insight into brain dynamics, as well as facilitating direct and intuitive comparisons between subjects/cohorts.
+6. Phase-diagram analysis: multi-observable placement on a shared reference surface with our custom cost function, reports on confidence intervals, and near-criticality indices.
---
@@ -121,7 +167,7 @@ For ELA, methods must enhance quality without compromising temporal structure. S
* Flat-blocks breaking: apply if replacements create long identical runs.
* Smoothing: only very mild, primarily cosmetic; avoid aggressive windows that could distort binarisation.
-Bandpass filtering can be applied if not already applied to the data - many recording devices perform it automatically. Temporal standardisation and spatial normalisation are not required for ELA itself but are retained for general use; spatial normalisation is applied for the imputation pipeline to align signs and amplitudes. LOESS may shift some series below zero; this is expected and accounted for. All parameterisations are chosen to remain strictly ELA-compatible.
+Bandpass filtering can be performed if not already applied to the data - many recording devices perform it automatically. Temporal standardisation and spatial normalisation are not required for ELA itself but are retained for general use; spatial normalisation is applied for the imputation pipeline to align signs and amplitudes. LOESS may shift some series below zero; this is expected and accounted for. All parameterisations are chosen to remain strictly ELA-compatible.
{% enddetails %}
@@ -131,7 +177,7 @@ Bandpass filtering can be applied if not already applied to the data - many reco
Goal: obtain shared, ELA-secure latent components whose semantics are stable across subjects and runs, so downstream binarisation → PMEM → ELA/PDA is directly comparable and computationally tractable.
-#### What’s new/strong here (one glance):
+#### Novelty factor at a glance:
* Pick-and-mix reductions: (Group PCA, Group ICA, SRM, MCCA) with robust tweaks (varimax, multi-restart ICA, PCA-initialised SRM, whitened MCCA).
* A population-aware, multi-metric objective (structural + temporal + method-specific) → auto-selection of dimensionality and hyperparameters.
@@ -152,11 +198,11 @@ All methods preserve temporal ordering (only linear projections or orthogonal tr
* Group PCA (variance capture + varimax): eigenvectors of the average subject covariance; optional varimax keeps components sparse/interpretable without breaking orthogonality.
*Extras:* elbow detection, subject-wise EVR, reconstruction error, and post-rotation low-variance pruning.
-* Group ICA (shared independent subspaces): subject-wise PCA → concatenation → FastICA with multi-restart selection (best negentropy proxy via kurtosis deviation), independence verification (mean abs(corr), kNN mutual information, squared-corr checks), sign harmonisation of subject contributions before averaging.
+* Group ICA (shared independent subspaces): subject-wise PCA → concatenation → FastICA with multi-restart selection (best negentropy proxy via kurtosis deviation), independence verification (mean $$\lvert\mathrm{corr}\rvert$$, kNN mutual information, squared-corr checks), sign harmonisation of subject contributions before averaging.
-* SRM (shared timecourses with subject-specific maps): iterative orthogonal subject mappings (W_i) and shared response (S), PCA-based initialisation, Hungarian-matched alignment of mappings across subjects, orthogonality diagnostics, shared variance quantification.
+* SRM (shared timecourses with subject-specific maps): iterative orthogonal subject mappings $$(W_i)$$ and shared response $$(S)$$, PCA-based initialisation, Hungarian-matched alignment of mappings across subjects, orthogonality diagnostics, shared variance quantification.
-* MCCA (maximising cross-subject correlation): per-subject whitening → SVD on concatenated features (SUMCOR-style) → iterative refinement of subject loadings (a_i). We report cross-subject alignment, canonical correlations to shared response, orthogonality in whitened space, and shared variance in native space.
+* MCCA (maximising cross-subject correlation): per-subject whitening → SVD on concatenated features (SUMCOR-style) → iterative refinement of subject loadings $$(a_i)$$. We report cross-subject alignment, canonical correlations to shared response, orthogonality in whitened space, and shared variance in native space.
> Why four methods? They trade off interpretability, independence, correlation sharing, and variance capture. We score them with the same unified, ELA-aware metric suite and fuse them, yielding a stable, population-universal latent space.
@@ -177,37 +223,53 @@ All methods preserve temporal ordering (only linear projections or orthogonal tr
### 2.3 Metrics & selection (multi-objective, normalised to [0,1])
-Structural fidelity (RSA-style): correlation between original region-by-region correlation structure and that reconstructed from latents; sign preservation weighted by abs(corr); Procrustes disparity; Kruskal stress on inter-region distances.
+Structural fidelity (RSA-style): correlation between original region-by-region correlation structure and that reconstructed from latents; sign preservation weighted by $$\lvert\mathrm{corr}\rvert$$; Procrustes disparity; Kruskal stress on inter-region distances.
Temporal faithfulness: trustworthiness/continuity of neighbour relations, temporal neighbourhood hits, autocorrelation preservation at task-relevant lags.
Method-specific:
* PCA: mean subject EVR, reconstruction error.
-* ICA: independence (mean abs(corr)↓, MI↓), kurtosis (≠3), sparsity.
+* ICA: independence (mean $$\lvert\mathrm{corr}\rvert$$↓, Mutual Information (MI)↓), kurtosis (≠3), sparsity.
* SRM: shared alignment (Hungarian-matched), orthogonality, shared variance.
* MCCA: cross-subject alignment, shared variance, canonical correlations, orthogonality (whitened).
Normalisation uses principled maps (e.g., $$x \mapsto \tfrac{1}{1+x}$$ for “smaller-is-better”, linear $$[-1,1]\to[0,1]$$ for correlations).
-Composite score: weighted average (user- or default weights), then auto-optimise (k) and hyperparameters via seeded, parallel grid search.
+Composite score: weighted average (user- or default weights), then auto-optimise $$(k)$$ and hyperparameters via seeded, parallel grid search.
{% details Click to expand: metric formulas %}
Kruskal stress (upper-triangle, variance-matched):
+
+{% raw %}
$$
-\operatorname{Stress}=\sqrt{\frac{\sum_{i
-
-
1. Preprocess and engineer: ELA-secure detrending, conservative handling of missing data, safe within-subject run concatenation, per-region standardisation.
2. Population-aware alignment and dimensionality reduction: shared latents via SRM, MCCA, or Group PCA or Group ICA with consensus; automatic dimensionality selection.
3. Binarisation: per-latent threshold (usually median or mean, unless domain-expertise justifies, e.g., percentile-based thresholding), yielding +/-1 time series.
@@ -120,34 +118,77 @@ Reflecting the UniReps community's interest in representational alignment and co
5. Energy-landscape analysis: attractors, barriers, disconnectivity graph, occupancies, kinetics, and many more descriptors providing mechanistic, biologically meaningful insight into brain dynamics, as well as facilitating direct and intuitive comparisons between subjects/cohorts.
6. Phase-diagram analysis: multi-observable placement on a shared reference surface with our custom cost function, reports on confidence intervals, and near-criticality indices.
+**Data summary** (used for the development and testing of the pipeline; details not central to current discussion in themselves):
+
+Resting-state fUS from N=8 mice (7 Cre-lox ASD models spanning 4 subtypes; 1 control with no symptomatic manifestation modelled); 54 bilateral regions (27 L/R pairs; whole-brain collection) - unified across all the subjects; two runs per mouse (1494 frames for each recording session; TR ≈ 0.6 s); runs concatenated per subject.
+
---
## 1) ELA-secure preprocessing (brief overview)
Aim: remove spikes, outliers, and slow global drifts while **preserving the on/off switching structure** that drives binarisation, PMEM fitting, and ELA/PDA. The procedure is modality-agnostic, non-invasive, and parameterised to be reproducible.
-* Adaptive per-region parameters are computed from simple statistics and Welch spectra, then re-adapted after each step if requested (robust mode).
+* Adaptive per-region parameters are computed from simple statistics and Welch spectra, then re-adapted after each step if requested (robust mode). PIPELINE_OVERVIEW_PLACEHOLDER
-
+ 
+ 
+ 
+ 
+
-
---
-
### 2.1 Methods (population-aware, ELA-secure)
-All methods preserve temporal ordering (only linear projections or orthogonal transforms over channels), and we quantify temporal faithfulness (trustworthiness/continuity, autocorrelation preservation).
+All methods preserve temporal ordering (only linear projections or orthogonal transforms over channels), and we quantify temporal faithfulness (trustworthiness/continuity, autocorrelation preservation) SHARED_COMPONENTS_PLACEHOLDER (e.g., heatmaps of method-specific and consensus components)
-
+
+ 
+ 
+
+
+
+
+ 
+ 
+
+
+
+
-
-
---
### 2.3 Metrics & selection (multi-objective, normalised to [0,1])
-Structural fidelity (RSA-style): correlation between original region-by-region correlation structure and that reconstructed from latents; sign preservation weighted by $$\lvert\mathrm{corr}\rvert$$; Procrustes disparity; Kruskal stress on inter-region distances.
+Structural fidelity (RSA-style): correlation between original region-by-region correlation structure and that reconstructed from latents; sign preservation weighted by $$\lvert\mathrm{corr}\rvert$$; Procrustes disparity; Kruskal stress ALIGNMENT_FLOW_PLACEHOLDER (schematic of Procrustes → sign fix → Hungarian → consensus)
-
-
+
+CONSENSUS_DIAGNOSTICS_PLACEHOLDER (barplots: component alignment & subject consistency with thresholds)
-
+ 
+ 
+ 
+ 
+ 
+ 
+
-
-
---
### 2.7 Outputs (what to expect)
@@ -441,28 +591,6 @@ Subject PCA → concatenation → FastICA. Restart with multiple seeds; select t
* Permutation/phase-randomisation thresholds provide statistical guardrails for declaring alignment/consistency “good enough.”
{% enddetails %}
----
-
-### References (core methods and metrics)
-
-* SRM: Chen, R. S., Chen, P.-H., et al. A Reduced-Dimension fMRI Shared Response Model. NeurIPS, 2015.
-* MCCA: Kettenring, J. R. Canonical analysis of several sets of variables. Biometrika, 1971.
-* CCA: Hotelling, H. Relations Between Two Sets of Variates. Biometrika, 1936.
-* Orthogonal Procrustes: Schönemann, P. H. A generalized solution of the orthogonal Procrustes problem. Psychometrika, 1966.
-* Hungarian algorithm: Kuhn, H. W. The Hungarian Method for the Assignment Problem. Naval Research Logistics Quarterly, 1955.
-* PCA/Varimax: Jolliffe, I. T. Principal Component Analysis; Kaiser, H. F. The Varimax Criterion. Psychometrika, 1958.
-* ICA / FastICA: Hyvärinen, A. and Oja, E. Independent Component Analysis: Algorithms and Applications. Neural Networks, 2000.
-* Trustworthiness/Continuity: Venna, J. and Kaski, S. Neighborhood Preservation in Nonlinear Projection Methods. 2001.
-* Kruskal stress: Kruskal, J. B. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 1964.
-* RSA: Kriegeskorte, N. et al. Representational Similarity Analysis. Frontiers in Systems Neuroscience, 2008.
-* RV coefficient: Robert, P. and Escoufier, Y. A Unifying Tool for Linear Multivariate Statistical Methods: The RV-Coefficient. 1976.
-
----
-
-SCREE_AND_VARIANCE_PLACEHOLDER (scree, per-component & cumulative variance; consensus variance explained)
-
-
-::contentReference[oaicite:0]{index=0}
---
@@ -473,6 +601,22 @@ After alignment and dimensionality reduction, each latent $$x_i(t)$$ is threshol
This respects component-specific baselines, keeps PMEM tractable, and standardises inputs for inter-subject comparability.
+
+PIPELINE_SUMMARY_PLACEHOLDER (overview from preprocessing → pick-and-mix methods → alignment → consensus → QC)
-
+ 
+ 
+
-
-
+ 
(A) Local-minimum patterns (binary states for each attractor); (B) 3-D energy surface with labelled minima (white dots) and most-probable transition paths (white arrows); (C) Direct transition counts between minima (Metropolis single-flip kernel); (D) Disconnectivity graph showing barrier heights that separate basins; (E) Basin visit frequencies (empirical occupancy); (F) Basin sizes (number of micro-states per basin in state-space); (G) Direct/indirect transition counts summarising multi-step reachability. Deeper basins and higher barriers indicate more stable, harder-to-leave states; denser transition lanes point to preferred switching routes.
-
-
+**Working vs display grids:** We build a high‑resolution *working* grid used for optimisation/placement and optionally a wider *display* grid for visuals. We automatically pick $$(\mu,\sigma)$$ limits by running quick PL fits per subject to estimate native $$(\hat\mu,\hat\sigma)$$, then expand by a safety factor; we refuse overly coarse grids (e.g., if $$\Delta\mu$$ or $$\Delta\sigma>0.01$$).
+
+**MC convergence safeguards:** We run multiple chains with increasing sweep budgets until *all* observables reach $$\hat R<1.05$$ and an effective sample size threshold, or a maximum sweep cap is hit (in which case a warning is issued).
---
-## Robustness, uncertainty and diagnostics
+### 6.3 Multi‑observable placement (cost projection with balanced weights)
-- Uncertainty via variational-Bayes posteriors for h and J; bootstrap intervals for mu and sigma and for near-criticality; block bootstrap for autocorrelation
-- Convergence checks including MCMC R-hat and effective sample size where applicable, pseudo-likelihood relative-norm stopping, and ELBO improvements for variational Bayes
-- Quality-control gates including permutation or null thresholds for spurious minima, component-stability filters, and sensitivity to number of latents and alignment choice
-- Ablations: pooled versus subgroup reference surfaces; method toggles among SRM, MCCA, and ICA; median versus mean thresholds
+Given a subject’s *observed* $$(m_o,q_o,\chi^{o}_{\mathrm{SG}},\chi^{o}_{\mathrm{Uni}})$$ from their binarised latents, we **project** onto the reference surface by minimising a *variance‑balanced* squared error:
+
+$$
+\underset{\mu,\sigma}{\arg\min}\;\sum_{k\in\{\mathrm{m},\mathrm{q},\chi_{\mathrm{SG}},\chi_{\mathrm{Uni}}\}}
+w_k\big(O_k^{\text{obs}}-\widehat{O}_k(\mu,\sigma)\big)^2,
+$$
+
+where $$\widehat{O}_k(\mu,\sigma)$$ is obtained by regular‑grid interpolation of the precomputed surfaces and weights $$w_k$$ default to the inverse *range* of each surface (less sensitive than $$1/\mathrm{var}$$). Optimisation uses L‑BFGS‑B on the working grid bounds; we also provide an **iso‑curve** fallback (intersect level sets of $$\chi_{\mathrm{SG}}$$ and $$\chi_{\mathrm{Uni}}$$ with a simple border‑safety check) and a brute‑force grid fallback.
+
+We return $$(\hat\mu,\hat\sigma)$$, the final cost value, and the method used (“cost_minimisation”, “iso_curves”, or “fallback_grid”).
---
-## Interactive figures
+### 6.4 Uncertainty, robustness, and diagnostics
-- Phase placements with confidence intervals
-
-
-
+**Bootstrap CIs:** For each subject we run a *circular block bootstrap* along time, refit the Ising per resample, and recompute $$(\mu,\sigma)$$ or $$(m,q,\chi_{\mathrm{SG}},\chi_{\mathrm{Uni}})$$ as needed to report 95% intervals. Specific heat $$C$$ can be bootstrapped likewise via short MC runs per resample.
-- Basin visits over time (stripe plot)
-
-
-
+**Control shrinkage (optional):** When a control is available, we stabilise its estimate by bootstrapping its $$J$$, pooling across the cohort, and forming a convex combination $$J^{\text{ctrl,shrunk}}=(1-\lambda)J^{\text{ctrl}}+\lambda J^{\text{pooled}}$$; we then summarise $$(\mu_c,\sigma_c)$$ from the shrunk $$J$$.
-- Barrier distribution and kinetics
-
-
-
+**Cost bowls:** Around each optimum we display the *local cost landscape* (2‑D filled contour and 3‑D surface) to judge identifiability; narrow, well‑curved bowls suggest precise placement.
+
+**Group‑level tests:** Small helpers allow groupwise comparisons in $$\sigma$$ (e.g., Welch ANOVA and permutation tests) using the bootstrapped distributions.
+
+**Critical structure:** We plot *critical contours* (e.g., a fixed fraction of the maximum of an observable) on the display surfaces; a simple near‑criticality index is the minimal Euclidean distance from $$(\hat\mu,\hat\sigma)$$ to the chosen contour.
+
+---
+
+### 6.5 Practical recipe (what the code actually does)
+
+1) **Prepare data**: shared latents → binarise per latent to $$\pm1$$.
+2) **Rough bounds**: quick PL fits to get subject‑wise $$(\hat\mu,\hat\sigma)$$; expand to working ranges; verify resolution is fine enough.
+3) **Build reference surface**: choose `reference_mode ∈ {pooled, control}`, set $$h\equiv 0$$, sweep $$(\mu,\sigma)$$ by affine off‑diagonal transforms of $$J_{\mathrm{ref}}$$, run multi‑chain MC with convergence checks, and cache $$\{\mathrm{m},\mathrm{q},\chi_{\mathrm{SG}},\chi_{\mathrm{Uni}},C\}$$ surfaces.
+4) **Place subjects**: minimise the balanced multi‑observable cost (or use the iso‑curve / grid fallbacks with border guard).
+5) **Uncertainty**: bootstrap along time to report CIs; optionally shrink the control to summarise $$(\mu_c,\sigma_c)$$.
+6) **Visuals**: 2‑D contour panels and interactive 3‑D surfaces, with group‑colours and critical lines; “cost bowls” per subject for identifiability.
+
+---
+
+### 6.6 Mathematical and implementation notes (exact to this workflow)
+
+- **Affine mapping of $$J$$ to $$(\mu,\sigma)$$** modifies *only* off‑diagonals and preserves $$J_{ii}=0$$. This avoids artefactual self‑coupling and keeps the spin‑glass structure intact. Fields $$h$$ are explicitly zeroed for the surface.
+- **Observables from data vs model.** Fast “data‑side” $$m,q,\chi$$ provide immediate checks, while MC‑side estimates (including $$C$$) are used to build the surface; both routes are available.
+- **Balanced weights** $$w_k$$ default to inverse *range* (normalises sensitivity without over‑penalising heavy‑tailed metrics). Equal weights are also supported.
+- **Convergence gating** uses multi‑chain Gelman-Rubin $$\hat R$$ and a conservative ESS check with geometric back‑off of sweeps up to a cap; we report if the cap is reached.
+- **Border guard** prevents pathological “corner locking” when intersecting $$\chi$$‑iso‑curves on coarse grids.
+
+---
+
+### 6.7 Interpretation tips
+
+- **$$\mu$$** (mean coupling) captures net ordering tendency; **$$\sigma$$** (coupling dispersion) captures disorder/glassiness. Movement along $$\sigma$$ at constant $$\mu$$ corresponds to increasing heterogeneity at fixed mean interaction strength.
+- **High $$\chi_{\mathrm{SG}}$$** with muted $$m$$ signals a spin‑glass‑like regime (multiple competing basins), while **high $$m$$ with low $$\chi_{\mathrm{SG}}$$** indicates ferromagnetic‑like ordering.
+- **Specific heat $$C$$** often peaks near phase boundaries and can indicate broad susceptibility to perturbations in this coarse‑grained description.
+
+---
+
+### 6.8 Reproducibility knobs (defaults)
+
+- MC: chains = 12, start sweeps = 8k, burn‑in = 1k, cap = 128k, $$\hat R$$ tol = 1.05.
+- Working grid: $$140\times140$$ by default; guard: $$\max(\Delta\mu,\Delta\sigma)\le 0.01$$.
+- Objective: weights = “balanced” (inverse range), optimiser = L‑BFGS‑B within bounds.
+- Bootstrap: circular blocks, user‑set size; control shrinkage parameter $$\lambda\in[0,1]$$.
+
+---
+
+### 6.9 Illustrative pseudocode (orientation only — code is not released here)
+
+> The snippet below mirrors the sequence used to generate the figures; it is descriptive rather than an API.
+
+```python
+# 1) Build the reference phase surfaces (pooled or control)
+phase = build_phase_surfaces(
+ binaries_pm1 = BINARIES_PM1_OR_01, # dict: subject → DataFrame (±1)
+ reference = {"mode": "pooled", "subject": None}, # or {"mode": "control","subject": "ID"}
+ grid_size = 150, # working grid resolution
+ mc_settings = {"chains": 12, "burn_in": 1000, "start_sweeps": 8000}
+)
+
+# 2) Place subjects via multi-observable cost (balanced weights by default)
+positions = place_subjects_on_surface(
+ binaries_pm1 = BINARIES_PM1_OR_01,
+ phase = phase,
+ method = "cost", # or "iso_curves"
+ weights_mode = "balanced"
+)
+
+# 3) Inspect local identifiability (“cost bowl”) for one subject
+w = objective_weights(phase, mode="balanced")
+plot_cost_landscape(
+ subject_id = "SUBJECT_ID",
+ positions = positions,
+ phase = phase,
+ window = 0.06,
+ steps = 200,
+ weights = w
+)
+
+```
+
+---
+
+### 6.10 Caveats specific to PDA
+
+- PDA assumes an SK‑like parameterisation (coupling distribution matters); it analyses **macrostates** and does not replace ELA’s mechanistic, basin‑level descriptors.
+- The choice of reference surface (pooled vs control) can shift placements; we therefore expose the cost bowls and allow both options to be reported.
+- Grid resolution and MC budgets matter near sharp boundaries; guards and diagnostics make this explicit.
+
+
+
+
+
+---
+
+## Results in a word
+On resting-state functional ultrasound (fUS) recordings (mesoscopic, whole-brain), we observe low placement residuals $$(10^{-6}–10^{-4})$$, tight bootstrap confidence regions, convergent models, and stable ordering under pooled vs subgroup phase references; example estimates span **σ ≈ 0.15–0.32** and **μ ≈ −0.01 to +0.03**. The outputs - susceptibility to perturbations, ordering vs glassiness, transition propensity - form compact, biologically meaningful fingerprints.
+
+For experimentalists, this is a mechanistic dashboard (what
+states exist, how deep, how likely to switch); for theorists, it anchors subjects in a physics-grounded phase
+space with interpretable axes.
+
+Overall, the multi-observable placement and the combination of shared embeddings with ELA/PDA provide reliability-minded, comparable, and interpretable read-outs that support discovery, phenotyping, and model-based hypothesis generation across cohorts, data modalities, tasks, and species.
---
+## Robustness, uncertainty and diagnostics
+
+- Uncertainty via variational-Bayes posteriors for h and J; bootstrap intervals for mu and sigma and for near-criticality; block bootstrap for autocorrelation