Cross‐Model Empirical Analysis of Alignment Correction Field Geometry
Eric J. Ostrander Independent Research Analyst, New York
Analysis date: May 11, 2026
This report presents a comprehensive empirical analysis of alignment correction field measurements across three instruction-tuned language models from two architectural families:
| Model | Family | Parameters | Hidden dim | Layers | Delta scale |
|---|---|---|---|---|---|
| Qwen2.5-0.5B-Instruct | Qwen2 | 0.49B | 896 | 24 | 0.84 |
| Qwen2.5-1.5B-Instruct | Qwen2 | 1.5B | 1536 | 28 | 0.31 |
| Llama-3.2-1B-Instruct | Llama3 | 1.2B | 2048 | 16 | 4.18 |
Each model was analyzed on an identical 500-prompt set (100 per category: benign, mild, dual-use, harmful, jailbreak) using the TAGM platform. All results were generated from matched prompt-level data, enabling direct per-prompt cross-model comparison. The analysis is organized around three questions: which signals are transferable across models, which category separations are robust, and where the picture breaks down.
**Figure 1.** Category-level means with standard errors for eight Tier 1 metrics across all three models. These metrics are dimensionless or normalized and are directly comparable across architectures without adjustment.
Three Tier 1 metrics show consistent cross-model category gradients:
Interior CV rises monotonically from benign through jailbreak in all three models. Jailbreak prompts produce dramatically higher interior CV (1.89, 1.80, 1.53 for the three models respectively) than any other category. The gap between harmful and jailbreak is the largest single-step jump in the entire metric suite, and it is significant in all three models (Mann–Whitney p < 0.001 for all).
Top-2 share declines from benign toward adversarial categories in both Qwen models, reflecting the shift from boundary-concentrated to interior-distributed attribution. In Llama-3.2-1B, however, the pattern is weak and the direction-of-effect reversal between mild and harmful is not significant. This metric is family-dependent.
Rank displacement metrics (Kendall τ and Jaccard overlap) show the expected directions — higher τ and lower overlap for adversarial prompts — but the magnitudes are small and the adjacent-category steps are often not significant, particularly in Qwen2.5-1.5B where the middle categories are nearly flat.
**Figure 2.** Category-level profiles for Tier 2 metrics. Raw stress score and net correction are not comparable across models due to different delta scales; stress/delta is the normalized variant. KL divergence is on a different scale for Llama (mean 1.6) than Qwen (mean 0.2–0.3).
Stress score rises monotonically from benign to jailbreak in all three models (3.16→3.40 in Qwen-0.5B, 3.18→3.62 in Qwen-1.5B, 2.12→2.47 in Llama). Because delta_scale is a constant per model, stress/delta preserves the same ordering. This is the most consistently monotone metric in the dataset.
KL divergence shows a striking non-monotone profile. In all three models, mild and harmful have lower mean KL than benign, while jailbreak has the highest KL of any category. This V-shaped profile (benign moderate, mild/harmful low, jailbreak high) is consistent across families — a qualitatively different pattern from what would be expected if KL simply measured alignment pressure. Jailbreak prompts are not merely harmful requests; they are structurally distinctive inputs that produce fundamentally different output distributions.
**Figure 3.** Category-level profiles for six LTP metrics. LTP mean M shows a sharp jailbreak drop in all three models. Profile shape fractions (steep, flat, inverted) show family-specific patterns.
LTP mean M (offset magnitude) shows a clear and consistent pattern: jailbreak prompts have dramatically lower offset magnitude than all other categories. This is the opposite direction from stress score — where stress rises for jailbreak, lateral tension drops. The interpretation is that jailbreak prompts navigate through regions where the alignment field is narrower or more directionally concentrated, producing less lateral displacement from the semantic trajectory. The harmful-to-jailbreak drop is highly significant in all three models (p < 0.001).
LTP profile shapes are overwhelmingly flat across all models and categories, with modest increases in steep-profile fraction for adversarial categories.
The central empirical question: do the same prompts produce the same measurements across models?
**Figure 4.** Per-prompt Spearman rank correlation across 500 matched prompts. Green = high transferability; red = low/no transferability. Metrics are sorted by mean cross-model ρ.
The 13 metrics tested fall into three clear tiers of per-prompt transferability:
Highly transferable (ρ > 0.80 in all pairs):
| Metric | Q0.5B vs Q1.5B | Q0.5B vs Llama | Q1.5B vs Llama |
|---|---|---|---|
| LTP mean M | 0.937 | 0.935 | 0.928 |
| Interior CV | 0.853 | 0.904 | 0.821 |
| Top-2 share | 0.872 | 0.453 | 0.514 |
LTP mean M and interior CV are the standout transferable metrics. LTP mean M achieves ρ > 0.92 across all three pairs, including the cross-architecture comparison. This is remarkable: a prompt that produces high lateral tension offset in the 896-dimensional Qwen-0.5B representation also produces high offset in the 2048-dimensional Llama-3.2 representation, despite completely different tokenizers, layer counts, and hidden dimensions.
Top-2 share is highly transferable within the Qwen family (0.87) but drops substantially cross-architecture (0.45–0.51), suggesting that the attribution distribution shape is partially architecture-specific.
Moderately transferable (ρ 0.40–0.80):
| Metric | Q0.5B vs Q1.5B | Q0.5B vs Llama | Q1.5B vs Llama |
|---|---|---|---|
| Stress score | 0.635 | 0.750 | 0.583 |
| KL divergence | 0.732 | 0.482 | 0.545 |
| Net correction | 0.621 | 0.564 | 0.668 |
| RD replacement | 0.465 | 0.635 | 0.510 |
| LTP mean V | 0.730 | 0.661 | 0.634 |
Stress score and KL divergence are moderately transferable. Stress is notably more transferable cross-architecture (0.75) than within-family for the 0.5B-to-1.5B comparison (0.64), which is unusual and may reflect the large parameter-count difference within the Qwen family.
Weakly or non-transferable (ρ < 0.40):
| Metric | Q0.5B vs Q1.5B | Q0.5B vs Llama | Q1.5B vs Llama |
|---|---|---|---|
| SFD density mean | 0.197 | 0.428 | 0.178 |
| RD mean τ | 0.257 | 0.092 | 0.023 |
| RD mean overlap | 0.234 | 0.423 | 0.172 |
Rank displacement (Kendall τ) is essentially non-transferable cross-architecture. The prompts that produce large rank reshuffling in one model are not the prompts that do so in another. SFD density mean is weak across the board.
**Figure 5.** Per-prompt scatter plots for the most and least transferable metrics (Qwen-0.5B vs Llama). Color indicates category. LTP mean M shows tight cross-model alignment; RD Kendall τ is a diffuse cloud.
Many TAGM metrics correlate strongly with sequence length. Stress score correlates at r = 0.79–0.85 with seq_len; interior CV at r = 0.80–0.88; LTP mean M at r = −0.79. Since categories differ in length distribution (jailbreak prompts are longer, median 44 tokens vs 10 for benign), any length-correlated metric will show apparent category separation even without alignment-specific signal.
After linear residualization on seq_len within each model:
| Metric | Raw ρ (Q0.5→Llama) | Residualized ρ | Change |
|---|---|---|---|
| LTP mean M | 0.935 | 0.887 | −0.048 |
| Interior CV | 0.904 | 0.777 | −0.127 |
| Top-2 share | 0.453 | 0.364 | −0.089 |
| Stress score | 0.750 | 0.491 | −0.259 |
| KL divergence | 0.482 | 0.323 | −0.159 |
LTP mean M is the most robust metric to length residualization. Its cross-model ρ drops by only 0.05, from 0.94 to 0.89. The signal it captures is largely independent of prompt length. Interior CV loses about 0.13 in ρ but remains substantial. Stress score loses the most (−0.26 cross-architecture), indicating that a large fraction of its apparent transferability reflects shared length distributions rather than shared alignment geometry.
**Figure 6.** Raw (blue) vs length-residualized (red) category profiles for stress score, interior CV, and LTP mean M. After residualization, stress category gradients flatten substantially; interior CV retains moderate gradients; LTP mean M loses its category gradient almost entirely — its raw separability was largely a length artifact.
The length residualization reveals an important distinction. For separability, removing length attenuates most category effects (Cohen's d for stress drops from 1.3–1.9 to 0.5–1.3; LTP mean M drops from d = 1.0 to d ≈ 0.05). For transferability, removing length barely affects LTP mean M (ρ drops from 0.94 to 0.89) while substantially reducing stress (ρ drops from 0.75 to 0.49). LTP mean M captures a prompt-level property that transfers across models and is not about length, even though its category-level separability is confounded with length. Stress captures a signal that is both length-dependent and model-dependent.
**Figure 7.** Cohen's d for each category relative to benign, across all three models. Red = category higher than benign; blue = lower. Consistent colors across models indicate cross-model robustness.
Ten of the fifteen metrics tested show same-sign Cohen's d across all three models for the benign-vs-(harmful+jailbreak) contrast:
| Metric | Direction | Q0.5B | Q1.5B | Llama | Robust? |
|---|---|---|---|---|---|
| Interior CV | ↑ | 1.59 | 1.22 | 1.64 | ✓ |
| Stress score | ↑ | 1.30 | 1.75 | 1.88 | ✓ |
| Net correction | ↑ | 1.59 | 1.02 | 0.68 | ✓ |
| LTP mean M | ↓ | −0.99 | −0.92 | −1.08 | ✓ |
| SFD density mean | ↓ | −0.72 | −0.57 | −1.32 | ✓ |
| RD mean overlap | ↑ | 0.51 | 0.21 | 1.31 | ✓ |
| RD mean replacement | ↓ | −0.53 | −0.53 | −1.27 | ✓ |
| LTP max PRC | ↑ | 0.72 | 0.46 | 0.45 | ✓ |
| RD mean τ | ↑ | 0.90 | 0.02 | 0.42 | ✓ (weak in Q1.5B) |
Five metrics show direction-of-effect reversals across models: entropy, top-2 share, middle share, KL divergence, and LTP mean V. These are not robust cross-model separators for the binary task despite sometimes achieving large within-model effect sizes.
**Figure 8.** Benign vs jailbreak distributions for six key metrics across all three models. The jailbreak signature is multi-dimensional and consistent: high interior CV, high stress, low LTP mean M, variable top-2 share and KL.
Jailbreak prompts are the most distinctive category in the dataset. Their signature across all three models is:
- Dramatically elevated interior CV (d = 1.6–2.0 vs benign). This is the single strongest individual-metric separator.
- Elevated stress score (d = 0.8–1.5). Alignment correction works harder on these inputs.
- Suppressed LTP mean M (d = −0.9 to −1.1). The lateral tension offset collapses — the alignment field around jailbreak prompts is narrower.
- Elevated KL divergence — but this is V-shaped, not monotone. The instruct model diverges more from base on jailbreak than on harmful content. The combination of high stress and low lateral offset is distinctive to jailbreak. Harmful prompts show elevated stress but do not show suppressed LTP. This suggests that jailbreak prompts occupy a geometrically different region of the alignment field: a narrow channel of high correction where the lateral structure collapses, rather than a broad region of elevated correction.
A striking scale-dependent finding: in Qwen2.5-1.5B, 50 out of 100 jailbreak prompts contain tokens with negative signed attribution — tokens that actively push against the alignment correction direction. This phenomenon is completely absent in Qwen2.5-0.5B (0/500 prompts across all categories) and in Llama-3.2-1B (0/500). It appears exclusively in jailbreak prompts at the 1.5B Qwen scale point.
This is the emergence, at intermediate scale, of the correction-suppressing tokens predicted by the theoretical framework but absent in the original 0.5B experiments. Whether this represents a genuine scale-dependent transition or a family-specific property of the Qwen 1.5B alignment recipe cannot be determined from three model points.
**Figure 9.** Spearman correlation matrices for 11 metrics within each model. Sequence length (bottom row/right column) shows which metrics are length-entangled.
The within-model correlation structure is broadly conserved across the three models:
- Stress, interior CV, and seq_len form a tight positive cluster (ρ > 0.7 in all models). These three are measuring overlapping signals, heavily entangled with prompt length.
- Top-2 share and entropy are strongly anti-correlated (ρ ≈ −0.8), as expected — concentrated attribution implies lower entropy.
- LTP mean M is strongly negatively correlated with stress and seq_len (ρ ≈ −0.8), and positively correlated with top-2 share (ρ ≈ 0.5–0.7). Longer prompts with more distributed attribution have lower lateral offset.
- SFD density mean and RD mean τ have weak, inconsistent correlations with the other metrics and with each other. The conservation of this structure across architectures suggests it reflects genuine properties of how alignment corrections interact with prompt geometry, not artifacts of a particular model's architecture.
**Figure 10.** PCA projection of the 8-metric feature space (standardized within model). Jailbreak (purple) is visually separated along PC1 in all three models.
An 8-metric PCA (stress, entropy, top-2 share, interior CV, KL, SFD density, RD τ, LTP mean M), standardized within each model, reveals consistent structure. PC1 explains 44–48% of variance and primarily separates jailbreak from non-jailbreak prompts. PC2 explains 18–22% and partially separates benign/dual-use from mild/harmful. The category geometry in this reduced space is qualitatively similar across all three models, with jailbreak forming a distinct cluster in the positive-PC1 direction.
**Figure 11.** AUROC for safe (benign + mild) vs risk (dual-use + harmful + jailbreak) classification using different approaches. The refusal direction analysis achieves the highest AUROC in Qwen-0.5B and Llama; the 11-feature logistic regression is competitive but does not clearly exceed the length-only baseline for Qwen-1.5B and Llama.
The mechanistic interpretability module reports:
| Model | LR (11 features) | Length-only | Refusal direction | Random |
|---|---|---|---|---|
| Qwen2.5-0.5B | 0.789 ± 0.033 | 0.669 | 0.811 | 0.497 |
| Qwen2.5-1.5B | 0.719 ± 0.036 | 0.669 | 0.768 | 0.500 |
| Llama-3.2-1B | 0.700 ± 0.036 | 0.671 | 0.819 | 0.500 |
Two patterns emerge. First, the refusal direction (a single direction in the residual stream, computed via difference-in-means) achieves the highest AUROC in two of three models, outperforming the 11-feature logistic regression. The refusal direction is the most efficient single feature for the binary safe/risk task. Second, the margin of the 11-feature LR over the length-only baseline is modest (0.03–0.12 AUROC), and the baseline itself is substantial (0.67). Length is doing much of the work. After residualizing out length, the incremental contribution of alignment-geometric features to binary classification is real but limited in these small models.
The KL divergence patterns in this dataset differ from those reported in the companion workshop paper on a 100-prompt subset measured at the first response-token position only. In the workshop data, Llama-3.2-1B showed jailbreak KL below benign baseline (1.34 vs 1.75). In the present dataset, using full-vocabulary KL computed by TAGM, jailbreak KL is the highest category in all three models (0.40, 0.48, 3.52).
Several methodological differences may explain the discrepancy: the workshop paper measures at a single token position (the first response token) while TAGM computes KL over the full next-token distribution at the end of the prompt; the prompt sets overlap but are not identical (100 vs 500 prompts, with the 500-prompt set including a dual-use category); and the TAGM KL computation uses the instruct model's full chat template formatting.
The pattern within TAGM is consistent: jailbreak prompts produce large KL divergence from base in all three models, while mild and harmful prompts show lower KL than benign. This V-shape (moderate baseline, suppressed middle, elevated jailbreak) suggests that the alignment correction acts differently on jailbreak inputs than on harmful-content inputs — consistent with the distinction between structural and content-based signals identified in the original experimental work.
- LTP mean M is the most transferable signal (ρ = 0.89–0.94 raw, 0.88–0.89 after length residualization). The lateral tension offset magnitude is a genuine per-prompt property that persists across architectures, parameter counts, and hidden dimensions. This is the strongest empirical evidence in the dataset for a prompt-level geometric quantity that is not a proxy for length or a recipe-specific artifact.
- Interior CV transfers strongly (ρ = 0.82–0.90 raw, 0.61–0.78 residualized). The coefficient of variation of interior attribution is robust, though partially length-dependent.
- Direction-of-effect is robust for most metrics. Ten of fifteen metrics show the same sign of Cohen's d (adversarial vs benign) across all three models. Disagreements concentrate in entropy, top-2 share, and KL divergence.
- Rank displacement (Kendall τ) is essentially non-transferable (ρ = 0.02–0.26). How alignment reshuffles the candidate token set is highly model-specific.
- SFD density mean is weakly transferable (ρ = 0.18–0.43). The QK routing subspace engagement varies too much across architectures.
- Top-2 share transfers within-family but not cross-architecture (0.87 within Qwen, 0.45–0.51 cross-family). The bookend-vs-distributed attribution pattern is partially architecture-dependent.
- Jailbreak is the most geometrically distinctive category — high interior CV, high stress, low LTP mean M, elevated KL. This signature is consistent across all three models.
- Benign and dual-use are geometrically similar across most metrics, and both are distinct from harmful and jailbreak. The benign/dual-use vs harmful/jailbreak boundary is the primary separation axis.
- Mild occupies an intermediate position that varies by model and metric. It is not reliably separated from either benign or harmful by any single metric.
- Negative tokens emerge at 1.5B scale in Qwen, exclusively in jailbreak prompts (50/100 jailbreaks show correction-suppressing tokens vs 0/400 in other categories, 0/500 in both other models). This is a scale-dependent phenomenon that warrants further investigation.
- The length confound is substantial. Many category-level effects (especially stress score, interior CV) are attenuated by a factor of 2–3× after length residualization. Any cross-model analysis of these metrics must control for length, and any benchmark using these signals must be length-balanced.
- The refusal direction outperforms multi-metric logistic regression for binary safe/risk classification in 2 of 3 models, despite using a single direction vs 11 engineered features. The marginal value of multi-metric alignment geometry over a single refusal direction, for the binary classification task, is limited at these model scales.
Statistical conventions. All reported Spearman correlations use 500 matched prompts. Mann–Whitney U tests are two-sided. Cohen's d uses pooled standard deviation. Bootstrap confidence intervals use 2000 resamples.
Length residualization. Computed as OLS linear residual of metric on seq_len, within each model separately.
Data integrity. All 1500 results (500 per model) passed dimensional consistency checks: per-token arrays match seq_len, heatmap dimensions match architecture, and proof-1 decomposition errors are below 1.22 × 10⁻⁴.
Analysis conducted on TAGM session exports from Qwen2.5-0.5B-Instruct (session_20260510_062330), Qwen2.5-1.5B-Instruct (session_20260510_173023), and Llama-3.2-1B-Instruct (session_20260510_220845). Platform: TAGM v2, commit as of May 10, 2026.