Layer Alignment

Layer Alignment

Layer alignment is an experimental, off-by-default presenter feature. It rotates each panorama layer that was drawn in an older engine frame so that it meets the newest centre camera. The panorama budget does not redraw every bearing every frame: the centre is redrawn every frame and the rest in rotation. A side band is therefore often a frame or two old, and the sky, floor and rear bearings can be a second or more old. The stick turns the camera by up to 2.3° per frame, so while the player turns, neighbouring pictures meet at a join with the world doubled across it. These are the "ghosted edges" at panel joins.

Each layer carries the engine camera it was drawn with. Turning the layer by the rotation from that camera to the newest centre camera puts every distant object where the newest camera sees it. The turn is applied rigidly in the vertex stage, and each panel's edge fade is recomputed per fragment on the GPU against where its neighbours now are, so no mesh is rebuilt on the CPU.

This page documents EngineLayerAlignment.swift in depth. Background on the panels and the baked fades it modifies is in Immersive Presenter. Where the poses, epochs and cut epoch come from is in Panorama System.

Source files

File	Role
EngineLayerAlignment.swift	EngineLayerPose; the EngineLayerAlignment planner (rotations, limits, coverage, CPU alpha, shader constants, Metal source); EngineLayerAlignmentState; EngineLayerAlignmentStatistics; EngineAlignedPipeline
EngineImmersiveScreenGeometry.swift	Panel cameras (basis(of:), imagePoint(of:)), baked fades (panelAlpha, panelAlphas) that alignment must reproduce when nothing is turned
EngineImmersive.swift	Calls update/measure/disable once per engine frame and binds the constants per panel (L995-L1008)
EnginePanoramaTexture.swift	Unpacks layer_pose (nine floats per layer) into [EngineLayerPose?]
native/EngineHost/halo_settings.c	HALO_LAYER_ALIGN default and the live layer_align field
Tests/LayerAlignmentValidation.swift	CPU validation against the host's camera construction
Tests/LayerAlignmentGPUValidation.swift	GPU validation of the shipped shader
native/EngineHost/tests/test_halo_settings_layer_align.c	The setting is off unless exactly 1; the live toggle works

Enabling it

Control	Default	Where
HALO_LAYER_ALIGN=1	off (any other value, including empty, 0 or yes, is off)	halo_settings.c:75-76
Settings window, section "Joins", toggle "Line up older views"	mirrors the above, live	EngineSettingsView.swift:90-92

The presenter reads halo_settings_layer_align() every frame. The setting stays off until "a b30 stick-turn A/B on the headset" shows that the joins it fixes outweigh its side effects (EngineLayerAlignment.swift:76-95, halo_settings.c:69-74). It has only been checked on synthetic pictures: pure rotation, no parallax, no real weapon. With it off, the panels are drawn exactly as baked, and each layer's camera offset is still measured as the A/B baseline.

Documented side effects (while the stick turns)

From the source comment:

• Side bands bend at eye level. The weapon zone's blend (−4° to +8° of elevation, 56° to 68° of azimuth) shears the side bands. Verticals tilt up to 16° at a 2.3° turn and 27° at the 4° limit. Below the blend, content at 56-68° is squashed to 0.49 across. A 7° pitch correction squashes the blend band to 0.23 of its height. Each effect comes and goes as the side bands are redrawn.
• Harder joins. A band turned away from its neighbour narrows the cross-fade at that join from about 3.5° to 0.5° at the 4° limit. Parallax, animation and lighting differences then show as a harder edge.
• Estimated bounds. The weapon zone's bounds have not been checked against real weapon renders or third-person vehicles, and the zone applies even when no weapon is drawn.

The settings window's note says the same thing in user terms: "the picture can bend at eye level in the views either side of the centre, and some joins can show a sharper edge".

Frames and poses

EngineLayerPose

The host exports, per layer, the engine camera the layer was drawn with, before the pass yawed or pitched it to the layer's bearing. It is nine floats: position (world units), forward and up, in Halo's world, where z is up (panorama.h). All zeros means unknown.

EngineLayerPose(floats:) returns nil for fewer than nine floats or all zeros (L19-L24).

presenterFromWorld (L32-L41) maps world directions into the presenter frame, which is the frame every panel is laid out in: x is the camera's right, y its up, z behind it.

f = normalize(forward)
u = normalize(up - (up·f) f)        // Gram-Schmidt so the result is a proper rotation
r = f × u                           // Halo's right is forward × up (panorama_hooks.inc)
presenterFromWorld = rows [ r, u, -f ]

It returns nil when any value is non-finite, or forward or up has length ≤ 1e-4, or up is parallel to forward.

Panels and layers

Constant	Value	Meaning
panelLayers	[0, 1, 2, 7, 8, 9, 5, 6]	Presenter panel (source index) to engine layer
panelCount	8
centrePanel	1	Never turned; the reference
ringPanels	[0, 1, 2, 3, 4, 5]
skyPanel, floorPanel	6, 7
weaponPanels	{0, 2}	The ±60° side bands, the only ones besides the centre that the first-person weapon is drawn into (HALO_PANORAMA_WORLD_FP)
beneath[k]	[[], [0], [1], [2], [3], [4, 0], [1,3,4,5,0,2], [1,3,4,5,0,2]]	Panels drawn before k that it can overlap; its fade is measured against these
coverers[k]	band: its two ring neighbours plus both caps; cap: the six bands	Panels that can cover k's edge

Source: L99-L112, L382-L383.

Algorithm

flowchart TD
    S["Snapshot: layerPoses, layerEpochs, cutEpoch, sequence"] --> K{"same sequence and panel signature as last plan?"}
    K -- yes --> C0["reuse cached constants"]
    K -- no --> D["desired(): per-panel rotation from its camera to the newest centre camera"]
    D --> P["plan(): try limit levels full, half, capsTwistOnly"]
    P --> CL["clamp(): ring joins outward from centre, caps: keep twist, tilt into each band's cone"]
    CL --> CV{"coverageHolds(): every turned panel edge inside another turned panel?"}
    CV -- yes --> OK["Plan at this level"]
    CV -- "no, next level" --> CL
    CV -- "no level passes" --> ID["identity plan (level none)"]
    OK --> CO["constants(): 3 float4 vertex + 30 float4 fragment per panel"]
    ID --> CO
    CO --> GPU["aligned_vertex turns panel, aligned_fragment recomputes fade per fragment"]

Loading

1. Desired rotations

desired(poses:epochs:cutEpoch:) (L275-L293) returns all identities unless there are at least ten poses and epochs and layer 1's pose is usable. Then, with centre = poses[1].presenterFromWorld and newest = epochs[1], for each non-centre panel:

Condition	Result
epochs[layer] == 0 or > newest	identity (never drawn, or claims to be newer than the centre)
cutEpoch > 0 and epochs[layer] < cutEpoch	identity, counted as a cut panel: the layer shows another shot
pose unusable	identity
q = shortest(quat(centre * cameraᵀ)), angle ≤ identityRadians (0.02°)	identity: same camera
angle > max(1, newest - epoch) * maximumRadiansPerFrame (12° per frame of age)	identity, counted as a cut: a rotation the stick cannot make, so a cut the host did not see
otherwise	rotations[panel] = q

q takes a pixel that was straight ahead of camera k to where camera 1 sees the same world direction. The test spells out the sign: if the camera turned right by 2.3°, what the older layer showed straight ahead is now 2.3° to the left.

2. Limits and clamping

Turning one layer against a neighbour opens the far join. Every join must stay inside its overlap: ring bands overlap by 4.5°, and bands and caps by 6.4° (L61-L67).

Limits	ringJoin	capTilt
.full	4.0°	4.5°
.half	2.0°	2.25°
.capsTwistOnly	0 (ring as drawn)	0 (caps twist about their own vertical only)

Join measure (L348-L351). For the relative turn between a panel and its neighbour, w = log(relative) is the rotation vector (axis times angle), and:

joinMeasure = |w.y| + 1.12 * |w · join|

Here join is the horizontal unit vector at the join's azimuth. The yaw part slides the band along the ring. The part about the join's own direction tilts the band's edge, and 1.12 = tan 48.1° is the band's height at its edge.

clampJoin(target, near: reference, join:, limit:) (L352-L358). It computes relative = shortest(reference⁻¹ * target). If the measure exceeds the limit, the relative rotation vector is scaled by limit / measure, which keeps the axis but shortens the angle. With a limit of 0 it returns reference.

clamp(desired, limits) (L317-L344) walks outwards from the centre, so the joins nearest the gaze are the ones fully aligned:

Panel	Clamped against	Join azimuth
0 (−60°)	centre (identity)	−30°
2 (+60°)	centre	+30°
3 (+120°)	panel 2	+90°
5 (−120°)	panel 0	−90°
4 (180°)	panel 3 at +150° and panel 5 at −150°, alternated 4 times	±150°
6, 7 (caps)	clampCap against the up vectors of all six turned bands plus +Y

A ring band whose desired turn exceeds farRadians (90°) follows its inner neighbour instead of its own camera, because so far round its own axis means little.

clampCap(target, ups:, limit:) (L361-L376) splits the cap's rotation into a twist about +Y, which is kept in full because twisting a cap about its own axis cannot uncover anything, and the image of its axis. That axis is then pulled into every band's cone of half-angle limit around the band's up, in eight passes over all cones. The result is quat(from: +Y, to: axis) * twist.

3. Coverage check

coverageHolds(sources:rotations:samples: 24, margin: 0.01) (L388-L413) samples 25 points along each of the four edges of each turned panel. For each point it finds a panel among coverers[k] whose turned image contains the point, with a 1 % margin. The panel that covered the previous sample is tried first. A hole in the sphere would have to be bounded by panel edges that nothing else covers, so if every edge is covered there is no hole.

4. Plan

plan(sources:desired:cut:) (L297-L311):

1. If no desired turn exceeds 0.02°, it returns the identity plan at level none.
2. Otherwise it tries full, half and capsTwistOnly in order and returns the first clamped set that passes coverageHolds.
3. If none passes, it returns the identity plan.

The plan records rotations, desiredDegrees, appliedDegrees, level and cutPanels.

5. The weapon zone (side bands only)

The first-person weapon is world geometry in the two side bands, attached to the camera. It is therefore already where it belongs, and turning it would tear it away from the centre band's copy. Side bands keep their lower inner region as drawn and bend smoothly into the full turn above and outside it.

weaponZone = (sin 56°, sin 68°, sin −4°, sin 8°) (L137-L138). correctionWeight(d) (L213-L220) is:

side   = |d.x| / sqrt(d.x² + d.z²)                         // sine of azimuth from straight ahead
across = d.z > 0 ? 1 : smoothstep(sin56°, sin68°, side)    // anything behind takes the full turn
up     = smoothstep(sin(-4°), sin8°, d.y)
weight = 1 - (1 - across) * (1 - up)

A side band applies angle * weight(d) about its axis at base direction d (Turn.place). This map is not rigid, so finding which base direction lands at a presenter direction p needs a solve. Turn.base(p) (L244-L257) looks for the weight w with h(w) = weight(rotate(p, -w)) - w = 0. h falls from h(0) ≥ 0 to h(1) ≤ 0, and regula falsi (Illinois) runs weightSteps = 5 iterations. The test shows this inverse is accurate to under 0.005° for a 4° turn. Rigid panels invert exactly with rotate(p, -1) (Rodrigues).

6. Per-fragment fade

alpha(panel:at:sources:rotations:) (L419-L436) is the CPU twin of the shader's align_alpha:

d = base direction of panel k at p          (outside its image -> 0)
edge = panelAlpha(st, all edges)            (same 0.15 smoothstep feather as the baked mesh)
if edge >= 1: return 1
laid = max over j in beneath[k] of smoothstep(0, laidSoftness, inside_j(p))
       where inside_j = min(1-|q.x|, 1-|q.y|) of p in turned panel j
return 1 - (1 - edge) * laid

laidSoftness = 0.015 is small enough that, with nothing turned, the fades equal the baked ones: a band's fade ends 0.017 inside its neighbour. It is large enough that a neighbour's edge crossing a turned fade leaves a ramp rather than a hard line. For side bands beneath, sideBandReject = -0.25 skips the costly inverse when p is clearly outside the band unturned, since a side band only turns a few degrees (L115-L121).

Shader constants

constants(sources:rotations:) (L450-L472) builds arrays of float4, so Swift and Metal cannot disagree on packing:

Vertex stage, buffer 3: vertexConstantCount = 3

Index	Content
0	turn as axis xyz and angle w (identity is (0, 1, 0, 0))
1	weaponZone
2	(zoned ? 1 : 0, 0, 0, 0)

Fragment stage, buffer 2: fragmentConstantCount = 6 + 4 * 6 = 30

Index	Content
0-3	This panel's camera, unturned: (right, px), (up, py), (forward, zoned), axisAngle(turn)
4	weaponZone
5	(beneath count, laidSoftness, sideBandReject, 0)
6 + 4j ...	Each beneath panel's camera. A rigid panel's turn is baked into its axes, so its image of a presenter direction is three dot products. A side band keeps its own axes plus its turn and is solved with align_base.
... 29	zero padding

The maximum is six panels beneath (for the caps), which gives exactly 30 entries.

Shader

EngineLayerAlignment.shaderSource (L477-L588) is appended to the renderer's own shader source. It reuses that source's Vertex struct and cubic_sample.

Function	Role
align_rotate(v, axisAngle, scale)	Rodrigues rotation by angle * scale
align_weight(d, zone)	Mirror of correctionWeight
aligned_vertex	base = vertex position, placed = align_rotate(base, turn, zoned ? weight(base) : 1), then position = matrices[eye] * placed. Passes both base and placed to the fragment stage.
align_image(d, camera, st)	Projects a direction into a camera's image; false behind it
align_base(placed, camera, zone)	Five-step Illinois solve, identical to Turn.base
align_ramp(x)	smoothstep over 0.15
align_alpha(base, placed, fade)	Mirror of alpha(panel:at:...)
aligned_fragment	curve_fragment's colour (cubic for the centre panel, bilinear otherwise; row wash and fade outside the source rectangle) with the alpha computed per fragment instead of baked

The aligned pipeline

EngineAlignedPipeline (L687-L735) is created in the renderer's init. It holds a copy of the panorama pipeline descriptor (formats, sample count, amplification, over blending), and nothing is compiled at construction.

• The first call to pipeline() sets the state to building and compiles on DispatchQueue.global(qos: .userInitiated). It compiles the whole base shader again plus the aligned functions, which takes about 200 ms on a Mac. The call itself returns nil.
• Later calls return nil until the state is ready, then the pipeline state.
• A build that fails (missing functions or a compile error) goes to failed for good, logs layer alignment unavailable, and leaves alignment off. It cannot affect the renderer.
• A session that never turns the setting on never builds the pipeline.
• settled exists for tests.

State and statistics

EngineLayerAlignmentState (L593-L634) is owned by the renderer:

Call	When (from drawFrame)	Behaviour
update(sequence:signature:sources:poses:epochs:cutEpoch:)	panorama active, setting on, pipeline ready	Plans only when the snapshot sequence or the panel signature changed; otherwise returns the cached constants. Records statistics once per sequence as aligned.
measure(sequence:poses:epochs:cutEpoch:)	panorama active, setting off or pipeline not ready	Clears the constants. Once per sequence, computes desired and records its angles as misalignment with applied 0 (framesOff). Costs one quaternion per layer per engine frame.
disable()	no panorama	Clears the constants; enabled = false

Planning is therefore done once per engine frame, not per display frame. The validation run reports plan timing (median and p90, in milliseconds).

EngineLayerAlignmentStatistics (L641-L679) is indexed by engine layer (10 entries; layers 3 and 4 stay 0). It is emitted as panoramaLayerAlign in every timeline record and in report.json:

Key	Meaning
enabled	Which mode the last counted frame ran in (the setting can change mid-run)
frames	Distinct engine frames counted
level	Last plan's level: full, half, capsTwistOnly or none
framesByLevel	Aligned frames per level
framesOff	Frames measured with alignment off
framesWithCut	Frames where at least one panel was left alone as pre-cut
appliedDegrees, misalignedDegrees	Last frame per layer: turn applied, and camera offset from the newest centre
appliedDegreesSum, misalignedDegreesSum	Running sums, so two records a second apart give that second's mean
appliedDegreesMax, misalignedDegreesMax	Run maxima

tools/engine_vision_report_summary.py splits consecutive timeline records by enabled into an on/off A/B and prints, per layer, the mean degrees misaligned against the mean degrees turned (engine_vision_report_summary.py:284-314). These numbers are camera offsets, not the visible join error. The tool's comment says to judge the joins from a recording.

Integration in the presenter

From EngineImmersive.swift:703-722 and L995-L1008:

1. panoramaPanelSources (the cameras the meshes were built from) and panoramaPanelSignature come from the most recent mesh rebuild.
2. If halo_settings_layer_align() != 0, alignedPipeline.pipeline() is non-nil, and update(...) returns constants, then alignmentDraw = (pipeline, constants).
3. In encode, alignment is used only if there are constants for all eight panels. Per panel, the vertex constants go to buffer 3 and the fragment constants to buffer 2. The same baked meshes and textures are drawn, and only the pipeline changes.
4. submitted.layerAlignment = alignment.statistics feeds diagnostics.

Head pose never enters the turn. The turn is applied on the sphere, before each eye's view matrix, so a rolled, pitched or turned head sees the turned older picture and the fresh centre agree on screen exactly as they agree on the sphere. Test 9 below asserts this.

Testing

Both tests are built with -O and run by tools/run_source_checks.py on macOS (run_source_checks.py:271-283). The GPU test receives EngineImmersive.swift as its argument so it compiles the shipped shader. Both use the headset's reported projections: ring (1.5847, 0.7593) (32.25° half width, 105° tall) and caps (0.8910, 0.7593) (48.3° half width).

LayerAlignmentValidation.swift (CPU)

#	Check	Threshold
1	presenterFromWorld maps Halo forward to −Z, right (−Y world) to +X, up to +Y; determinant 1; all-zero, NaN and parallel poses rejected	1e-6
2	With one camera for all layers, every panel is laid exactly where the host's bearing construction (hostBearing, rebuilt from panorama_hooks.inc) draws that layer	< 0.01°
3	For nine old/new camera pairs (stick yaw and pitch, pitched cameras, the caps, a 46° and an 80° turn), turned layers meet the newest centre; unturned they were > 40° off somewhere	< 0.01° aligned
3b	Direction of turn: a 2.3° right turn moves older pictures 2.3° left	< 0.001°
4	A still camera gives exactly identity, level none, identity constants, and per-fragment fades whose per-panel contribution differs from the baked mesh by under 0.002 (half an 8-bit level) over 12,000 sphere directions; the layout passes coverage, worst coverage > 0.9999
5	A side band turned 10° outward fails coverageHolds, and worst coverage < 0.5
6	300 plans: stick rates −2.3/1.1/2.3/4.6° per frame, pitches −60...55°, pitch rates, five Build75-measured age sets. Each passes coverage with worst composite > 0.9999; sides get > 50 % and caps > 70 % of their desired turn; one-frame-old side bands at stick rate turn fully; the centre never turns; sides never over-turn
6b	A level 2.3° turn: side 0 gets exactly 2.3°, sky 69°, floor 92°, side 2 is held between 3.5° and 4.6° by its join
7	Weapon region (azimuth 28-55°, elevation −50 to −5°) of both side bands moves < 0.001° under a ~3° turn, and is turned fully outside the zone; weapon copies in the centre/side overlap coincide	< 0.001°
8	Illinois inverse of a 4° bent turn	< 0.005°
9	Head pose does not matter: two arbitrary head rotations see stale-turned and fresh positions agree	< 1e-3 m
10	20° over 5 frames is aligned; a cut at epoch 97 leaves the six older layers alone (cut == 6) but still turns layer 8, drawn after it; 30° in one frame counts as a cut; a missing centre camera, an all-zero pose, a layer newer than the centre or a never-drawn layer turn nothing
11	State: the same sequence plans once, a new sequence plans again, statistics sums and levels are correct, the dictionary is valid JSON with 10 entries; disable, measure (counted once per sequence, not double-counted after a plan) and re-enable behave

LayerAlignmentGPUValidation.swift (Metal)

#	Check	Threshold
1	Nothing turned: the aligned pipeline draws what the baked pipeline draws, over nine views (including near the poles)	max channel difference < 0.02, mean < 0.0005
2	Three headset-like plans, all-white pictures: every composited pixel stays fully covered	darkest > 0.995
3	A synthetic world where each panel is drawn from its own older camera. Through the aligned pipeline every pixel is compared with the newest camera's truth: no view is worse than drawn-as-is, and mean error < 0.3x (stick scenario) or < 0.95x (older layers with aim moving) of the unaligned error. A fully-turnable plan matches the all-newest reference to within 0.003 mean colour (excluding the deliberately unturned weapon zone)
(cost)	Times both pipelines drawing all eight panels into a 1920x1824 eye; reported only
4	EngineAlignedPipeline: nothing compiled before the first request; the first request returns nil in < 20 ms and builds in the background; the original descriptor is left untouched; a deliberately broken source stays nil

Both tests describe themselves as synthetic: no parallax, animation or weapon geometry. The source comment states that a headset A/B is still required.

Related pages

• Immersive Presenter: panel geometry, baked fades, render loop
• Panorama System: layer_pose, layer_epoch and cut_epoch production (panorama_hooks.inc)
• Panorama Budget and LOD: why layers have different ages
• Runtime Settings: HaloSettings.layer_align
• Diagnostics and Telemetry: panoramaLayerAlign fields and the summary tool