Radial Fog

Radial Fog

Radial fog is an opt-in (HALO_RADIAL_FOG=1, default off) correction for a seam the panorama introduces. Halo measures atmospheric and planar fog as depth along the view's forward axis. On PC there is one view, so that is invisible; the Panorama System renders the same world from several bearings, each with its own forward axis, so a point near the join between a ring bearing and the up or down cap is fogged by cos(elevation) of its distance in one bearing and sin(elevation) in the other, which can show as a brightness step along the join. Radial fog rewrites the eligible vertex shaders (and the fixed-function fog path) so fog is measured from the eye instead; every bearing then agrees, and on each bearing's own axis the result is unchanged. It applies only while a panorama bearing is being drawn; the flat window keeps PC fog.

Source files

File	Role
radial_fog.h	The analysis (comment), the vs_1_1 token scanner and rewriter, and the fixed-function range-fog formula.
d3d9.c	Counts eligible terms per vertex shader at creation.
d3d9_render.inc	Draw-time mode selection for programmable draws; range fog in the fixed-function 3D path.
metalrenderer.m	Rewritten translations, pipeline keys, fog varyings for fixed pipelines, prewarming, manifest field.
halo_settings.c / halo_settings.h	halo_settings_radial_fog(): the process-wide switch (see Runtime Settings).

Halo's fog constants

From the analysis at the top of radial_fog.h:

Constant	Written by	Value
c4	00518F40 (called by 005175C0 after 005176D0), rewritten with the same eye by 00519F70	(eye, 2)
c6	005176D0, SetVertexShaderConstantF(6, ..., 4) at 00517A4F, once per view	s · (fwd, -(near + fwd·eye)), atmospheric fog, s = 1/(far - near)
c7	same	the fog plane; normally a world plane. When the camera stands in a fog region with no plane, 00555330 sets the planar fog type 007C1424 to 2 and 005176D0 lays the plane across the view: (fwd, fwd·eye + [007C1268]) with 007C144C forced to 1, so c7 = (-fwd, fwd·eye + k) is a depth along fwd with the opposite sign.
c8	same	a · (fwd, -fwd·eye), planar fog's viewing depth, a = 1/[007C1448]

fwd is the view's forward vector (007C1234) and eye its position (007C1228). Nothing else writes c4, c6 or c8: other engine uploads start at c10 or above, and the D3DX effects in fx.bin carry only pixel shaders. The shaders in shaders\vsh.bin evaluate c6/c8 as planes, either dp4 P, cK or dp3 P, cK followed by adding cK.w, so fog grows with depth along fwd.

The rewrite

Each fog term T (an instruction that evaluates cK as a plane against a position) has s · (|P - eye| - fwd·(P - eye)) added to it. The sum is s · (distance - near): identical in every bearing, and equal to the original on each bearing's axis (where depth equals distance). Because fwd is a unit vector, s = |cK.xyz|, so no new constants are needed. The term's own instruction is kept, so everything the shader does with the value afterwards (density, planar blend, oFog, a texture coordinate for the fog-plane pass) is untouched; the result is what PC shows for that point when the player turns to face it.

With t a temporary the shader never uses, halo_radial_fog_rewrite replaces T (4 tokens) with ten instructions (39 tokens, HALO_RADIAL_FOG_GROWTH = 35):

add t.xyz, P, -c4        d = P - eye
dp3 t.w, t, t            d.d
dp3 t.x, t, cK           cK.xyz . d          (s times depth along fwd)
dp3 t.y, cK, cK          s * s
mul t.w, t.w, t.y        s^2 |d|^2
rsq t.y, t.w             1 / (s |d|); MojoShader returns FLT_MAX for rsq(0)
mul t.w, t.w, t.y        s |d|  (a point at the eye yields 0, not NaN)
add t.w, t.w, -t.x       s (|d| - depth)     (-t.w for c7, whose sign is reversed)
T   t.x, P, cK           the original term, destination replaced by t.x
add D, t.x, t.w          the original destination, write mask and modifiers

Every instruction reads at most one constant register, as vs_1_1 requires.

Eligibility

halo_rf_scan walks the token stream and returns the number of terms and the highest free temporary. A shader is left unchanged (0 terms) when:

• it is not vs_1_1 (0xFFFE0101), is malformed, or exceeds 1 MiB;
• it contains an instruction the scanner cannot step over (halo_rf_operands knows nop, mov, add, sub, mad, mul, rcp, rsq, dp3, dp4, min, max, slt, sge, exp, log, lit, dst, lrp, frc, expp, logp, dcl, def). Matrix instructions (m4x4 and friends) read implicit register spans, so the scan could not prove a scratch register unused; none of Halo's 64 shaders uses them, and custom shaders containing them are preserved unchanged;
• it references a temporary at or above r12, or every temporary r0-r11 is used;
• it has no term.

A term (halo_rf_term) is either dp4 D, P, cK, or dp3 D, P, cK without a result modifier that is immediately followed by add D, D, cK.w covering every lane it wrote (halo_rf_plane_w), where P is an unswizzled, unmodified temporary or input and cK is an unswizzled, unmodified, non-relative read of c6 or c8, or of c7 in HALO_RADIAL_FOG_VIEW_PLANE mode. Any other dp3 of these constants is left alone. tests/test_radial_fog.c checks that every read of c6, c7 and c8 in vsh.bin is a recognised term.

Two modes exist (radial_fog.h:80-86): HALO_RADIAL_FOG_DEPTH (1) rewrites c6 and c8 terms; HALO_RADIAL_FOG_VIEW_PLANE (2) also rewrites c7 terms.

Accuracy

The distance is evaluated per vertex and interpolated. Every bearing interpolates the same vertex values, so joins agree to floating-point precision for the same geometry and eye. Inside a large triangle the interpolated distance can exceed the true fragment distance: for an equal-range edge subtending 2θ the midpoint ratio is 1/cos θ.

Data flow

flowchart TD
    A["CreateVertexShader"] --> B["halo_radial_fog_terms(DEPTH) and (VIEW_PLANE)"]
    B --> C["radial_fog_terms, radial_fog_view_plane stored on the shader"]
    C --> D{"HALO_RADIAL_FOG=1?"}
    D -->|"yes"| E["Worker prewarm: translate radial variants of eligible shaders"]
    F["Programmable draw"] --> G{"setting on, panorama pass >= 0, shader has terms?"}
    G -->|"no"| H["radial_fog = 0: original translation and pipeline key"]
    G -->|"yes"| I{"rewrites c7 and planar fog type 007C1424 == 2?"}
    I -->|"yes"| J["radial_fog = VIEW_PLANE"]
    I -->|"no"| K["radial_fog = DEPTH"]
    J --> L["xlat_get(stage, tokens, key, mode): halo_radial_fog_rewrite then MojoShader"]
    K --> L
    L --> M["Fix oFog output index to 0, pipeline key includes mode"]
    N["Fixed-function 3D draw"] --> O{"setting on, panorama pass, FOGENABLE, FOGTABLEMODE NONE?"}
    O -->|"yes"| P["Per-vertex halo_fixed_radial_fog from view-space distance"]
    P --> Q["Fixed RHW/clip pipeline with fog varying and fog-colour mix"]
    O -->|"no"| R["No fog (as before)"]

Loading

Draw-time selection

Programmable draws (radial_fog_mode) return 0 unless the setting is on, a panorama pass is active (panorama_pass >= 0) and the bound vertex shader has at least one depth-mode term. They return VIEW_PLANE when the shader's c7 reads make the view-plane rewrite differ from the depth rewrite and the planar fog type at 007C1424 is 2 (005176D0 sets it once per view, before any of that view's draws), otherwise DEPTH. Shaders without fog terms never see the flag, so they keep a single pipeline whichever way the switch is set. D3DRS_FOGENABLE and FOGCOLOR are passed as usual; fog is blended in the fragment stage from the vertex shader's oFog.

Fixed-function range fog

The CPU fixed-function 3D path previously omitted vertex fog entirely. With radial fog on, during a panorama pass, when FOGENABLE is set and FOGTABLEMODE is NONE (Halo's low-capability renderer sets FOGTABLEMODE=NONE and FOGVERTEXMODE=LINEAR in 005176D0), ff3d_draw reads FOGSTART/FOGEND/FOGDENSITY and computes, per vertex, halo_fixed_radial_fog of the view-space position. The view transform already puts the eye at the origin, so the vector length is the range:

FOGVERTEXMODE	Factor
1 EXP	exp(-density · d)
2 EXP2	exp(-(density · d)²)
3 LINEAR	(end - d)/(end - start); if end == start, 1 when d < end else 0
other (NONE)	the application-supplied factor, the specular colour's alpha

The factor is clamped to [0, 1] and carried in the vertex's fog field; the pipeline is built in HALO_RADIAL_FOG_DEPTH mode with a fog varying.

Renderer integration

• Translation. xlat_get adds the mode to the translation key only when it is non-zero, so mode 0 keeps the exact original translation identity. It rewrites the tokens before calling MojoShader (an allocation failure fails the translation; a shader the rewrite declines is translated unchanged). For rewritten shaders only, it corrects MojoShader's oFog output index from 1 to 0 so the fixed fragment finds the unindexed [[user(fog)]] it actually emits; with the switch off, the earlier behaviour is preserved.
• Pipeline keys. program_key hashes radial_fog only when non-zero; fixed_rhw_key hashes fog_enable and radial_fog only when radial fog is active. Existing pipelines and manifest recipes therefore keep their keys with the switch off.
• Fixed pipelines. When fog and radial fog are both set, the generated RHW/clip vertex function gets a float4 fog field (GPU stride 128 instead of 112 bytes) and a [[user(fog)]] output, and the fixed fragment declares fog uniforms with explicit float padding so the fog colour is at byte 32, where the CPU writes it. The original layout and stride are kept when the switch is off.
• Prewarming. compiler_for caches the setting in radial_fog_prewarm. When on, workers translate the radial variants of every eligible vertex shader at creation and prewarm the fog-carrying fixed vertex functions; when off, recipes recorded with a radial mode stay in the manifest but are not queued.
• Manifest. Recipe format version 2 adds the radial_fog field after the 1080-byte version-1 prefix (_Static_assert at metalrenderer.m:1311-1313); version-1 files are migrated on load and left intact (see Shader Translation).

The switch

halo_settings_radial_fog() reads HALO_RADIAL_FOG once per process under pthread_once; only the exact string 1 enables it. Caching it makes the value immutable, so all bearings, the renderer and diagnostics see the same answer even if the environment changes later. Translation units that may be linked without halo_settings.c (device tests that include d3d9.c or metalrenderer.m) provide a weak default of 0. The comment in d3d9_render.inc states the reason it is off by default: pending A/B validation on the headset.

Variable	Default	Effect	Read at
HALO_RADIAL_FOG	off	1 enables radial fog in panorama passes.	halo_settings.c (initialize_radial_fog)
HALO_RADIAL_FOG_MSL_DIR	unset	Test only: test_radial_fog_renderer.m saves the emitted MSL there for offline metal -c on visionOS.	test
HALO_RADIAL_FOG_GENERATED_DIR	generated engine	Test only (run_source_checks.py): where test_radial_fog.c finds the translated engine.	test runner

Tests

Test	Run by run_source_checks.py	What it asserts
test_radial_fog.c	yes	An interpreter with MojoShader's Metal semantics (rsq/rcp of 0 give FLT_MAX) runs original and rewritten shaders for nine bearings (six around the ring, up, down and an arbitrary direction): outputs not depending on the fog constants are bit-identical; every output equals the original with the fog constants aimed straight at the vertex; fog-only outputs agree across bearings (the original differs); nothing changes on a bearing's axis or at the eye. Covers synthetic shaders of every form Halo uses (dp4, dp3+.w, oFog, fog-plane texture coordinates, skinned with a0), the planeless-fog case, and malformed input. With the generated engine and .rdata present it takes the view constants from the translated 005176D0/00518F40 themselves for all three planar fog types; with vsh.bin present it checks all 64 shaders; it also checks MojoShader translates every rewritten shader, binds c4, keeps the same outputs and guards rsq(0). Missing inputs print SKIP.
test_radial_fog_renderer.m	yes (macOS)	Production renderer with real MojoShader: 25 pipelines; with the switch off the VS is byte-identical and the fixed fog source/key unchanged; unknown, no-temporary and matrix shaders fall back; fog varyings link; RGB-only blending; cache segregation and reuse; recipe round-trips; saved radial recipes are not queued while disabled; manifest v1 fields/keys preserved, newest duplicate kept, torn tail removed, legacy file untouched. --gpu additionally draws and reads back on the Mac GPU (24 draws, two bearings).
test_d3d9_radial_fog.c	yes (macOS)	The production FF3D bridge with three world vertices in nine bearings produces factors 1, 0.625 and 0 identically; colour and state preserved; default-off, flat window, FOGENABLE and table-fog gates; programmable mode selection (depth, type-2 view plane, no-fog shader).
test_radial_fog_settings.c	yes	Strict opt-in over 12 environment values; 16 concurrent first readers; the value stays cached after the environment changes.

Related pages

• Panorama System
• Shader Translation
• Metal Renderer
• Direct3D 9 Bridge
• Runtime Settings
• Layer Alignment