Certifiable Active Inference - Probing the Identifiability Wall

[DanBoringName]

This is the first research programme built on cpomdp and jumps in at the deep end to see if it's really worth its salt. The library itself is a general toolbox with no thesis of its own, so treat the following as an example of the kind of questions it makes accessible to Active Inference research.

---

The Koudahl Collapse

If you take the textbook linear-Gaussian agent: Gaussian belief, Kalman recursion, expected free energy over some action set; and observe the epistemic term you'll notice it does absolutely nothing. This is what Koudahl and his team found, and what this project conveniently calls the Koudahl collapse (with Koudahl's approval for the name). That is to say every policy has the same epistemic value, therefore no preference based on information gain, and collapses into a classical LQR controller.

Meaning a lot of what makes active inference active inference is unavailable by construction in the logical first setting for building these continuous agents - me a few months ago for example.

cpomdp treats state-dependent noise as an ordinary model component, and that way epistemics is revived, loyal to the active inference equations as a minimal change.

---

The Catalyst

Paper 1 (complete, archived, pins cpomdp v0.4.2) proves the state dependence properly. Once observation noise depends on the state, R(x), no fixed sensor model reproduces the agent. Within its stated hypothesis class there's no linear-Gaussian reduction to fall back on and the epistemic value becomes genuinely policy dependent.

That opened a second question.

Standard free energy accounting makes a move usually called the ergodic identification: the process generating the data gets identified with the agent's model of it, p* = p. It's rarely argued for, mostly it just seems to happen (from what my research turned up), and it's what allows free energy to be reported as a single number. Under it, variational free energy scored against the true process collapses to the entropy of sensory states, and minimising surprise becomes one scalar.

Refuse the identification. Hold p* and p apart, score the agent's free energy under the true process, and one term becomes three:

E_p*(·|u)[F]  =  H(p*(·|u))  +  D_KL[ p*(y|u) ‖ p(y|u) ]  +  E_p*(·|u)[ D_KL[ q(x) ‖ p(x|y) ] ]
                  ⌊ floor ⌋      ⌊  misspecification  ⌋       ⌊       inference gap       ⌋

The floor is the irreducible entropy of the world. It's a property of p* and no policy touches it. The misspecification term is zero if and only if p = p*. It's a measurement of how wrong the agent's model is compared to the true world model. The inference gap is zero if and only if q is the exact posterior. It measures how badly the agent is at executing its own model.

Two agents can score identically under the one-term version while failing for completely unrelated reasons only visible in the three-term decomposition.

---

Where "certifiable" comes from

There's a second problem that you will have come across if you've been around active inference for a little bit.

Free-energy minimisation is to a defined relative to a generative model and a preference distribution, and both are free. Given behaviour alone you can't recover either. Give an agent a belief and a compensating preference and it will behave identically.

Say you watch a car holding 60 on the motorway. How would you explain that behaviour? Perhaps it's a driver who thinks the limit is 60 and doesn't want to break the limit. Perhaps it's a speed demon stuck behind a slower moving lorry. Maybe they have a microchip implanted at the base of their skull that will explode if they don't stay driving at exactly 60. All three result in the same speed reading and no amount of external observation separates them. That's the identifiability wall.

The underdetermination itself is old news, the inverse-RL degeneracy in different clothes. What the programme adds is calling it a gauge freedom: a transformation group acting on the pair (model, preferences) that leaves behaviour invariant. Gauge freedoms come with three standard responses. Fix the gauge by convention, work only with gauge-invariant quantities, or find out the freedom was physical all along.

Now going back to the three-term decomposition. None of them contain a preference. Preferences enter the expected free energy that scores policies, not the variational free energy that scores inference. So, the decomposition is invariant under the preference half of the gauge group, which means it measures the part of an agent that isn't hidden behind the worst of the problem. The wall isn't uniform, and treating it as one flat impossibility gives away more than the maths requires.

v0.4.5 through v0.5 are the library work this needs.

This is a claim about accounting, and about what a certified instrument can and can't separate. It is not a claim that anything has been refuted. The Free Energy Principle isn't the sort of object a library can test, and every result here concerns the machinery around it.

---

Roadmap

	Status	Pins	What it establishes
Paper 1	✅ Complete	v0.4.2	No fixed sensor reproduces an R(x) agent; epistemic value is policy-dependent
Paper 2, Pt 1	🔨 Drafting	v0.5	The three terms separate, and separate attributability
Paper 2, Pt 2	⛔ Gated	v0.5	The same numbers, certified against a reference filter with a stated bound
Paper 3	📋 Scoped	v0.6	Do the action-selection functionals in the literature actually differ, or coincide?
Paper 4	🔒 Blocked	v0.6	What the instrument establishes, and where measurability stops

The gate on Part 2 is a certified discretisation bound for a grid-based reference filter. It's the hardest gate on the roadmap and it might not land in the form I would ideally want. If it doesn't, I'll say so, and those numbers stay labelled computed rather than certified, and distinction the library already enforces in its own output.

One preliminary number is banked, from the multi-step enumerator that shipped in v0.4.4. Sweeping the planning horizon, epistemic pull overtakes the pragmatic gradient at H* = 7. Exhaustive over the declared action set {0, ±1, ±2}, certified complete at each horizon, and stated as an upper bound because the grid clips how far the agent can reach.

---

Threads

Opened as separate comments so they can be argued independently.

1. Where does state-dependent sensing turn up in your models? Range, angle, saturation, occlusion? A real application that research needs is better than me guessing.
2. Does p ≠ p* matter in your work? If you've ever scored a model against data it didn't generate, you've hit this. Where did it bite?
3. Which action-selection functional do you actually use, and why that one? Writing these down exactly, including the places sources contradict themselves, is the current bottleneck.
4. Priority check, and I'd like to be scooped. Has anyone stated the model/preference underdetermination as a gauge symmetry, in those terms or equivalent ones? Has anyone noticed that variational free energy scored against a true process is preference-free while expected free energy isn't, and drawn the identifiability consequence? "Already known, see X" is a better outcome for me than novelty, and much cheaper now than at review.
5. Numerical analysts, hello. The reference filter with a certified bound is the hard item. If quadrature and grid-based Bayesian filtering are your thing, this is the highest-value place to help and it needs no active inference at all.
6. What would make you actually use this? Missing features, missing docs, an interface that fights you. Be blunt, I can take it.

---

Contributing

Whilst the Certifiable Active Inference (CAIF) programme isn't looking for direct hands-on help, comments, review, feedback are all requested.

cpomdp is very much asking for hands-on help. Pick an issue, create one yourself, docs, examples. Read DECISIONS.md and CONTRIBUTING.md to ground yourself in the material if you wish. Open an issue before a PR, and I'm happy to coach a first contribution.

If you want to reach me. The Active Inference Institute Discord should be your first bet. I'm an active member there with cpomdp acquiring its own project section soon. Alternatively, email me at dan@inferogenesis.com.

[DanBoringName]

1) Where does state-dependent sensing turn up in your models?

Range, angle, saturation, occlusion? A real application that research needs is better than me guessing.

[DanBoringName]

2) Does p ≠ p* matter in your work?

If you've ever scored a model against data it didn't generate, you've hit this. Where did it bite?

[DanBoringName]

3) Which action-selection functional do you actually use, and why that one?

Writing these down exactly, including the places sources contradict themselves, is the current bottleneck.

[DanBoringName]

4) Priority check, and I'd like to be scooped.

Has anyone stated the model/preference underdetermination as a gauge symmetry, in those terms or equivalent ones? Has anyone noticed that variational free energy scored against a true process is preference-free while expected free energy isn't, and drawn the identifiability consequence?

[DanBoringName]

5) Numerical analysts, hello.

The reference filter with a certified bound is the hard item. If quadrature and grid-based Bayesian filtering are your thing, this is the highest-value place to help and it needs no active inference at all.

[DanBoringName]

6. What would make you actually use this?
   Missing features, missing docs, an interface that fights you. Be blunt, I can take it.