Ordinal factor analysis model

[wjn0]

I'm hoping to implement a (probit) ordinal factor analysis model in Infer.NET. There were a few references on the old help board (from almost 10 years ago - see here) to the problem that were helpful. This is my model so far:

            // MODEL
            // Latent dimensions
            VariableArray2D<double> vX = Variable.Array<double>(rN, rP).Named("X");
            vX[rN, rP] = Variable.GaussianFromMeanAndPrecision(0.0, 1.0).ForEach(rN, rP);
            // Question weights
            VariableArray2D<double> vC = Variable.Array<double>(rP, rQ).Named("C");
            vC[rP, rQ] = Variable.GaussianFromMeanAndPrecision(0.0, 1.0).ForEach(rP, rQ);
            // Question 'errors'
            VariableArray<double> vR = Variable.Array<double>(rQ).Named("R");
            vR[rQ] = Variable.GammaFromShapeAndScale(1, 1).ForEach(rQ);
            // Continuous question states
            VariableArray2D<double> vY = Variable.Array<double>(rN, rQ).Named("Y");
            vY = Variable.MatrixMultiply(vX, vC);
            VariableArray2D<double> vYY = Variable.Array<double>(rN, rQ).Named("YY");
            vYY[rN, rQ] = Variable.GaussianFromMeanAndPrecision(vY[rN, rQ], vR[rQ]);
            // Thresholds
            VariableArray2D<double> lT = Variable.Array<double>(rQ, rR).Named("LT");
            VariableArray2D<double> uT = Variable.Array<double>(rQ, rR).Named("UT");
            // Observed question values
            var vZ = Variable.Array(Variable.Array(Variable.Array<bool>(rR), rQ), rN);
            vZ[rN][rQ][rR] = Variable.IsBetween(vYY[rN, rQ], lT[rQ, rR], uT[rQ, rR]);

            // INFERENCE
            InferenceEngine engine = new InferenceEngine();
            engine.Algorithm = new VariationalMessagePassing();
            lT.ObservedValue = lowerThresholds;
            uT.ObservedValue = upperThresholds;
            vZ.ObservedValue = observed;

            var XInferred = engine.Infer(vX);

The part that seems to be giving the inference engine trouble is the Variable.IsBetween. It errors with: TruncatedGaussian requires isBetween=true. I have a hunch this might be a limitation of the VMP implementation (to do with non-conjugacy at the IsBetween node?) - I had hoped to use EP to fit the model, but the MatrixMultiply factor doesn't seem to be supported. Any pointers would be much appreciated! And sorry if any of this is basic, I'm very new to both Infer.NET and .NET in general. Thanks for a great piece of software.

[tminka]

The issue is that you are using Variable.IsBetween on data where the output is sometimes false. This leads to a weird constraint that requires vYY to not be between two bounds. Do you actually need this for ordinal regression? I would think that each observation in ordinal regression asserts that vYY is between two bounds.

[tminka]

By the way, the matchbox recommendation model is similar to ordinal factor analysis.

[wjn0]

Ah, of course, that makes perfect sense. That was a simple enough fix. I'd like to constrain the weights vectors to be unit length, but it seems that the two more obvious methods for doing so (specifically, taking the inner product with itself or sum of squares and constraining it to be unit 1) don't work. Is this possible under VMP or EP?

[wjn0]

Thanks much for the prompt reply, by the way. I'll take a look at the matchbox recommendation model.

[wjn0]

P.S. One more question - is there a straightforward way to constrain an array to be ordered (i.e. for learning the thresholds)? It seems like there should be a way to string together Variable.ConstrainPositive calls to do so but I haven't quite grasped the range syntax well enough yet.

[tminka]

It is theoretically possible to have a unit length constraint in VMP or EP. A vector of unit length is equivalent to a point on a hyper-sphere, so the posterior would be a distribution over the hyper-sphere, and various approximate distributions could be used. However, they are not yet implemented in Infer.NET.

You can indeed string together ConstrainPositive calls to constrain an array to be ordered, but you don't need to do this for ordinal regression, since the IsBetween constraints already do it. See the matchbox model.

Since your original question was answered, I'm closing this issue.