Locally Linear Latent Variable Model (LL-LVM) is a probabilistic model for non-linear manifold discovery that describes a joint distribution over observations, their manifold coordinates and locally linear maps conditioned on a set of neighbourhood relationships.
Mijung Park, Wittawat Jitkrittum, Ahmad Qamar, Zoltan Szabo, Lars Buesing, Maneesh Sahani "Bayesian Manifold Learning: The Locally Linear Latent Variable Model" NIPS, 2015
Running LL-LVM on a dataset is straightforward. For full demo script see, swissroll_demo.m.
% In Matlab, change directory into the code/ folder. % Run startup.m to add necessary paths. % % Assume that we are given a dataset as a dy x n matrix Y. % k: k in the k-nearest-neighbours graph construction k = 9; % Construct a neighbourhood graph with kNN. n x n matrix. G = makeKnnG(Y, k); % options to lllvm_1ep. Include initializations % Most options are optional. See lllvm_1ep file directly for possible options. % The only mandatory settings are G and dx. op = struct(); % The neighbourhood graph as an n x n matrix. op.G = G; % The desired reduced dimension. Say we want to reduce to 2 dimensions. op.dx = 2; % Call lllvm_1ep to run LL-LVM. % Relevant output variables are in the struct "results". [results, op ] = lllvm_1ep(Y, op); % In particular, results.mean_x is a dx*n x #iterations matrix. % To get the result x as a dx x n matrix, we can do x = reshape(results.mean_x(:, end), 2, );
Assuming that the data Y is a set of points forming a Swiss roll in the three-dimensional space. Here are what we obtain after running LL-LVM. These results can be obtained by running swissroll_demo.m.
Here is the set of local tangent planes learned by LL-LVM.
The following figure shows the evidence lower bounds during the EM, and the dimensionally reduced points in the two-dimensional space.
The unrolling will be cleaner with larger sample size.