Implement kernel PCA #74

Closed
rcurtin opened this Issue Dec 29, 2014 · 6 comments

Projects

None yet

1 participant

@rcurtin
Member
rcurtin commented Dec 29, 2014

Reported by rcurtin on 8 Mar 40826052 23:06 UTC
The previous implementation of kernel_pca (in mlpack/kernel_pca) requires sparse matrix support and actually does more than just kernel PCA. While this is nice, it does not fit into our new framework, and it would be less work to just reimplement it.

Here is an idea for the class (similar to #47):

PARAM_INT("num_pc", "Number of principal components to keep.", "kernel_pca");

template<typename Kernel>
class KernelPCA {
 public:
  KernelPCA(const arma::mat& data, Kernel kernel = Kernel());

  void Run(arma::mat& data_transformed);
  void Run(arma::mat& eigenvectors, arma::vec& eigenvalues);
  void Run(arma::mat& data_transformed, arma::mat& eigenvectors, arma::vec& eigenvalues);
};

and then a user can get their transformed data or just the eigenvalues and eigenvectors associated with the kernel covariance.

@rcurtin rcurtin self-assigned this Dec 29, 2014
@rcurtin rcurtin added this to the mlpack 1.0.1 milestone Dec 29, 2014
@rcurtin rcurtin closed this Dec 29, 2014
@rcurtin
Member
rcurtin commented Dec 29, 2014

Commented by rcurtin on 14 Feb 41684598 16:20 UTC
Because this is not compiled into MLPACK until sparse support is added, it is no longer blocking other tickets.

@rcurtin
Member
rcurtin commented Dec 29, 2014

Commented by rcurtin on 19 Jun 41738133 00:23 UTC
Ok, let's change the goal here. Instead of revamping what we've got, we'll start from scratch once we have PCA implemented (#126). No longer blocked by #46 because sparse matrix support is not necessary.

@rcurtin
Member
rcurtin commented Dec 29, 2014

Commented by rcurtin on 18 Jan 41738138 15:00 UTC
Const-correctness is important.

@rcurtin
Member
rcurtin commented Dec 29, 2014

Commented by rcurtin on 13 Aug 41905647 12:29 UTC
So we currently have PCA:

class PCA
{
 public:
  PCA();

  void Apply(const arma::mat& data, arma::mat& transformedData, arma::vec&
             eigVal, arma::mat& coeff);
  void Apply(const arma::mat& data, arma::mat& transformedData,
             arma::vec& eigVal);
  void Apply(arma::mat& data, const int newDimension);

  /**
   * Delete PCA object
   */
  ~PCA();

}; // class PCA

and we want Kernel PCA:

template<typename Kernel>
class KernelPCA
{
 public:
  KernelPCA(const Kernel& kernel);

  void Apply(const arma::mat& data, arma::mat& transformedData, arma::vec&
             eigVal, arma::mat& coeff);
  void Apply(const arma::mat& data, arma::mat& transformedData,
             arma::vec& eigVal);
  void Apply(arma::mat& data, const int newDimension);

  // Accessors / mutators.
  const Kernel& Kernel() const;
  Kernel& kernel();

  ~KernelPCA();

 private:
  Kernel& kernel;
}; // class KernelPCA

and in that setting, simple PCA is KernelPCA with the LinearKernel. But keep in mind that we can use template specialization to specify different (and faster) behavior for that case:

kernel_pca_impl.hpp:

template< >
void KernelPCA<LinearKernel>::Apply(..)
{
  // Now we just use what the old PCA class used.
}

Once all that is done, we can replace the current PCA class with KernelPCA, and then keep a typedef to simple PCA:

typedef KernelPCA<LinearKernel> PCA;
@rcurtin
Member
rcurtin commented Dec 29, 2014

Commented by rcurtin on 19 Feb 41961379 11:10 UTC
Working code and tests did not show up for 1.0 release.

@rcurtin
Member
rcurtin commented Dec 29, 2014

Commented by rcurtin on 11 Sep 42162213 16:46 UTC
Okay, I brushed up the executable for this and I think we can call it done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment