An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom. More...
Public Member Functions | |
| LocalCoordinateCoding (const arma::mat &data, const size_t atoms, const double lambda) | |
| Set the parameters to LocalCoordinateCoding. | |
| arma::mat & | Codes () |
| Modify the codes. | |
| const arma::mat & | Codes () const |
| Accessor the codes. | |
| const arma::mat & | Data () const |
| Access the data. | |
| arma::mat & | Dictionary () |
| Mutator for dictionary. | |
| const arma::mat & | Dictionary () const |
| Accessor for dictionary. | |
| void | Encode (const size_t maxIterations=0, const double objTolerance=0.01) |
| Run local coordinate coding. | |
| double | Objective (arma::uvec adjacencies) const |
| Compute objective function given the list of adjacencies. | |
| void | OptimizeCode () |
| Code each point via distance-weighted LARS. | |
| void | OptimizeDictionary (arma::uvec adjacencies) |
| Learn dictionary by solving linear system. | |
Private Attributes | |
| size_t | atoms |
| Number of atoms in dictionary. | |
| arma::mat | codes |
| Codes (columns are points). | |
| const arma::mat & | data |
| Data matrix (columns are points). | |
| arma::mat | dictionary |
| Dictionary (columns are atoms). | |
| double | lambda |
| l1 regularization term. | |
An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.
Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective: min_{D,Z} ||X - D Z||_{Fro}^2 + lambda sum_{i=1}^m sum_{j=1}^k dist(X_i,D_j)^2 Z_i^j where lambda > 0.
This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D by solving a linear system (note that the objective is a positive definite quadratic program). The sparse coding step involves solving a large number of weighted l1-norm regularized linear regression problems problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO (paper below).
The papers are listed below.
@incollection{NIPS2009_0719,
title = {Nonlinear Learning using Local Coordinate Coding},
author = {Kai Yu and Tong Zhang and Yihong Gong},
booktitle = {Advances in Neural Information Processing Systems 22},
editor = {Y. Bengio and D. Schuurmans and J. Lafferty and C. K. I. Williams
and A. Culotta},
pages = {2223--2231},
year = {2009}
}
@article{efron2004least,
title={Least angle regression},
author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.},
journal={The Annals of statistics},
volume={32},
number={2},
pages={407--499},
year={2004},
publisher={Institute of Mathematical Statistics}
}
Definition at line 91 of file lcc.hpp.
| mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::LocalCoordinateCoding | ( | const arma::mat & | data, | |
| const size_t | atoms, | |||
| const double | lambda | |||
| ) |
Set the parameters to LocalCoordinateCoding.
| data | Data matrix. | |
| atoms | Number of atoms in dictionary. | |
| lambda | Regularization parameter for weighted l1-norm penalty. |
| arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes | ( | ) | [inline] |
Modify the codes.
Definition at line 146 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes.
| const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes | ( | ) | const [inline] |
Accessor the codes.
Definition at line 144 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes.
| const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Data | ( | ) | const [inline] |
| arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary | ( | ) | [inline] |
Mutator for dictionary.
Definition at line 141 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary.
| const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary | ( | ) | const [inline] |
Accessor for dictionary.
Definition at line 139 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary.
| void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Encode | ( | const size_t | maxIterations = 0, |
|
| const double | objTolerance = 0.01 | |||
| ) |
Run local coordinate coding.
| nIterations | Maximum number of iterations to run algorithm. | |
| objTolerance | Tolerance of objective function. When the objective function changes by a value lower than this tolerance, the optimization terminates. |
| double mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Objective | ( | arma::uvec | adjacencies | ) | const |
Compute objective function given the list of adjacencies.
| void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::OptimizeCode | ( | ) |
Code each point via distance-weighted LARS.
| void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::OptimizeDictionary | ( | arma::uvec | adjacencies | ) |
Learn dictionary by solving linear system.
| adjacencies | Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms) |
size_t mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::atoms [private] |
arma::mat mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes [private] |
Codes (columns are points).
Definition at line 159 of file lcc.hpp.
Referenced by mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes().
const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::data [private] |
arma::mat mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary [private] |
Dictionary (columns are atoms).
Definition at line 156 of file lcc.hpp.
Referenced by mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary().
double mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::lambda [private] |
1.6.1