Declaration of the WeightNorm layer class. More...

Public Member Functions
	WeightNorm (LayerTypes< CustomLayers...> layer=LayerTypes< CustomLayers...>())
	Create the WeightNorm layer object. More...

	~WeightNorm ()
	Destructor to release allocated memory. More...

template<typename eT >
void	Backward (const arma::Mat< eT > &&input, arma::Mat< eT > &&gy, arma::Mat< eT > &&g)
	Backward pass through the layer. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

template<typename eT >
void	Forward (arma::Mat< eT > &&input, arma::Mat< eT > &&output)
	Forward pass of the WeightNorm layer. More...

template<typename eT >
void	Gradient (arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&gradient)
	Calculate the gradient using the output delta, input activations and the weights of the wrapped layer. More...

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

LayerTypes< CustomLayers...> const &	Layer ()
	Get the wrapped layer. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()
	Reset the layer parameters. More...

template<typename Archive >
void	serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

Detailed Description

template<typename InputDataType, typename OutputDataType, typename... CustomLayers>
class mlpack::ann::WeightNorm< InputDataType, OutputDataType, CustomLayers >

Declaration of the WeightNorm layer class.

The layer reparameterizes the weight vectors in a neural network, decoupling the length of those weight vectors from their direction. This reparameterization does not introduce any dependencies between the examples in a mini-batch.

This class will be a wrapper around existing layers. It will just modify the calculation and updation of weights of the layer.

For more information, refer to the following paper,

* @inproceedings{Salimans2016WeightNorm,
*   title = {Weight Normalization: A Simple Reparameterization to Accelerate
*            Training of Deep Neural Networks},
*   author = {Tim Salimans, Diederik P. Kingma},
*   booktitle = {Neural Information Processing Systems 2016},
*   year = {2016}
* }
* 

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
CustomLayers	Additional custom layers that can be added.

Definition at line 176 of file layer_types.hpp.

Constructor & Destructor Documentation

WeightNorm ( LayerTypes< CustomLayers...> layer = LayerTypes< CustomLayers...>() )

Create the WeightNorm layer object.

Parameters

layer The layer whose weights are needed to be normalized.

~WeightNorm ( )

Destructor to release allocated memory.

Member Function Documentation

void Backward	(	const arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	gy,
		arma::Mat< eT > &&	g
	)

Backward pass through the layer.

This function calls the Backward() function of the wrapped layer.

Parameters

input	The input activations.
gy	The backpropagated error.
g	The calculated gradient.

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 117 of file weight_norm.hpp.

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 119 of file weight_norm.hpp.

void Forward	(	arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	output
	)

Forward pass of the WeightNorm layer.

Calculates the weights of the wrapped layer from the parameter vector v and the scalar parameter g. It then calulates the output of the wrapped layer from the calculated weights.

Parameters

input	Input data for the layer.
output	Resulting output activations.

void Gradient	(	arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	error,
		arma::Mat< eT > &&	gradient
	)

Calculate the gradient using the output delta, input activations and the weights of the wrapped layer.

Parameters

input	The input activations.
error	The calculated error.
gradient	The calculated gradient.

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 122 of file weight_norm.hpp.

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 124 of file weight_norm.hpp.

LayerTypes<CustomLayers...> const& Layer ( )

inline

Get the wrapped layer.

Definition at line 137 of file weight_norm.hpp.

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 127 of file weight_norm.hpp.

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 129 of file weight_norm.hpp.

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 132 of file weight_norm.hpp.

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 134 of file weight_norm.hpp.

void Reset ( )

Reset the layer parameters.

void serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the layer.

The documentation for this class was generated from the following files:

src/mlpack/methods/ann/layer/layer_types.hpp
src/mlpack/methods/ann/layer/weight_norm.hpp

Public Member Functions

Detailed Description

template<typename InputDataType, typename OutputDataType, typename... CustomLayers> class mlpack::ann::WeightNorm< InputDataType, OutputDataType, CustomLayers >

Constructor & Destructor Documentation

Member Function Documentation

template<typename InputDataType, typename OutputDataType, typename... CustomLayers>
class mlpack::ann::WeightNorm< InputDataType, OutputDataType, CustomLayers >