Layer normalization
Layer normalization normalizes activations across features within an example and applies learned scale and shift. Unlike batch normalization, its statistics do not require other examples in the mini-batch.
\operatorname{LN}(x)=\gamma\frac{x-\mu}{\sqrt{\sigma^2+\epsilon}}+\beta
Overview. Layer normalization normalizes activations across features within an example and applies learned scale and shift.
Context. Unlike batch normalization, its statistics do not require other examples in the mini-batch.
Claims and evidence
Claim ledger, 2Layer normalization normalizes activations across features within an example and applies learned scale and shift.
Unlike batch normalization, its statistics do not require other examples in the mini-batch.