# Local Response Normalization (LRN) in different frameworks.

2018-09-19

Local Response Normalization (LRN) has different implementations in different Frameworks.

### Caffe

Document

• Parameters (LRNParameter lrn_param) Optional

• local_size [default 5]: the number of channels to sum over (for cross channel LRN) or the side length of the square region to sum over (for within channel LRN)
• alpha [default 1]: the scaling parameter (see below)
• beta [default 5]: the exponent (see below)
• norm_region [default ACROSS_CHANNELS]: whether to sum over adjacent channels (ACROSS_CHANNELS) or nearby spatial locations (WITHIN_CHANNEL)
• 计算公式

$b_{x,y}^i = a_{x,y}^i \left(1 + \frac{\alpha}{n} \sum_{j=\max(0, i-n/2)}^{\min(N-1,i+n/2)}a_{x,y}^2\right)^{-\beta}$

### TensorFlow

different from Caffe version. refer to this post.

• 计算公式
$b_{x,y}^i = a_{x,y}^i \left(b + \alpha \sum_{j=\max(0, i-r)}^{\min(N-1,i+r)}a_{x,y}^2\right)^{-\beta}$

### PyTorch

Document

same as Caffe version, the parameters can be used directly.

• 计算公式
$b_{c} = a_{c}\left(k + \frac{\alpha}{n} \sum_{c'=\max(0, c-n/2)}^{\min(N-1,c+n/2)}a_{c'}^2\right)^{-\beta}$

Content