Zhen

Local Response Normalization (LRN) in different frameworks.

2018-09-19

Local Response Normalization (LRN) has different implementations in different Frameworks.

Caffe

Document

  • Parameters (LRNParameter lrn_param) Optional

    • local_size [default 5]: the number of channels to sum over (for cross channel LRN) or the side length of the square region to sum over (for within channel LRN)
    • alpha [default 1]: the scaling parameter (see below)
    • beta [default 5]: the exponent (see below)
    • norm_region [default ACROSS_CHANNELS]: whether to sum over adjacent channels (ACROSS_CHANNELS) or nearby spatial locations (WITHIN_CHANNEL)
  • 计算公式

\[b_{x,y}^i = a_{x,y}^i \left(1 + \frac{\alpha}{n} \sum_{j=\max(0, i-n/2)}^{\min(N-1,i+n/2)}a_{x,y}^2\right)^{-\beta}\]

TensorFlow

different from Caffe version. refer to this post.

  • 计算公式
\[b_{x,y}^i = a_{x,y}^i \left(b + \alpha \sum_{j=\max(0, i-r)}^{\min(N-1,i+r)}a_{x,y}^2\right)^{-\beta}\]

PyTorch

Document

same as Caffe version, the parameters can be used directly.

  • 计算公式
\[b_{c} = a_{c}\left(k + \frac{\alpha}{n} \sum_{c'=\max(0, c-n/2)}^{\min(N-1,c+n/2)}a_{c'}^2\right)^{-\beta}\]

Content