BatchNormForwardTraining¶
General¶
BatchNormForwardTraining operation performs batch normalization at training mode.
Mean and variance are computed at runtime, the following formulas are used:
\(\mu(c) = \frac{1}{NHW} \sum\limits_{nhw} \src(n, c, h, w)_{}\),
\(\sigma^2(c) = \frac{1}{NHW} \sum\limits_{nhw} {}_{} (\src(n, c, h, w) - \mu(c))^2\).
Operation attributes¶
Attribute Name |
De |
|||
---|---|---|---|---|
A number to be added to the variance to avoid division by zero. |
f32 |
A positive f32 value |
Required |
|
A number to be used to calculate running mean and running variance. |
f32 |
A positive f32 value |
Optional |
|
Controls how to interpret the shape of |
string |
|
Optional |
Execution arguments¶
The inputs and outputs must be provided according to below index order when constructing an operation.
Inputs¶
Index |
Argu |
|
---|---|---|
0 |
|
Required |
1 |
|
Required |
2 |
|
Required |
3 |
|
Optional |
4 |
|
Optional |
Note
gamma
and beta
should be either both provided or neither provided.
Outputs¶
Index |
Argu |
|
---|---|---|
0 |
|
Required |
1 |
|
Required |
2 |
|
Required |
3 |
|
Required |
4 |
|
Required |
Supported data types¶
BatchNormInference operation supports the following data type combinations.
Src / |
|
---|---|
f32 |
f32 |
bf16 |
f32, bf16 |
f16 |
f32 |