.. index:: pair: group; Batch Normalization
.. _doxid-group__dnnl__api__batch__normalization:

Batch Normalization
===================

.. toctree::
	:hidden:

	struct_dnnl_batch_normalization_backward.rst
	struct_dnnl_batch_normalization_forward.rst
	struct_dnnl_batch_normalization_desc_t.rst

Overview
~~~~~~~~

A primitive to perform batch normalization. :ref:`More...<details-group__dnnl__api__batch__normalization>`


.. ref-code-block:: cpp
	:class: doxyrest-overview-code-block

	
	// structs

	struct :ref:`dnnl::batch_normalization_backward<doxid-structdnnl_1_1batch__normalization__backward>`;
	struct :ref:`dnnl::batch_normalization_forward<doxid-structdnnl_1_1batch__normalization__forward>`;
	struct :ref:`dnnl_batch_normalization_desc_t<doxid-structdnnl__batch__normalization__desc__t>`;

	// global functions

	:ref:`dnnl_status_t<doxid-group__dnnl__api__utils_1gad24f9ded06e34d3ee71e7fc4b408d57a>` DNNL_API :ref:`dnnl_batch_normalization_forward_desc_init<doxid-group__dnnl__api__batch__normalization_1ga91d12ab341b92fea39ea123ea6c4d578>`(
		:ref:`dnnl_batch_normalization_desc_t<doxid-structdnnl__batch__normalization__desc__t>`* bnrm_desc,
		:ref:`dnnl_prop_kind_t<doxid-group__dnnl__api__primitives__common_1gae3c1f22ae55645782923fbfd8b07d0c4>` prop_kind,
		const :ref:`dnnl_memory_desc_t<doxid-structdnnl__memory__desc__t>`* data_desc,
		float epsilon,
		unsigned flags
		);

	:ref:`dnnl_status_t<doxid-group__dnnl__api__utils_1gad24f9ded06e34d3ee71e7fc4b408d57a>` DNNL_API :ref:`dnnl_batch_normalization_backward_desc_init<doxid-group__dnnl__api__batch__normalization_1ga5ce82f890ff9b95e4c61c63f95819e7a>`(
		:ref:`dnnl_batch_normalization_desc_t<doxid-structdnnl__batch__normalization__desc__t>`* bnrm_desc,
		:ref:`dnnl_prop_kind_t<doxid-group__dnnl__api__primitives__common_1gae3c1f22ae55645782923fbfd8b07d0c4>` prop_kind,
		const :ref:`dnnl_memory_desc_t<doxid-structdnnl__memory__desc__t>`* diff_data_desc,
		const :ref:`dnnl_memory_desc_t<doxid-structdnnl__memory__desc__t>`* data_desc,
		float epsilon,
		unsigned flags
		);

.. _details-group__dnnl__api__batch__normalization:

Detailed Documentation
~~~~~~~~~~~~~~~~~~~~~~

A primitive to perform batch normalization.

Both forward and backward propagation primitives support in-place operation; that is, src and dst can refer to the same memory for forward propagation, and diff_dst and diff_src can refer to the same memory for backward propagation.

The batch normalization primitives computations can be controlled by specifying different :ref:`dnnl::normalization_flags <doxid-group__dnnl__api__primitives__common_1gad8ef0fcbb7b10cae3d67dd46892002be>` values. For example, batch normalization can compute the mean and variance on its own or take them as inputs. It can either perform scaling and shifting using gamma and beta parameters or not. Optionally, it can also perform a fused ReLU, which in case of training would also require a workspace.



.. rubric:: See also:

:ref:`Batch Normalization <doxid-dev_guide_batch_normalization>` in developer guide

Global Functions
----------------

.. index:: pair: function; dnnl_batch_normalization_forward_desc_init
.. _doxid-group__dnnl__api__batch__normalization_1ga91d12ab341b92fea39ea123ea6c4d578:

.. ref-code-block:: cpp
	:class: doxyrest-title-code-block

	:ref:`dnnl_status_t<doxid-group__dnnl__api__utils_1gad24f9ded06e34d3ee71e7fc4b408d57a>` DNNL_API dnnl_batch_normalization_forward_desc_init(
		:ref:`dnnl_batch_normalization_desc_t<doxid-structdnnl__batch__normalization__desc__t>`* bnrm_desc,
		:ref:`dnnl_prop_kind_t<doxid-group__dnnl__api__primitives__common_1gae3c1f22ae55645782923fbfd8b07d0c4>` prop_kind,
		const :ref:`dnnl_memory_desc_t<doxid-structdnnl__memory__desc__t>`* data_desc,
		float epsilon,
		unsigned flags
		)

Initializes a descriptor for a batch normalization forward propagation primitive.

.. note:: 

   In-place operation is supported: the dst can refer to the same memory as the src.



.. rubric:: Parameters:

.. list-table::
	:widths: 20 80

	*
		- bnrm_desc

		- Output descriptor for batch normalization primitive.

	*
		- prop_kind

		- Propagation kind. Possible values are :ref:`dnnl_forward_training <doxid-group__dnnl__api__primitives__common_1ggae3c1f22ae55645782923fbfd8b07d0c4a992e03bebfe623ac876b3636333bbce0>` and :ref:`dnnl_forward_inference <doxid-group__dnnl__api__primitives__common_1ggae3c1f22ae55645782923fbfd8b07d0c4a2f77a568a675dec649eb0450c997856d>`.

	*
		- data_desc

		- Source and destination memory descriptor.

	*
		- epsilon

		- Batch normalization epsilon parameter.

	*
		- flags

		- Batch normalization flags (:ref:`dnnl_normalization_flags_t <doxid-group__dnnl__api__primitives__common_1ga301f673522a400c7c1e75f518431c9a3>`).



.. rubric:: Returns:

:ref:`dnnl_success <doxid-group__dnnl__api__utils_1ggad24f9ded06e34d3ee71e7fc4b408d57aaa31395e9dccc103cf166cf7b38fc5b9c>` on success and a status describing the error otherwise.

.. index:: pair: function; dnnl_batch_normalization_backward_desc_init
.. _doxid-group__dnnl__api__batch__normalization_1ga5ce82f890ff9b95e4c61c63f95819e7a:

.. ref-code-block:: cpp
	:class: doxyrest-title-code-block

	:ref:`dnnl_status_t<doxid-group__dnnl__api__utils_1gad24f9ded06e34d3ee71e7fc4b408d57a>` DNNL_API dnnl_batch_normalization_backward_desc_init(
		:ref:`dnnl_batch_normalization_desc_t<doxid-structdnnl__batch__normalization__desc__t>`* bnrm_desc,
		:ref:`dnnl_prop_kind_t<doxid-group__dnnl__api__primitives__common_1gae3c1f22ae55645782923fbfd8b07d0c4>` prop_kind,
		const :ref:`dnnl_memory_desc_t<doxid-structdnnl__memory__desc__t>`* diff_data_desc,
		const :ref:`dnnl_memory_desc_t<doxid-structdnnl__memory__desc__t>`* data_desc,
		float epsilon,
		unsigned flags
		)

Initializes a descriptor for a batch normalization backward propagation primitive.

.. note:: 

   In-place operation is supported: the diff_dst can refer to the same memory as the diff_src.



.. rubric:: Parameters:

.. list-table::
	:widths: 20 80

	*
		- bnrm_desc

		- Output descriptor for batch normalization primitive.

	*
		- prop_kind

		- Propagation kind. Possible values are :ref:`dnnl_backward_data <doxid-group__dnnl__api__primitives__common_1ggae3c1f22ae55645782923fbfd8b07d0c4a524dd6cb2ed9680bbd170ba15261d218>` and :ref:`dnnl_backward <doxid-group__dnnl__api__primitives__common_1ggae3c1f22ae55645782923fbfd8b07d0c4a326a5e31769302972e7bded555e1cc10>` (diffs for all parameters are computed in this case).

	*
		- diff_data_desc

		- Diff source and diff destination memory descriptor.

	*
		- data_desc

		- Source memory descriptor.

	*
		- epsilon

		- Batch normalization epsilon parameter.

	*
		- flags

		- Batch normalization flags (:ref:`dnnl_normalization_flags_t <doxid-group__dnnl__api__primitives__common_1ga301f673522a400c7c1e75f518431c9a3>`).



.. rubric:: Returns:

:ref:`dnnl_success <doxid-group__dnnl__api__utils_1ggad24f9ded06e34d3ee71e7fc4b408d57aaa31395e9dccc103cf166cf7b38fc5b9c>` on success and a status describing the error otherwise.