.. index:: pair: struct; dnnl::primitive_attr .. _doxid-structdnnl_1_1primitive__attr: struct dnnl::primitive_attr =========================== .. toctree:: :hidden: Overview ~~~~~~~~ Primitive attributes. :ref:`More...` .. ref-code-block:: cpp :class: doxyrest-overview-code-block #include struct primitive_attr: public :ref:`dnnl::handle` { // construction :ref:`primitive_attr`(); :ref:`primitive_attr`(:ref:`dnnl_primitive_attr_t` attr); // methods :ref:`fpmath_mode` :ref:`get_fpmath_mode`() const; void :ref:`get_fpmath_mode`(:ref:`fpmath_mode`& mode, bool& apply_to_int) const; void :ref:`set_fpmath_mode`(:ref:`fpmath_mode` mode, bool apply_to_int = false); :ref:`accumulation_mode` :ref:`get_accumulation_mode`() const; void :ref:`set_accumulation_mode`(:ref:`accumulation_mode` mode); bool :ref:`get_deterministic`() const; void :ref:`set_deterministic`(bool value); :ref:`scratchpad_mode` :ref:`get_scratchpad_mode`() const; void :ref:`set_scratchpad_mode`(:ref:`scratchpad_mode` mode); void :ref:`set_scales_mask`(int arg, int mask); void :ref:`set_scales`( int arg, int mask, const :ref:`memory::dims`& groups, :ref:`memory::data_type` data_type = :ref:`memory::data_type::f32` ); void :ref:`set_zero_points_mask`(int arg, int mask); void :ref:`set_zero_points`( int arg, int mask, const :ref:`memory::dims`& groups, :ref:`memory::data_type` data_type = :ref:`memory::data_type::s32` ); const :ref:`post_ops` :ref:`get_post_ops`() const; void :ref:`set_post_ops`(const :ref:`post_ops` ops); void :ref:`set_rnn_data_qparams`(float scale, float shift); void :ref:`get_rnn_data_qparams`(float& scale, float& shift); void :ref:`set_rnn_weights_qparams`(int mask, const std::vector& scales); void :ref:`get_rnn_weights_qparams`(int& mask, std::vector& scales); void :ref:`set_rnn_weights_projection_qparams`( int mask, const std::vector& scales ); void :ref:`get_rnn_weights_projection_qparams`(int& mask, std::vector& scales); }; Inherited Members ----------------- .. ref-code-block:: cpp :class: doxyrest-overview-inherited-code-block public: // methods :ref:`handle`& :ref:`operator =` (const :ref:`handle`&); :ref:`handle`& :ref:`operator =` (:ref:`handle`&&); void :ref:`reset`(T t, bool weak = false); T :ref:`get`(bool allow_empty = false) const; :ref:`operator T` () const; :ref:`operator bool` () const; bool :ref:`operator ==` (const :ref:`handle`& other) const; bool :ref:`operator !=` (const :ref:`handle`& other) const; .. _details-structdnnl_1_1primitive__attr: Detailed Documentation ~~~~~~~~~~~~~~~~~~~~~~ Primitive attributes. .. rubric:: See also: :ref:`Primitive Attributes ` Construction ------------ .. index:: pair: function; primitive_attr .. _doxid-structdnnl_1_1primitive__attr_1acfbfd85b7ca82bf97e2b07c2427427de: .. ref-code-block:: cpp :class: doxyrest-title-code-block primitive_attr() Constructs default (empty) primitive attributes. .. index:: pair: function; primitive_attr .. _doxid-structdnnl_1_1primitive__attr_1aafb54e73f3abe59555f1cfe62407280e: .. ref-code-block:: cpp :class: doxyrest-title-code-block primitive_attr(:ref:`dnnl_primitive_attr_t` attr) Creates primitive attributes from a C API :ref:`dnnl_primitive_attr_t ` handle. The resulting handle is not weak and the C handle will be destroyed during the destruction of the C++ object. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - attr - The C API primitive attributes. Methods ------- .. index:: pair: function; get_fpmath_mode .. _doxid-structdnnl_1_1primitive__attr_1af335f8e1e74b69c5b2d4da0ecf8a23d5: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`fpmath_mode` get_fpmath_mode() const Returns the fpmath mode. .. index:: pair: function; get_fpmath_mode .. _doxid-structdnnl_1_1primitive__attr_1a4c5566628d4c893a321a8f053a5cc5fd: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_fpmath_mode(:ref:`fpmath_mode`& mode, bool& apply_to_int) const Returns the fpmath mode. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mode - Specified fpmath mode. * - apply_to_int - Use floating-point arithmetic for integer primitives. .. index:: pair: function; set_fpmath_mode .. _doxid-structdnnl_1_1primitive__attr_1ab00639157a283596834ee5b0e8478a2d: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_fpmath_mode(:ref:`fpmath_mode` mode, bool apply_to_int = false) Sets fpmath mode. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mode - Specified fpmath mode. * - apply_to_int - Boolean. Use of floating-point arithmetic for integer primitives. .. index:: pair: function; get_accumulation_mode .. _doxid-structdnnl_1_1primitive__attr_1a054e1d0097e9e85411cfb539bd45e247: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`accumulation_mode` get_accumulation_mode() const Returns the accumulation mode. .. index:: pair: function; set_accumulation_mode .. _doxid-structdnnl_1_1primitive__attr_1a8348fcd2259553c3537194430b7de4f4: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_accumulation_mode(:ref:`accumulation_mode` mode) Sets accumulation mode. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mode - Specified accumulation mode. .. index:: pair: function; get_deterministic .. _doxid-structdnnl_1_1primitive__attr_1a29e003998b46762e5fa0532844ca12c4: .. ref-code-block:: cpp :class: doxyrest-title-code-block bool get_deterministic() const Returns the deterministic attribute value. .. index:: pair: function; set_deterministic .. _doxid-structdnnl_1_1primitive__attr_1ad4c1ea7156ce31cd231a00d4e3399add: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_deterministic(bool value) Sets deterministic attribute value. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - value - Specified deterministic mode. .. index:: pair: function; get_scratchpad_mode .. _doxid-structdnnl_1_1primitive__attr_1af4131b946ec3af3bc2974b603d30029b: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`scratchpad_mode` get_scratchpad_mode() const Returns the scratchpad mode. .. index:: pair: function; set_scratchpad_mode .. _doxid-structdnnl_1_1primitive__attr_1a91a597649afa13b7d2416b708d0620d2: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_scratchpad_mode(:ref:`scratchpad_mode` mode) Sets scratchpad mode. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mode - Specified scratchpad mode. .. index:: pair: function; set_scales_mask .. _doxid-structdnnl_1_1primitive__attr_1ac3dc9efa6702a5eba6f289f1b3907590: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_scales_mask(int arg, int mask) Sets scaling factors for primitive operations for a given memory argument. The scaling factors must be passed at execution time as an argument with index :ref:`DNNL_ARG_ATTR_SCALES ` \| arg. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Scaling factors correspondence mask that defines the correspondence between the tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor is used for each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. .. rubric:: See also: :ref:`dnnl_primitive_attr_set_scales_mask ` .. index:: pair: function; set_scales .. _doxid-structdnnl_1_1primitive__attr_1a29e8f33119d42bf7d259eafc6e6548d6: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_scales( int arg, int mask, const :ref:`memory::dims`& groups, :ref:`memory::data_type` data_type = :ref:`memory::data_type::f32` ) Sets scaling factors for primitive operations for a given memory argument. The scaling factors must be passed at execution time as an argument with index :ref:`DNNL_ARG_ATTR_SCALES ` \| arg. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Scales correspondence mask that defines the correspondence between the tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scale is used for each index along that dimension. Set the mask to 0 to use a common scale for the whole output tensor. * - groups - Scaling factors correspondence groups that define the correspondence between the tensor dimensions and the scales array. The set i-th dimension indicates a number of groups of scaling factors used for that logical dimension in a memory indicated by ``arg``. * - data_type - Scaling factors data_type. .. rubric:: See also: :ref:`dnnl_primitive_attr_set_scales ` .. index:: pair: function; set_zero_points_mask .. _doxid-structdnnl_1_1primitive__attr_1a8935d36d48fe5db9476b30b02791d822: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_zero_points_mask(int arg, int mask) Sets zero points for primitive operations for a given memory argument. The zero points must be passed at execution time as an argument with index :ref:`DNNL_ARG_ATTR_ZERO_POINTS ` \| arg. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Zero point correspondence mask that defines the correspondence between the tensor dimensions and the ``zero_points`` vector. The set i-th bit indicates that a dedicated zero point is used for each index along that dimension. Set the mask to 0 to use a common zero point for the whole output tensor. .. rubric:: See also: :ref:`dnnl_primitive_attr_set_zero_points_mask ` .. index:: pair: function; set_zero_points .. _doxid-structdnnl_1_1primitive__attr_1aa7a57b0ba198c418981d41c5289fed8e: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_zero_points( int arg, int mask, const :ref:`memory::dims`& groups, :ref:`memory::data_type` data_type = :ref:`memory::data_type::s32` ) Sets zero points for primitive operations for a given memory argument. The zero points must be passed at execution time as an argument with index :ref:`DNNL_ARG_ATTR_ZERO_POINTS ` \| arg. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Zero point correspondence mask that defines the correspondence between the tensor dimensions and the ``zero_points`` vector. The set i-th bit indicates that a dedicated zero point is used for each index along that dimension. Set the mask to 0 to use a common zero point for the whole output tensor. * - groups - Zero point factors correspondence groups that define the correspondence between the tensor dimensions and the zero_points array. The set i-th dimension indicates a number of groups of zero point factors used for that logical dimension in a memory indicated by ``arg``. * - data_type - Zero point factors data_type. .. rubric:: See also: :ref:`dnnl_primitive_attr_set_zero_points ` .. index:: pair: function; get_post_ops .. _doxid-structdnnl_1_1primitive__attr_1a05664ef63c94acbcc59e921c4a4da6b8: .. ref-code-block:: cpp :class: doxyrest-title-code-block const :ref:`post_ops` get_post_ops() const Returns post-ops previously set via :ref:`set_post_ops() `. .. rubric:: Returns: Post-ops. .. index:: pair: function; set_post_ops .. _doxid-structdnnl_1_1primitive__attr_1ac830fa9f4fcf480b494d73153ad579bf: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_post_ops(const :ref:`post_ops` ops) Sets post-ops. .. note:: There is no way to check whether the post-ops would be supported by the target primitive. Any error will be reported by the respective primitive descriptor constructor. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - ops - Post-ops object to copy post-ops from. .. index:: pair: function; set_rnn_data_qparams .. _doxid-structdnnl_1_1primitive__attr_1a39ce5aa8b06ed331d8e2158108cc8324: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_rnn_data_qparams(float scale, float shift) Sets quantization scale and shift parameters for RNN data tensors. For performance reasons, the low-precision configuration of the RNN primitives expect input activations to have the unsigned 8-bit integer data type. The scale and shift parameters are used to quantize floating-point data to unsigned integer and must be passed to the RNN primitive using attributes. The quantization formula is ``scale * data + shift``. Example usage: .. ref-code-block:: cpp // RNN parameters int l = 2, t = 2, mb = 32, sic = 32, slc = 32, dic = 32, dlc = 32; // Activations quantization parameters float scale = 63.f, shift = 64.f; primitive_attr attr; // Set scale and shift for int8 quantization of activation attr.set_rnn_data_qparams(scale, shift); // Create an RNN primitive descriptor. vanilla_rnn_forward::primitive_desc rnn_d( engine, /* arguments */, attr); .. note:: Quantization scale and shift are common for src_layer, src_iter, dst_iter, and dst_layer. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - scale - The value to scale the data by. * - shift - The value to shift the data by. .. index:: pair: function; get_rnn_data_qparams .. _doxid-structdnnl_1_1primitive__attr_1a47d567defa762761daa2af604798d799: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_rnn_data_qparams(float& scale, float& shift) Returns the quantization scale and shift parameters for RNN data tensors. .. note:: Quantization scale and shift are common for src_layer, src_iter, dst_iter, and dst_layer. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - scale - The value to scale the data by. * - shift - The value to shift the data by. .. index:: pair: function; set_rnn_weights_qparams .. _doxid-structdnnl_1_1primitive__attr_1a61bd70f97baa628fd49b2c8b334b913e: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_rnn_weights_qparams(int mask, const std::vector& scales) Sets quantization scaling factors for RNN weights tensors. The low-precision configuration of the RNN primitives expect input weights to use the signed 8-bit integer data type. The scaling factors are used to quantize floating-point data to signed integer and must be passed to RNN primitives using attributes. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. note:: Quantization scales are common for weights_layer and weights_iteration .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor. .. index:: pair: function; get_rnn_weights_qparams .. _doxid-structdnnl_1_1primitive__attr_1a3bbe9ac516e3aabe7dfea214210a3335: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_rnn_weights_qparams(int& mask, std::vector& scales) Returns the quantization scaling factors for RNN projection weights tensors. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor. .. index:: pair: function; set_rnn_weights_projection_qparams .. _doxid-structdnnl_1_1primitive__attr_1a6e5a8c12f28421c249633bf2092fbe3f: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_rnn_weights_projection_qparams( int mask, const std::vector& scales ) Sets quantization scaling factors for RNN projection weights tensors. passed to RNN primitives using attributes. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. note:: Quantization scales are common for weights_layer and weights_iteration .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor. .. index:: pair: function; get_rnn_weights_projection_qparams .. _doxid-structdnnl_1_1primitive__attr_1a53c41b29c5cd74d9485b5e753ba57f1d: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_rnn_weights_projection_qparams(int& mask, std::vector& scales) Returns the quantization scaling factors for RNN projection weights tensors. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor.