Memory object. More...

#include <dnnl.hpp>

Inheritance diagram for dnnl::memory:

Collaboration diagram for dnnl::memory:

Classes
struct	desc
	A memory descriptor. More...

Public Types
enum	data_type
	Data type specification. More...

enum	format_kind
	Memory format kind. More...

enum	format_tag
	Memory format tag specification. More...

typedef dnnl_dim_t	dim
	Integer type for representing dimension sizes and indices.

typedef std::vector< dim >	dims
	Vector of dimensions. More...

Public Member Functions
	memory ()=default
	Default constructor. More...

	memory (const desc &md, const engine &aengine, void *handle)
	Constructs a memory object. More...

	memory (const desc &md, const engine &aengine)
	Constructs a memory object. More...

desc	get_desc () const
	Returns the associated memory descriptor.

engine	get_engine () const
	Returns the associated engine.

void *	get_data_handle () const
	Returns the underlying memory buffer. More...

void	set_data_handle (void *handle, const stream &astream) const
	Sets the underlying memory buffer. More...

void	set_data_handle (void *handle) const
	Sets the underlying memory buffer. More...

template<typename T = void>
T *	map_data () const
	Maps a memory object and returns a host-side pointer to a memory buffer with a copy of its contents. More...

void	unmap_data (void *mapped_ptr) const
	Unmaps a memory object and writes back any changes made to the previously mapped memory buffer. More...

	handle ()=default
	Constructs an empty handle object. More...

	handle (const handle< T, traits > &)=default
	Copy constructor.

	handle (handle< T, traits > &&)=default
	Move constructor.

	handle (T t, bool weak=false)
	Constructs a handle wrapper object from a C API handle. More...

Public Member Functions inherited from dnnl::handle< dnnl_memory_t >
bool	operator== (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &other) const
	Equality operator. More...

bool	operator!= (const handle &other) const
	Inequality operator. More...

	handle ()=default
	Constructs an empty handle object. More...

	handle (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &)=default
	Copy constructor.

	handle (handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &&)=default
	Move constructor.

	handle (dnnl_memory_t t, bool weak=false)
	Constructs a handle wrapper object from a C API handle. More...

handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &	operator= (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &)=default
	Assignment operator.

handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &	operator= (handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &&)=default
	Move assignment operator.

void	reset (dnnl_memory_t t, bool weak=false)
	Resets the handle wrapper objects to wrap a new C API handle. More...

dnnl_memory_t	get (bool allow_empty=false) const
	Returns the underlying C API handle. More...

	operator dnnl_memory_t () const
	Converts a handle to the underlying C API handle type. More...

	operator bool () const
	Checks whether the object is not empty. More...

Static Public Member Functions
template<typename T >
static void	validate_dims (const std::vector< T > &v, int min_size=0)
	Helper function that validates that an `std::vector` of dimensions can be safely converted to the C API array dnnl_dims_t. More...

Detailed Description

Memory object.

A memory object encapsulates a handle to a memory buffer allocated on a specific engine, tensor dimensions, data type, and memory format, which is the way tensor indices map to offsets in linear memory space. Memory objects are passed to primitives during execution.

Examples:: batch_normalization.cpp, binary.cpp, cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_bf16.cpp, cnn_training_f32.cpp, concat.cpp, convolution.cpp, cpu_matmul_quantization.cpp, cpu_rnn_inference_f32.cpp, cpu_rnn_inference_int8.cpp, cpu_sgemm_and_matmul.cpp, cross_engine_reorder.cpp, eltwise.cpp, getting_started.cpp, gpu_opencl_interop.cpp, inference_int8_matmul.cpp, inner_product.cpp, layer_normalization.cpp, logsoftmax.cpp, lrn.cpp, lstm.cpp, matmul.cpp, memory_format_propagation.cpp, performance_profiling.cpp, pooling.cpp, prelu.cpp, reduction.cpp, reorder.cpp, resampling.cpp, rnn_training_f32.cpp, shuffle.cpp, softmax.cpp, sum.cpp, and sycl_interop_buffer.cpp.

Member Typedef Documentation

◆ dims

typedef std::vector<dim> dnnl::memory::dims

Vector of dimensions.

Implementations are free to force a limit on the vector's length.

Examples:: cpu_sgemm_and_matmul.cpp.

Member Enumeration Documentation

◆ data_type

enum dnnl::memory::data_type

strong

Data type specification.

Enumerator
undef	Undefined data type (used for empty memory descriptors).
f16	16-bit/half-precision floating point.
bf16	non-standard 16-bit floating point with 7-bit mantissa.
f32	32-bit/single-precision floating point.
s32	32-bit signed integer.
s8	8-bit signed integer.
u8	8-bit unsigned integer.

Examples:: batch_normalization.cpp, binary.cpp, cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_bf16.cpp, cnn_training_f32.cpp, concat.cpp, convolution.cpp, cpu_matmul_quantization.cpp, eltwise.cpp, inner_product.cpp, layer_normalization.cpp, logsoftmax.cpp, lrn.cpp, lstm.cpp, matmul.cpp, pooling.cpp, prelu.cpp, reduction.cpp, reorder.cpp, resampling.cpp, rnn_training_f32.cpp, shuffle.cpp, softmax.cpp, and sum.cpp.

◆ format_kind

enum dnnl::memory::format_kind

strong

Memory format kind.

Enumerator
undef	Undefined memory format kind, used for empty memory descriptors.
any	Unspecified format kind. The primitive selects a format automatically.
blocked	A tensor in a generic format described by the stride and blocking values in each dimension. See dnnl_blocking_desc_t for more information.
wino	Weights format used in 8-bit Winograd convolution.
packed	Packed weights format used in RNN.

◆ format_tag

enum dnnl::memory::format_tag

strong

Memory format tag specification.

Memory format tags can be further divided into two categories:

Domain-agnostic names, i.e. names that do not depend on the tensor usage in the specific primitive. These names use letters from a to f to denote logical dimensions and form the order in which the dimensions are laid in memory. For example, dnnl::memory::format_tag::ab is used to denote a 2D tensor where the second logical dimension (denoted as b) is the innermost, i.e. has stride = 1, and the first logical dimension (a) is laid out in memory with stride equal to the size of the second dimension. On the other hand, dnnl::memory::format_tag::ba is the transposed version of the same tensor: the outermost dimension (a) becomes the innermost one.
Domain-specific names, i.e. names that make sense only in the context of a certain domain, such as CNN. These names are aliases to the corresponding domain-agnostic tags and used mostly for convenience. For example, dnnl::memory::format_tag::nc is used to denote 2D CNN activations tensor memory format, where the channels dimension is the innermost one and the batch dimension is the outermost one. Moreover, dnnl::memory::format_tag::nc is an alias for dnnl::memory::format_tag::ab, because for CNN primitives the logical dimensions of activations tensors come in order: batch, channels, spatial. In other words, batch corresponds to the first logical dimension (a), and channels correspond to the second one (b).

The following domain-specific notation applies to memory format tags:

'n' denotes the mini-batch dimension
'c' denotes a channels dimension
When there are multiple channel dimensions (for example, in convolution weights tensor), 'i' and 'o' denote dimensions of input and output channels
'g' denotes a groups dimension for convolution weights
'd', 'h', and 'w' denote spatial depth, height, and width respectively

See dnnl_format_tag_t for a detailed description.

Enumerator
undef	Undefined memory format tag.
any	Placeholder memory format tag. Used to instruct the primitive to select a format automatically.
a	plain 1D tensor
ab	plain 2D tensor
ba	permuted 2D tensor
abc	plain 3D tensor
acb	permuted 3D tensor
bac	permuted 3D tensor
bca	permuted 3D tensor
cba	permuted 3D tensor
abcd	plain 4D tensor
abdc	permuted 4D tensor
acdb	permuted 4D tensor
bacd	permuted 4D tensor
bcda	permuted 4D tensor
cdba	permuted 4D tensor
dcab	permuted 4D tensor
abcde	plain 5D tensor
abdec	permuted 5D tensor
acbde	permuted 5D tensor
acdeb	permuted 5D tensor
bacde	permuted 5D tensor
bcdea	permuted 5D tensor
cdeba	permuted 5D tensor
decab	permuted 5D tensor
abced	permuted 5D tensor
abcdef	plain 6D tensor
abdfce	permuted 6D tensor
acbdef	permuted 6D tensor
abdefc	permuted 6D tensor
defcab	permuted 6D tensor
abcdfe	permuted 6D tensor
abcdefg	plain 7D tensor
abcdegf	permuted 7D tensor
abcdefgh	plain 8D tensor
abcdefhg	permuted 8D tensor
abcdefghi	plain 9D tensor
abcdefgih	permuted 9D tensor
abcdefghij	plain 10D tensor
abcdefghji	permuted 10D tensor
abcdefghijk	plain 11D tensor
abcdefghikj	permuted 11D tensor
abcdefghijkl	plain 12D tensor
abcdefghijlk	permuted 12D tensor
x	1D tensor; an alias for dnnl::memory::format_tag::a
nc	2D CNN activations tensor; an alias for dnnl::memory::format_tag::ab
cn	2D CNN activations tensor; an alias for dnnl::memory::format_tag::ba
tn	2D RNN statistics tensor; an alias for dnnl::memory::format_tag::ab
nt	2D RNN statistics tensor; an alias for dnnl::memory::format_tag::ba
ncw	3D CNN activations tensor; an alias for dnnl::memory::format_tag::abc
nwc	3D CNN activations tensor; an alias for dnnl::memory::format_tag::acb
nchw	4D CNN activations tensor; an alias for dnnl::memory::format_tag::abcd
nhwc	4D CNN activations tensor; an alias for dnnl::memory::format_tag::acdb
chwn	4D CNN activations tensor; an alias for dnnl::memory::format_tag::bcda
ncdhw	5D CNN activations tensor; an alias for dnnl::memory::format_tag::abcde
ndhwc	5D CNN activations tensor; an alias for dnnl::memory::format_tag::acdeb
oi	2D CNN weights tensor; an alias for dnnl::memory::format_tag::ab
io	2D CNN weights tensor; an alias for dnnl::memory::format_tag::ba
oiw	3D CNN weights tensor; an alias for dnnl::memory::format_tag::abc
owi	3D CNN weights tensor; an alias for dnnl::memory::format_tag::acb
wio	3D CNN weights tensor; an alias for dnnl::memory::format_tag::cba
iwo	3D CNN weights tensor; an alias for dnnl::memory::format_tag::bca
oihw	4D CNN weights tensor; an alias for dnnl::memory::format_tag::abcd
hwio	4D CNN weights tensor; an alias for dnnl::memory::format_tag::cdba
ohwi	4D CNN weights tensor; an alias for dnnl::memory::format_tag::acdb
ihwo	4D CNN weights tensor; an alias for dnnl::memory::format_tag::bcda
iohw	4D CNN weights tensor; an alias for dnnl::memory::format_tag::bacd
oidhw	5D CNN weights tensor; an alias for dnnl::memory::format_tag::abcde
dhwio	5D CNN weights tensor; an alias for dnnl::memory::format_tag::cdeba
odhwi	5D CNN weights tensor; an alias for dnnl::memory::format_tag::acdeb
iodhw	5D CNN weights tensor; an alias for dnnl::memory::format_tag::bacde
idhwo	5D CNN weights tensor; an alias for dnnl::memory::format_tag::bcdea
goiw	4D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcd
gowi	4D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abdc
wigo	4D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::dcab
gohwi	5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abdec
goihw	5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcde
hwigo	5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::decab
giohw	5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::acbde
goidhw	6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcdef
giodhw	6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcdef
godhwi	6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abdefc
dhwigo	6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::defcab
tnc	3D RNN data tensor in the format (seq_length, batch, input channels).
ntc	3D RNN data tensor in the format (batch, seq_length, input channels).
ldnc	4D RNN states tensor in the format (num_layers, num_directions, batch, state channels).
ldigo	5D RNN weights tensor in the format (num_layers, num_directions, input_channels, num_gates, output_channels). For LSTM cells, the gates order is input, forget, candidate and output gate. For GRU cells, the gates order is update, reset and output gate.
ldgoi	5D RNN weights tensor in the format (num_layers, num_directions, num_gates, output_channels, input_channels). For LSTM cells, the gates order is input, forget, candidate and output gate. For GRU cells, the gates order is update, reset and output gate.
ldio	4D LSTM projection tensor in the format (num_layers, num_directions, num_channels_in_hidden_state, num_channels_in_recurrent_projection).
ldoi	4D LSTM projection tensor in the format (num_layers, num_directions, num_channels_in_recurrent_projection, num_channels_in_hidden_state).
ldgo	4D RNN bias tensor in the format (num_layers, num_directions, num_gates, output_channels). For LSTM cells, the gates order is input, forget, candidate and output gate. For GRU cells, the gates order is update, reset and output gate.
aBcd8b	4D tensor blocked by 2nd dimension with block size 8
ABcd8b8a	4D tensor blocked by 1st and 2nd dimension with block size 8

Examples:: batch_normalization.cpp, binary.cpp, cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_bf16.cpp, cnn_training_f32.cpp, concat.cpp, convolution.cpp, eltwise.cpp, inner_product.cpp, layer_normalization.cpp, logsoftmax.cpp, lrn.cpp, lstm.cpp, matmul.cpp, pooling.cpp, prelu.cpp, reduction.cpp, reorder.cpp, resampling.cpp, rnn_training_f32.cpp, shuffle.cpp, softmax.cpp, and sum.cpp.

Constructor & Destructor Documentation

◆ memory() [1/3]

dnnl::memory::memory ( )

default

Default constructor.

Constructs an empty memory object, which can be used to indicate absence of a parameter.

◆ memory() [2/3]

dnnl::memory::memory	(	const desc &	md,
		const engine &	aengine,
		void *	handle
	)

inline

Constructs a memory object.

Unless handle is equal to DNNL_MEMORY_NONE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if dnnl::memory::set_data_handle() had been called.

See also: memory::set_data_handle()

Parameters

md	Memory descriptor.
aengine	Engine to store the data on.
handle	Handle of the memory buffer to use. A pointer to the user-allocated buffer. In this case the library doesn't own the buffer. The DNNL_MEMORY_ALLOCATE special value. Instructs the library to allocate the buffer for the memory object. In this case the library owns the buffer. DNNL_MEMORY_NONE to create dnnl::memory without an underlying buffer.

◆ memory() [3/3]

dnnl::memory::memory	(	const desc &	md,
		const engine &	aengine
	)

inline

Constructs a memory object.

The underlying buffer for the memory will be allocated by the library.

Parameters

md	Memory descriptor.
aengine	Engine to store the data on.

Member Function Documentation

◆ validate_dims()

template<typename T >

static void dnnl::memory::validate_dims	(	const std::vector< T > &	v,
		int	min_size = `0`
	)

inlinestatic

Helper function that validates that an std::vector of dimensions can be safely converted to the C API array dnnl_dims_t.

Throws if validation fails.

Parameters

v	Vector of dimensions.
min_size	Minimum expected size of the vector.

◆ get_data_handle()

void* dnnl::memory::get_data_handle ( ) const

inline

Returns the underlying memory buffer.

On the CPU engine, or when using USM, this is a pointer to the allocated memory.

Examples:: cpu_rnn_inference_f32.cpp, and cpu_rnn_inference_int8.cpp.

◆ set_data_handle() [1/2]

void dnnl::memory::set_data_handle	(	void *	handle,
		const stream &	astream
	)		const

inline

Sets the underlying memory buffer.

This function may write zero values to the memory specified by the handle if the memory object has a zero padding area. This may be time consuming and happens each time this function is called. The operation is always blocking and the stream parameter is a hint.

Note: The zero padding is required by memory objects created with blocked memory format tags like dnnl_aBcd8b when any of the dimensions is not a multiple of the corresponding block size. For "plain" formats like dnnl::memory::format_tag::nchw or dnnl::memory::format_tag::nhwc zero padding area needs to be set up explicitly when creating the corresponding memory descriptors. See Understanding Memory Formats for more details.; Even when the memory object is used to hold values that stay constant during the execution of the program (pre-packed weights during inference, for example), the function will still write zeroes to the padding area if it exists. Hence, the handle parameter cannot and does not have a const qualifier.

Parameters

handle	Memory buffer to use. On the CPU engine or when USM is used, the memory buffer is a pointer to the actual data. For OpenCL it is a cl_mem. It must have at least dnnl::memory::desc::get_size() bytes allocated.
astream	Stream to use to execute padding in.

Examples:: cpu_rnn_inference_f32.cpp, and cpu_rnn_inference_int8.cpp.

◆ set_data_handle() [2/2]

void dnnl::memory::set_data_handle ( void * handle ) const

inline

Sets the underlying memory buffer.

See documentation for dnnl::memory::set_data_handle(void *, const stream &) const for more information.

Parameters

handle Memory buffer to use. For the CPU engine, the memory buffer is a pointer to the actual data. For OpenCL it is a cl_mem. It must have at least dnnl::memory::desc::get_size() bytes allocated.

◆ map_data()

template<typename T = void>

T* dnnl::memory::map_data ( ) const

inline

Maps a memory object and returns a host-side pointer to a memory buffer with a copy of its contents.

Mapping enables read/write directly from/to the memory contents for engines that do not support direct memory access.

Mapping is an exclusive operation - a memory object cannot be used in other operations until it is unmapped via dnnl::memory::unmap_data() call.

Note: Any primitives working with the memory should be completed before the memory is mapped. Use dnnl::stream::wait() to synchronize the corresponding execution stream.; The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.

Template Parameters

T	Data type to return a pointer to.

Returns: Pointer to the mapped memory.

◆ unmap_data()

void dnnl::memory::unmap_data ( void * mapped_ptr ) const

inline

Unmaps a memory object and writes back any changes made to the previously mapped memory buffer.

Note: The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.

Parameters

mapped_ptr A pointer previously returned by dnnl::memory::map_data().

◆ handle() [1/2]

dnnl::handle< T, traits >::handle

default

Constructs an empty handle object.

Warning: Uninitialized object cannot be used in most library calls and is equivalent to a null pointer. Any attempt to use its methods, or passing it to the other library function, will cause an exception to be thrown.

◆ handle() [2/2]

dnnl::handle< T, traits >::handle

inlineexplicit

Constructs a handle wrapper object from a C API handle.

Parameters

t	The C API handle to wrap.
weak	A flag specifying whether to construct a weak wrapper; defaults to `false`.

The documentation for this struct was generated from the following file:

include/oneapi/dnnl/dnnl.hpp

Classes

Public Types

Public Member Functions

Static Public Member Functions

Detailed Description

Member Typedef Documentation

◆ dims

Member Enumeration Documentation

◆ data_type

◆ format_kind

◆ format_tag

Constructor & Destructor Documentation

◆ memory() [1/3]

◆ memory() [2/3]

◆ memory() [3/3]

Member Function Documentation

◆ validate_dims()

◆ get_data_handle()

◆ set_data_handle() [1/2]

◆ set_data_handle() [2/2]

◆ map_data()

◆ unmap_data()

◆ handle() [1/2]

◆ handle() [2/2]