Memory object. More...

#include <dnnl.hpp>

Inheritance diagram for dnnl::memory:

Collaboration diagram for dnnl::memory:

Classes
struct	desc
	A memory descriptor. More...

Public Types
enum	data_type
	Data type specification. More...

enum	format_kind
	Memory format kind. More...

enum	format_tag
	Memory format tag specification. More...

typedef dnnl_dim_t	dim
	Integer type for representing dimension sizes and indices.

typedef std::vector< dim >	dims
	Vector of dimensions. More...

Public Member Functions
	memory (const desc &md, const engine &engine, void *handle)
	Constructs a memory object. More...

	memory (const desc &md, const engine &engine)
	Constructs a memory object. More...

desc	get_desc () const
	Returns the associated memory descriptor.

engine	get_engine () const
	Returns the associated engine.

void *	get_data_handle () const
	Returns the underlying memory buffer. More...

void	set_data_handle (void *handle) const
	Sets memory buffer. More...

template<typename T = void>
T *	map_data () const
	Maps the data of the memory. More...

void	unmap_data (void *mapped_ptr) const
	Unmaps the previously mapped data for the memory. More...

cl_mem	get_ocl_mem_object () const
	Returns the OpenCL memory object associated with the memory.

void	set_ocl_mem_object (cl_mem mem_object)
	Sets the OpenCL memory object `mem_object` associated with the memory. More...

Public Member Functions inherited from dnnl::handle< dnnl_memory_t >
bool	operator== (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &other) const
	Equality operator. More...

bool	operator!= (const handle &other) const
	Inequality operator. More...

	handle ()=default
	Constructs an empty handle object. More...

	handle (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &)=default
	Copy constructor.

	handle (handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &&)=default
	Move constructor.

	handle (dnnl_memory_t t, bool weak=false)
	Constructs a handle wrapper object from a C API handle. More...

handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &	operator= (const handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &)=default
	Assignment operator.

handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &	operator= (handle< dnnl_memory_t, handle_traits< dnnl_memory_t > > &&)=default
	Move assignment operator.

void	reset (dnnl_memory_t t, bool weak=false)
	Resets the handle wrapper objects to wrap a new C API handle. More...

dnnl_memory_t	get (bool allow_empty=false) const
	Returns the underlying C API handle. More...

	operator dnnl_memory_t () const
	Converts a handle to the underlying C API handle type. More...

	operator bool () const
	Checks whether the object is empty. More...

Static Public Member Functions
template<typename T >
static void	validate_dims (const std::vector< T > &v, int min_size=0)
	Helper function that validates that an `std::vector` of dimensions can be safely converted to the C API array dnnl_dims_t. More...

Detailed Description

Memory object.

A memory object encapsulates a handle to a memory buffer allocated on a specific engine, tensor dimensions, data type, and memory format, which is the way tensor indices map to offsets in linear memory space. Memory objects are passed to primitives during execution.

Examples:: cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_f32.cpp, cpu_cnn_training_bf16.cpp, cpu_matmul_quantization.cpp, cpu_rnn_inference_f32.cpp, cpu_rnn_inference_int8.cpp, cpu_sgemm_and_matmul.cpp, cross_engine_reorder.cpp, getting_started.cpp, gpu_opencl_interop.cpp, inference_int8_matmul.cpp, memory_format_propagation.cpp, performance_profiling.cpp, and rnn_training_f32.cpp.

Member Typedef Documentation

◆ dims

typedef std::vector<dim> dnnl::memory::dims

Vector of dimensions.

Implementations are free to force a limit on the vector's length.

Examples:: cpu_sgemm_and_matmul.cpp.

Member Enumeration Documentation

◆ data_type

enum dnnl::memory::data_type

strong

Data type specification.

Enumerator
undef	Undefined data type (used for empty memory descriptors).
f16	16-bit/half-precision floating point.
bf16	non-standard 16-bit floating point with 7-bit mantissa.
f32	32-bit/single-precision floating point.
s32	32-bit signed integer.
s8	8-bit signed integer.
u8	8-bit unsigned integer.

Examples:: cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_f32.cpp, cpu_cnn_training_bf16.cpp, cpu_matmul_quantization.cpp, and rnn_training_f32.cpp.

◆ format_kind

enum dnnl::memory::format_kind

strong

Memory format kind.

Enumerator
undef	Undefined memory format kind, used for empty memory descriptors.
any	Unspecified format kind. The primitive selects a format automatically.
blocked	A tensor in a generic format described by the stride and blocking values in each dimension. See dnnl_blocking_desc_t for more information.
wino	Weights format used in 8bit Winograd convolution.
packed	Packed weights format used in RNN.

◆ format_tag

enum dnnl::memory::format_tag

strong

Memory format tag specification.

Memory format tags can be further divided into two categories:

Domain-agnostic names, i.e. names that do not depend on the tensor usage in the specific primitive. These names use letters from a to l to denote logical dimensions and form the order in which the dimensions are laid in memory. For example, dnnl::memory::format_tag::ab is used to denote a 2D tensor where the second logical dimension (denoted as b) is the innermost, i.e. has stride = 1, and the first logical dimension (a) is laid out in memory with stride equal to the size of the second dimension. On the other hand, dnnl::memory::format_tag::ba is the transposed version of the same tensor: the outermost dimension (a) becomes the innermost one.
Domain-specific names, i.e. names that make sense only in the context of a certain domain, such as CNN. These names are aliases to the corresponding domain-agnostic tags and used mostly for convenience. For example, dnnl::memory::format_tag::nc is used to denote 2D CNN activations tensor memory format, where the channels dimension is the innermost one and the batch dimension is the outermost one. Moreover, dnnl::memory::format_tag::nc is an alias for dnnl::memory::format_tag::ab, because for CNN primitives the logical dimensions of activations tensors come in order: batch, channels, spatial. In other words, batch corresponds to the first logical dimension (a), and channels correspond to the second one (b).

The following domain-specific notation applies to memory format tags:

'n' denotes the mini-batch dimension
'c' denotes a channels dimension
When there are multiple channel dimensions (for example, in convolution weights tensor), 'i' and 'o' denote dimensions of input and output channels
'g' denotes a groups dimension for convolution weights
'd', 'h', and 'w' denote spatial depth, height, and width respectively

See dnnl_format_tag_t for a detailed description.

Enumerator
undef	Undefined memory format tag.
any	Placeholder memory format tag. Used to instruct the primitive to select a format automatically.
a	plain 1D tensor
ab	plain 2D tensor
ba	permuted 2D tensor
abc	plain 3D tensor
acb	permuted 3D tensor
bac	permuted 3D tensor
bca	permuted 3D tensor
cba	permuted 3D tensor
abcd	plain 4D tensor
acdb	permuted 4D tensor
bacd	permuted 4D tensor
bcda	permuted 4D tensor
cdba	permuted 4D tensor
abcde	plain 5D tensor
abdec	permuted 5D tensor
acbde	permuted 5D tensor
acdeb	permuted 5D tensor
bcdea	permuted 5D tensor
cdeba	permuted 5D tensor
decab	permuted 5D tensor
abcdef	plain 6D tensor
acbdef	plain 6D tensor
defcab	plain 6D tensor
x	1D tensor; an alias for dnnl::memory::format_tag::a
nc	2D CNN activations tensor; an alias for dnnl::memory::format_tag::ab
cn	2D CNN activations tensor; an alias for dnnl::memory::format_tag::ba
tn	2D RNN statistics tensor; an alias for dnnl::memory::format_tag::ab
nt	2D RNN statistics tensor; an alias for dnnl::memory::format_tag::ba
ncw	3D CNN activations tensor; an alias for dnnl::memory::format_tag::abc
nwc	3D CNN activations tensor; an alias for dnnl::memory::format_tag::acb
nchw	4D CNN activations tensor; an alias for dnnl::memory::format_tag::abcd
nhwc	4D CNN activations tensor; an alias for dnnl::memory::format_tag::acdb
chwn	4D CNN activations tensor; an alias for dnnl::memory::format_tag::bcda
ncdhw	5D CNN activations tensor; an alias for dnnl::memory::format_tag::abcde
ndhwc	5D CNN activations tensor; an alias for dnnl::memory::format_tag::acdeb
oi	2D CNN weights tensor; an alias for dnnl::memory::format_tag::ab
io	2D CNN weights tensor; an alias for dnnl::memory::format_tag::ba
oiw	3D CNN weights tensor; an alias for dnnl::memory::format_tag::abc
owi	3D CNN weights tensor; an alias for dnnl::memory::format_tag::acb
wio	3D CNN weights tensor; an alias for dnnl::memory::format_tag::cba
iwo	3D CNN weights tensor; an alias for dnnl::memory::format_tag::bca
oihw	4D CNN weights tensor; an alias for dnnl::memory::format_tag::abcd
hwio	4D CNN weights tensor; an alias for dnnl::memory::format_tag::cdba
ohwi	4D CNN weights tensor; an alias for dnnl::memory::format_tag::acdb
ihwo	4D CNN weights tensor; an alias for dnnl::memory::format_tag::bcda
iohw	4D CNN weights tensor; an alias for dnnl::memory::format_tag::bacd
oidhw	5D CNN weights tensor; an alias for dnnl::memory::format_tag::abcde
dhwio	5D CNN weights tensor; an alias for dnnl::memory::format_tag::cdeba
odhwi	5D CNN weights tensor; an alias for dnnl::memory::format_tag::acdeb
idhwo	5D CNN weights tensor; an alias for dnnl::memory::format_tag::bcdea
goiw	4D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcd
goihw	5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcde
hwigo	5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::decab
giohw	5D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::acbde
goidhw	6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcdef
giodhw	6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::abcdef
dhwigo	6D CNN weights tensor with groups; an alias for dnnl::memory::format_tag::defcab
tnc	3D RNN data tensor in the format (seq_length, batch, input channels).
ntc	3D RNN data tensor in the format (batch, seq_length, input channels).
ldnc	4D RNN states tensor in the format (num_layers, num_directions, batch, state channels).
ldigo	5D RNN weights tensor in the format (num_layers, num_directions, input_channels, num_gates, output_channels). For LSTM cells, the gates order is input, forget, candidate and output gate. For GRU cells, the gates order is update, reset and output gate.
ldgoi	5D RNN weights tensor in the format (num_layers, num_directions, num_gates, output_channels, input_channels). For LSTM cells, the gates order is input, forget, candidate and output gate. For GRU cells, the gates order is update, reset and output gate.
ldgo	4D RNN bias tensor in the format (num_layers, num_directions, num_gates, output_channels). For LSTM cells, the gates order is input, forget, candidate and output gate. For GRU cells, the gates order is update, reset and output gate.
aBcd8b	4D tensor blocked by 2nd dimension with block size 8
ABcd8b8a	4D tensor blocked by 1st and 2nd dimension with block size 8

Examples:: cnn_inference_f32.cpp, cnn_inference_int8.cpp, cnn_training_f32.cpp, cpu_cnn_training_bf16.cpp, and rnn_training_f32.cpp.

Constructor & Destructor Documentation

◆ memory() [1/2]

dnnl::memory::memory	(	const desc &	md,
		const engine &	engine,
		void *	handle
	)

inline

Constructs a memory object.

Unless handle is equal to DNNL_MEMORY_NONE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if memory::set_data_handle() had been called.

See also: memory::set_data_handle()

Parameters

md	Memory descriptor.
engine	Engine to store the data on.
handle	Handle of the memory buffer to use as an underlying storage. A pointer to the user-allocated buffer. In this case the library doesn't own the buffer. The DNNL_MEMORY_ALLOCATE special value. Instructs the library to allocate the buffer for the memory object. In this case the library owns the buffer. DNNL_MEMORY_NONE to create dnnl_memory without an underlying buffer.

◆ memory() [2/2]

dnnl::memory::memory	(	const desc &	md,
		const engine &	engine
	)

inline

Constructs a memory object.

The underlying storage for the memory will be allocated by the library.

Parameters

md	Memory descriptor.
engine	Engine to store the data on.

Member Function Documentation

◆ validate_dims()

template<typename T >

static void dnnl::memory::validate_dims	(	const std::vector< T > &	v,
		int	min_size = `0`
	)

inlinestatic

Helper function that validates that an std::vector of dimensions can be safely converted to the C API array dnnl_dims_t.

Throws if validation fails.

Parameters

v	Vector of dimensions.
min_size	Minimum expected size of the vector.

◆ get_data_handle()

void* dnnl::memory::get_data_handle ( ) const

inline

Returns the underlying memory buffer.

On the CPU engine this is a pointer to the allocated memory.

Examples:: cpu_rnn_inference_f32.cpp, and cpu_rnn_inference_int8.cpp.

◆ set_data_handle()

void dnnl::memory::set_data_handle ( void * handle ) const

inline

Sets memory buffer.

This function may write zeroes to the specified data handle if the memory object has padding to maintain data consistency.

Note: The padding is performed for memory objects created with blocked memory format tags like dnnl_aBcd8b when any of the dimensions is not a multiple of a corresponding block size. The padding is performed only for memory objects created with plain memory format tags like dnnl_nchw or dnnl_nhwc if requested explicitly. More information is available in Understanding Memory Formats.

The write can be time consuming and happens each time the function is called. Furthermore, it is performed using an internal service stream in a blocking manner.

Warning: Even if the memory object is used to hold values that stay constant (e.g., pre-packed weights during inference), the function will still write zeroes to the padding area if it exists. Hence, the handle parameter cannot and does not have a const qualifier.

Parameters

handle Output data handle. For the CPU engine, the data handle is a pointer to the actual data. For OpenCL it is a cl_mem.

Examples:: cpu_rnn_inference_f32.cpp, and cpu_rnn_inference_int8.cpp.

◆ map_data()

template<typename T = void>

T* dnnl::memory::map_data ( ) const

inline

Maps the data of the memory.

Mapping enables read/write directly from/to the memory contents for engines that do not support direct memory access.

Mapping is an exclusive operation - a memory object cannot be used in other operations until this memory object is unmapped.

Note: Any primitives working with the memory should be completed before the memory is mapped. Use stream::wait() to synchronize the corresponding execution stream.; The map/unmap API is provided mainly for debug/testing purposes and its performance may be suboptimal.

Template Parameters

T	Data type to return a pointer to.

Returns: Pointer to the mapped memory.

◆ unmap_data()

void dnnl::memory::unmap_data ( void * mapped_ptr ) const

inline

Unmaps the previously mapped data for the memory.

Any changes of the mapped data are synchronized back to the memory after the call is complete. The mapped pointer must be obtained through a map_data() call.

Note: The map/unmap API is provided mainly for debug/testing purposes and its performance may be suboptimal.

Parameters

mapped_ptr A pointer previously returned by map_data().

◆ set_ocl_mem_object()

void dnnl::memory::set_ocl_mem_object ( cl_mem mem_object )

inline

Sets the OpenCL memory object mem_object associated with the memory.

For behavioral details see memory::set_data_handle().

Parameters

mem_object OpenCL cl_mem object to use as the underlying storage. It must have at least get_desc().get_size() bytes allocated.

The documentation for this struct was generated from the following file:

include/dnnl.hpp

Classes

Public Types

Public Member Functions

Static Public Member Functions

Detailed Description

Member Typedef Documentation

◆ dims

Member Enumeration Documentation

◆ data_type

◆ format_kind

◆ format_tag

Constructor & Destructor Documentation

◆ memory() [1/2]

◆ memory() [2/2]

Member Function Documentation

◆ validate_dims()

◆ get_data_handle()

◆ set_data_handle()

◆ map_data()

◆ unmap_data()

◆ set_ocl_mem_object()