Memory

A primitive to describe and store data. More...

Functions
dnnl_status_t DNNL_API	dnnl_memory_desc_init_by_strides (dnnl_memory_desc_t *memory_desc, int ndims, const dnnl_dims_t dims, dnnl_data_type_t data_type, const dnnl_dims_t strides)
	Initializes a memory_desc memory descriptor using ndims, dims, data_type, and strides. More...
dnnl_status_t DNNL_API	dnnl_memory_desc_init_by_tag (dnnl_memory_desc_t *memory_desc, int ndims, const dnnl_dims_t dims, dnnl_data_type_t data_type, dnnl_format_tag_t tag)
	Initializes a memory_desc memory descriptor using ndims, dims, data_type, and format tag. More...
dnnl_status_t DNNL_API	dnnl_memory_desc_init_submemory (dnnl_memory_desc_t *memory_desc, const dnnl_memory_desc_t *parent_memory_desc, const dnnl_dims_t dims, const dnnl_dims_t offsets)
	Initializes a memory_desc for a given parent_memory_desc, with dims sizes and offsets. More...
dnnl_status_t DNNL_API	dnnl_memory_desc_reshape (dnnl_memory_desc_t *out_memory_desc, const dnnl_memory_desc_t *in_memory_desc, int ndims, const dnnl_dims_t dims)
	Initializes an out_memory_desc with new ndims and dims from a in_memory_desc. More...
int DNNL_API	dnnl_memory_desc_equal (const dnnl_memory_desc_t *lhs, const dnnl_memory_desc_t *rhs)
	Compares two memory descriptors. More...
size_t DNNL_API	dnnl_memory_desc_get_size (const dnnl_memory_desc_t *memory_desc)
	Returns the size (in bytes) that is required for given memory_desc.
dnnl_status_t DNNL_API	dnnl_memory_create (dnnl_memory_t *memory, const dnnl_memory_desc_t *memory_desc, dnnl_engine_t engine, void *handle)
	Creates a memory for given memory_desc and engine. More...
dnnl_status_t DNNL_API	dnnl_memory_get_memory_desc (const_dnnl_memory_t memory, const dnnl_memory_desc_t **memory_desc)
	Returns a memory_desc associated with memory.
dnnl_status_t DNNL_API	dnnl_memory_get_engine (const_dnnl_memory_t memory, dnnl_engine_t *engine)
	Returns an engine associated with memory.
dnnl_status_t DNNL_API	dnnl_memory_map_data (const_dnnl_memory_t memory, void **mapped_ptr)
	For a memory, maps the data of the memory to mapped_ptr. More...
dnnl_status_t DNNL_API	dnnl_memory_unmap_data (const_dnnl_memory_t memory, void *mapped_ptr)
	For a memory, unmaps a mapped pointer to the data of the memory. More...
dnnl_status_t DNNL_API	dnnl_memory_get_data_handle (const_dnnl_memory_t memory, void **handle)
	For a memory, returns the data handle. More...
dnnl_status_t DNNL_API	dnnl_memory_set_data_handle (dnnl_memory_t memory, void *handle)
	For a memory, sets the data handle.
dnnl_status_t DNNL_API	dnnl_memory_get_ocl_mem_object (const_dnnl_memory_t memory, cl_mem *mem_object)
	For a memory returns the OpenCL memory object associated with it.
dnnl_status_t DNNL_API	dnnl_memory_set_ocl_mem_object (dnnl_memory_t memory, cl_mem mem_object)
	For a memory sets the OpenCL memory object associated with it.
dnnl_status_t DNNL_API	dnnl_memory_destroy (dnnl_memory_t memory)
	Deletes a memory.

Detailed Description

A primitive to describe and store data.

The library supports various data types and formats. Memory hierarchy consists of three levels of abstraction:

1. Memory descriptor – engine agnostic logical description of data (number of dimensions, dimensions themselves, and data type), and optionally the format/layout that describes the physical representation of data in memory. If the format is not known yet, one can pass dnnl_format_tag_any. This approach is used to allow compute-intensive primitives to specify the most appropriate format on their own with users required to reorder the data if the incoming format doesn't match the primitive's selection. Memory descriptor can be initialized with dnnl_memory_desc_init_by_tag() or dnnl_memory_desc_init_by_strides() functions, or by directly filling the dnnl_memory_desc_t structure. The latter requires deep knowledge of how the physical data representation is mapped to the structure. The Understanding Memory Formats topic should shed some light on that. For the fully defined memory descriptors (i.e. where the format kind is not equal to dnnl_format_kind_any) a user can the size, using the dnnl_memory_desc_get_size() function. As described in Understanding Memory Formats, the size of data sometimes cannot be computed as the product of dimensions times the size of the data type. So users are encouraged to use this function for better code portability. Two memory descriptors can be compared with dnnl_memory_desc_equal(). The comparison is especially useful when checking whether a primitive requires reorder from the user's data format to the primitive's format.
2. Memory – an engine-specific object that handles the data and its description (a memory descriptor). For CPU enigne, the data handle is simply a pointer to void. The data handle can be queried using dnnl_memory_get_data_handle() and set using dnnl_memory_set_data_handle(). The latter function always sets the memory in the padding region to zero, which is the invariant maintained by all the primitives in DNNL. See Understanding Memory Formats for more details. A memory can be created using dnnl_memory_create() function. A memory can also be queried for the underlying memory descriptor and engine using dnnl_memory_get_memory_desc() and dnnl_memory_get_engine() functions.

Along with ordinary memory with all dimensions being positive, Intel DNNL supports zero-volume memory with one or more dimensions set to zero. This is to support the NumPy* convention. If a zero-volume memory is passed to a primitive, the primitive does not perform any computations on this memory. For example:

• Convolution with (0 batch, 3 input channels, 13 height, 13 width) source and (16 output channels, 3 input channels, 3 height, 3 width) weights would produce (0 batch, 16 output channels, 11 height, 11 width) destination (assuming strides are 1 and paddings are zero) and perform zero multiply-add operations.
• Concatenation of three memories of shapes (3, 4, 13, 13), (3, 0, 13, 13), and (3, 1, 13, 13) along the second axis would produce the output of the shape (3, 5, 13, 13), effectively ignoring the second input (however, if the user created a concatenation primitive descriptor with three inputs they should also provide all three memories to the concatenation primitive, including the one with zero second dimension).
• However, DNNL would return an error when attempting to create a convolution with zero-volume memory passed for weights because such a convolution is not well-defined:

  dst(1, 16, 11, 11) <-- src(1, 0, 13, 13) (*) wei(16, 0, 3, 3)

  Should the values in the destination be zeroes or just not accessed at all? Moreover, backward pass w.r.t. weights in such cases is also not well-defined.

Data handle of zero-volume memory is never accessed and hence can be unset (NULL in case of CPU engine).

See also

Understanding Memory Formats

Function Documentation

dnnl_memory_desc_init_by_strides()

dnnl_status_t DNNL_API dnnl_memory_desc_init_by_strides	(	dnnl_memory_desc_t *	memory_desc,
		int	ndims,
		const dnnl_dims_t	dims,
		dnnl_data_type_t	data_type,
		const dnnl_dims_t	strides
	)

Initializes a memory_desc memory descriptor using ndims, dims, data_type, and strides.

The strides might be NULL, which means the order of physical dimensions is the same as the order of logical ones.

Note

The logical order of dimensions is defined by a primitive that consumes the memory.

dnnl_memory_desc_init_by_tag()

dnnl_status_t DNNL_API dnnl_memory_desc_init_by_tag	(	dnnl_memory_desc_t *	memory_desc,
		int	ndims,
		const dnnl_dims_t	dims,
		dnnl_data_type_t	data_type,
		dnnl_format_tag_t	tag
	)

Initializes a memory_desc memory descriptor using ndims, dims, data_type, and format tag.

tag can be dnnl_format_tag_any, which allows a primitive to define the appropriate memory format. In this case, the format_kind would be set to dnnl_format_kind_any

Examples:

cnn_inference_f32.c, cpu_cnn_training_f32.c, and cross_engine_reorder.c.

dnnl_memory_desc_init_submemory()

dnnl_status_t DNNL_API dnnl_memory_desc_init_submemory	(	dnnl_memory_desc_t *	memory_desc,
		const dnnl_memory_desc_t *	parent_memory_desc,
		const dnnl_dims_t	dims,
		const dnnl_dims_t	offsets
	)

Initializes a memory_desc for a given parent_memory_desc, with dims sizes and offsets.

May fail if layout used does not allow obtain desired submemory. In this case consider using extract or insert primitive

dnnl_memory_desc_reshape()

dnnl_status_t DNNL_API dnnl_memory_desc_reshape	(	dnnl_memory_desc_t *	out_memory_desc,
		const dnnl_memory_desc_t *	in_memory_desc,
		int	ndims,
		const dnnl_dims_t	dims
	)

Initializes an out_memory_desc with new ndims and dims from a in_memory_desc.

If in_memory_desc and out_memory_desc point to the same memory descriptor, then in_memory_desc is re-initialized with new values. out_memory_desc inherits data type from in_memory_desc and has same blocked format_kind. Others format_kind's are not supported.

Note

Limitation: the only currently supported reshape operation is appending 1-sized dimensions to the end.

dnnl_memory_desc_equal()

int DNNL_API dnnl_memory_desc_equal	(	const dnnl_memory_desc_t *	lhs,
		const dnnl_memory_desc_t *	rhs
	)

Compares two memory descriptors.

Returns

1 if the descriptors are the same.

0 if the descriptors are different.

Use this function to identify whether a reorder is required between the two memories

Examples:

cnn_inference_f32.c, and cpu_cnn_training_f32.c.

dnnl_memory_create()

dnnl_status_t DNNL_API dnnl_memory_create	(	dnnl_memory_t *	memory,
		const dnnl_memory_desc_t *	memory_desc,
		dnnl_engine_t	engine,
		void *	handle
	)

Creates a memory for given memory_desc and engine.

Also sets handle to one of the following:

• pointer to the user allocated memory, i.e. valid handle. In this case the library doesn't own allocated memory.
• DNNL_MEMORY_ALLOCATE to ask the library to allocate and attach memory. In this case the library owns allocated memory.
• DNNL_MEMORY_NONE to create dnnl_memory w/o attached memory.

Examples:

cnn_inference_f32.c, cpu_cnn_training_f32.c, and cross_engine_reorder.c.

dnnl_memory_map_data()

dnnl_status_t DNNL_API dnnl_memory_map_data	(	const_dnnl_memory_t	memory,
		void **	mapped_ptr
	)

For a memory, maps the data of the memory to mapped_ptr.

Mapping allows to read/write directly from/to the memory contents for engines that do not support direct memory access.

Mapping is an exclusive operation - a memory object cannot be used in other operations until this memory object is unmapped.

Note

Any primitives working with memory should be completed before mapping the memory. Use dnnl_stream_wait to synchronize the corresponding execution stream.

Map/unmap API is provided mainly for debug/testing purposes and its performance may be suboptimal.

Examples:

cross_engine_reorder.c.

dnnl_memory_unmap_data()

dnnl_status_t DNNL_API dnnl_memory_unmap_data	(	const_dnnl_memory_t	memory,
		void *	mapped_ptr
	)

For a memory, unmaps a mapped pointer to the data of the memory.

Any changes of the mapped data are synchronized back to the memory after the call is complete. The mapped pointer must be obtained through a dnnl_memory_map_data call.

Note

Map/unmap API is provided mainly for debug/testing purposes and its performance may be suboptimal.

Examples:

cross_engine_reorder.c.

dnnl_memory_get_data_handle()

dnnl_status_t DNNL_API dnnl_memory_get_data_handle	(	const_dnnl_memory_t	memory,
		void **	handle
	)

For a memory, returns the data handle.

For the CPU engine, the data handle is a pointer to the actual data.

Examples:

cpu_cnn_training_f32.c.