Memory

A primitive to describe and store data. More...

Functions
mkldnn_status_t MKLDNN_API	mkldnn_memory_desc_init_by_strides (mkldnn_memory_desc_t *memory_desc, int ndims, const mkldnn_dims_t dims, mkldnn_data_type_t data_type, const mkldnn_dims_t strides)
	Initializes a memory_desc memory descriptor using ndims, dims, data_type, and strides. More...
mkldnn_status_t MKLDNN_API	mkldnn_memory_desc_init_by_tag (mkldnn_memory_desc_t *memory_desc, int ndims, const mkldnn_dims_t dims, mkldnn_data_type_t data_type, mkldnn_format_tag_t tag)
	Initializes a memory_desc memory descriptor using ndims, dims, data_type, and format tag. More...
mkldnn_status_t MKLDNN_API	mkldnn_memory_desc_init_submemory (mkldnn_memory_desc_t *memory_desc, const mkldnn_memory_desc_t *parent_memory_desc, const mkldnn_dims_t dims, const mkldnn_dims_t offsets)
	Initializes a memory_desc for a given parent_memory_desc, with dims sizes and offsets. More...
int MKLDNN_API	mkldnn_memory_desc_equal (const mkldnn_memory_desc_t *lhs, const mkldnn_memory_desc_t *rhs)
	Compares two memory descriptors. More...
size_t MKLDNN_API	mkldnn_memory_desc_get_size (const mkldnn_memory_desc_t *memory_desc)
	Returns the size (in bytes) that is required for given memory_desc.
mkldnn_status_t MKLDNN_API	mkldnn_memory_create (mkldnn_memory_t *memory, const mkldnn_memory_desc_t *memory_desc, mkldnn_engine_t engine, void *handle)
	Creates a memory for given memory_desc and engine. More...
mkldnn_status_t MKLDNN_API	mkldnn_memory_get_memory_desc (const_mkldnn_memory_t memory, const mkldnn_memory_desc_t **memory_desc)
	Returns a memory_desc associated with memory.
mkldnn_status_t MKLDNN_API	mkldnn_memory_get_engine (const_mkldnn_memory_t memory, mkldnn_engine_t *engine)
	Returns an engine associated with memory.
mkldnn_status_t MKLDNN_API	mkldnn_memory_map_data (const_mkldnn_memory_t memory, void **mapped_ptr)
	For a memory, maps the data of the memory to mapped_ptr. More...
mkldnn_status_t MKLDNN_API	mkldnn_memory_unmap_data (const_mkldnn_memory_t memory, void *mapped_ptr)
	For a memory, unmaps a mapped pointer to the data of the memory. More...
mkldnn_status_t MKLDNN_API	mkldnn_memory_get_data_handle (const_mkldnn_memory_t memory, void **handle)
	For a memory, returns the data handle. More...
mkldnn_status_t MKLDNN_API	mkldnn_memory_set_data_handle (mkldnn_memory_t memory, void *handle)
	For a memory, sets the data handle.
mkldnn_status_t MKLDNN_API	mkldnn_memory_destroy (mkldnn_memory_t memory)
	Deletes a memory.

Detailed Description

A primitive to describe and store data.

The library supports various data types and formats. Memory hierarchy consists of three levels of abstraction:

1. Memory descriptor – engine agnostic logical description of data (number of dimensions, dimensions themselves, and data type), and optionally the format/layout that describes the physical representation of data in memory. If the format is not known yet, one can pass mkldnn_format_tag_any. This approach is used to allow compute-intensive primitives to specify the most appropriate format on their own with users required to reorder the data if the incoming format doesn't match the primitive's selection. Memory descriptor can be initialized with mkldnn_memory_desc_init_by_tag() or mkldnn_memory_desc_init_by_strides() functions, or by directly filling the mkldnn_memory_desc_t structure. The latter requires deep knowledge of how the physical data representation is mapped to the structure. The Understanding Memory Formats topic should shed some light on that. For the fully defined memory descriptors (i.e. where the format kind is not equal to mkldnn_format_kind_any) a user can the size, using the mkldnn_memory_desc_get_size() function. As described in Understanding Memory Formats, the size of data sometimes cannot be computed as the product of dimensions times the size of the data type. So users are encouraged to use this function for better code portability. Two memory descriptors can be compared with mkldnn_memory_desc_equal(). The comparison is especially useful when checking whether a primitive requires reorder from the user's data format to the primitive's format.
2. Memory – an engine-specific object that handles the data and its description (a memory descriptor). For CPU enigne, the data handle is simply a pointer to void. The data handle can be queried using mkldnn_memory_get_data_handle() and set using mkldnn_memory_set_data_handle(). The latter function always sets the memory in the padding region to zero, which is the invariant maintained by all the primitives in Intel MKL-DNN. See Understanding Memory Formats for more details. A memory can be created using mkldnn_memory_create() function. A memory can also be queried for the underlying memory descriptor and engine using mkldnn_memory_get_memory_desc() and mkldnn_memory_get_engine() functions.

Along with ordinary memory with all dimensions being positive, Intel MKL-DNN supports zero-volume memory with one or more dimensions set to zero. This is to support the NumPy* convention. If a zero-volume memory is passed to a primitive, the primitive does not perform any computations on this memory. For example:

• Convolution with (0 batch, 3 input channels, 13 height, 13 width) source and (16 output channels, 3 inputs, channel, 3 height, 3 width) weights would produce (0 batch, 16 output channels, 11 height, 11 width) destination (assuming strides are 1 and paddings are zero) and perform zero multiply-add operations.
• Concatenation of three memories of shapes (3, 4, 13, 13), (3, 0, 13, 13), and (3, 1, 13, 13) along the second axis would produce the output of the shape (3, 5, 13, 13), effectively ignoring the second input (however, if the user created a concatenation primitive descriptor with three inputs they should also provide all three memories to the concatenation primitive, including the one with zero second dimension).
• However, Intel MKL-DNN would return an error when attempting to create a convolution with zero-volume memory passed for weights because such a convolution is not well-defined:

  dst(1, 16, 11, 11) <-- src(1, 0, 13, 13) (*) wei(16, 0, 3, 3)

  Should the values in the destination be zeroes or just not accessed at all? Moreover, backward pass w.r.t. weights in such cases is also not well-defined.

Data handle of zero-volume memory is never accessed and hence can be unset (NULL in case of CPU engine).

See also

Understanding Memory Formats

Function Documentation

mkldnn_memory_desc_init_by_strides()

mkldnn_status_t MKLDNN_API mkldnn_memory_desc_init_by_strides	(	mkldnn_memory_desc_t *	memory_desc,
		int	ndims,
		const mkldnn_dims_t	dims,
		mkldnn_data_type_t	data_type,
		const mkldnn_dims_t	strides
	)

Initializes a memory_desc memory descriptor using ndims, dims, data_type, and strides.

The strides might be NULL, which means the order of physical dimensions is the same as the order of logical ones.

Note

The logical order of dimensions is defined by a primitive that consumes the memory.

mkldnn_memory_desc_init_by_tag()

mkldnn_status_t MKLDNN_API mkldnn_memory_desc_init_by_tag	(	mkldnn_memory_desc_t *	memory_desc,
		int	ndims,
		const mkldnn_dims_t	dims,
		mkldnn_data_type_t	data_type,
		mkldnn_format_tag_t	tag
	)

Initializes a memory_desc memory descriptor using ndims, dims, data_type, and format tag.

tag can be mkldnn_format_tag_any, which allows a primitive to define the appropriate memory format. In this case, the format_kind would be set to mkldnn_format_kind_any

Examples:

cpu_cnn_inference_f32.c, cpu_cnn_training_f32.c, and gpu_getting_started.c.

mkldnn_memory_desc_init_submemory()

mkldnn_status_t MKLDNN_API mkldnn_memory_desc_init_submemory	(	mkldnn_memory_desc_t *	memory_desc,
		const mkldnn_memory_desc_t *	parent_memory_desc,
		const mkldnn_dims_t	dims,
		const mkldnn_dims_t	offsets
	)

Initializes a memory_desc for a given parent_memory_desc, with dims sizes and offsets.

May fail if layout used does not allow obtain desired submemory. In this case consider using extract or insert primitive

mkldnn_memory_desc_equal()

int MKLDNN_API mkldnn_memory_desc_equal	(	const mkldnn_memory_desc_t *	lhs,
		const mkldnn_memory_desc_t *	rhs
	)

Compares two memory descriptors.

Returns

1 if the descriptors are the same.

0 if the descriptors are different.

Use this function to identify whether a reorder is required between the two memories

Examples:

cpu_cnn_inference_f32.c, and cpu_cnn_training_f32.c.

mkldnn_memory_create()

mkldnn_status_t MKLDNN_API mkldnn_memory_create	(	mkldnn_memory_t *	memory,
		const mkldnn_memory_desc_t *	memory_desc,
		mkldnn_engine_t	engine,
		void *	handle
	)

Creates a memory for given memory_desc and engine.

Also sets handle to one of the following:

• pointer to the user allocated memory, i.e. valid handle. In this case the library doesn't own allocated memory.
• MKLDNN_MEMORY_ALLOCATE to ask the library to allocate and attach memory. In this case the library owns allocated memory.
• MKLDNN_MEMORY_NONE to create mkldnn_memory w/o attached memory.

Examples:

cpu_cnn_inference_f32.c, cpu_cnn_training_f32.c, and gpu_getting_started.c.

mkldnn_memory_map_data()

mkldnn_status_t MKLDNN_API mkldnn_memory_map_data	(	const_mkldnn_memory_t	memory,
		void **	mapped_ptr
	)

For a memory, maps the data of the memory to mapped_ptr.

Mapping allows to read/write directly from/to the memory contents for engines that do not support direct memory access.

Mapping is an exclusive operation - a memory object cannot be used in other operations until this memory object is unmapped.

Note

Any primitives working with memory should be completed before mapping the memory. Use mkldnn_stream_wait to synchronize the corresponding execution stream.

Map/unmap API is provided mainly for debug/testing purposes and its performance may be suboptimal.

Examples:

gpu_getting_started.c.

mkldnn_memory_unmap_data()

mkldnn_status_t MKLDNN_API mkldnn_memory_unmap_data	(	const_mkldnn_memory_t	memory,
		void *	mapped_ptr
	)

For a memory, unmaps a mapped pointer to the data of the memory.

Any changes of the mapped data are synchronized back to the memory after the call is complete. The mapped pointer must be obtained through a mkldnn_memory_map_data call.

Note

Map/unmap API is provided mainly for debug/testing purposes and its performance may be suboptimal.

Examples:

gpu_getting_started.c.

mkldnn_memory_get_data_handle()

mkldnn_status_t MKLDNN_API mkldnn_memory_get_data_handle	(	const_mkldnn_memory_t	memory,
		void **	handle
	)

For a memory, returns the data handle.

For the CPU engine, the data handle is a pointer to the actual data.

Examples:

cpu_cnn_training_f32.c.