Memory¶

Overview¶

A container that describes and stores data. More…

// typedefs

typedef int64_t dnnl_dim_t;
typedef dnnl_dim_t dnnl_dims_t[DNNL_MAX_NDIMS];
typedef struct dnnl_memory* dnnl_memory_t;
typedef const struct dnnl_memory* const_dnnl_memory_t;

// enums

enum dnnl_data_type_t;
enum dnnl_format_kind_t;
enum dnnl_format_tag_t;
enum dnnl_memory_extra_flags_t;
enum dnnl_rnn_packed_memory_format_t;
enum dnnl_wino_memory_format_t;

// structs

struct dnnl_blocking_desc_t;
struct dnnl_memory;
struct dnnl_memory_desc_t;
struct dnnl_memory_extra_desc_t;
struct dnnl_rnn_packed_desc_t;
struct dnnl_wino_desc_t;
struct dnnl::memory;

// global functions

bool dnnl::operator == (dnnl_data_type_t a, memory::data_type b);
bool dnnl::operator != (dnnl_data_type_t a, memory::data_type b);
bool dnnl::operator == (memory::data_type a, dnnl_data_type_t b);
bool dnnl::operator != (memory::data_type a, dnnl_data_type_t b);
bool dnnl::operator == (dnnl_format_tag_t a, memory::format_tag b);
bool dnnl::operator != (dnnl_format_tag_t a, memory::format_tag b);
bool dnnl::operator == (memory::format_tag a, dnnl_format_tag_t b);
bool dnnl::operator != (memory::format_tag a, dnnl_format_tag_t b);

dnnl_status_t DNNL_API dnnl_memory_desc_init_by_strides(
    dnnl_memory_desc_t* memory_desc,
    int ndims,
    const dnnl_dims_t dims,
    dnnl_data_type_t data_type,
    const dnnl_dims_t strides
    );

dnnl_status_t DNNL_API dnnl_memory_desc_init_by_tag(
    dnnl_memory_desc_t* memory_desc,
    int ndims,
    const dnnl_dims_t dims,
    dnnl_data_type_t data_type,
    dnnl_format_tag_t tag
    );

dnnl_status_t DNNL_API dnnl_memory_desc_init_submemory(
    dnnl_memory_desc_t* memory_desc,
    const dnnl_memory_desc_t* parent_memory_desc,
    const dnnl_dims_t dims,
    const dnnl_dims_t offsets
    );

dnnl_status_t DNNL_API dnnl_memory_desc_reshape(
    dnnl_memory_desc_t* out_memory_desc,
    const dnnl_memory_desc_t* in_memory_desc,
    int ndims,
    const dnnl_dims_t dims
    );

dnnl_status_t DNNL_API dnnl_memory_desc_permute_axes(
    dnnl_memory_desc_t* out_memory_desc,
    const dnnl_memory_desc_t* in_memory_desc,
    const int* permutation
    );

int DNNL_API dnnl_memory_desc_equal(
    const dnnl_memory_desc_t* lhs,
    const dnnl_memory_desc_t* rhs
    );

size_t DNNL_API dnnl_memory_desc_get_size(const dnnl_memory_desc_t* memory_desc);
size_t DNNL_API dnnl_data_type_size(dnnl_data_type_t data_type);

dnnl_status_t DNNL_API dnnl_memory_create(
    dnnl_memory_t* memory,
    const dnnl_memory_desc_t* memory_desc,
    dnnl_engine_t engine,
    void* handle
    );

dnnl_status_t DNNL_API dnnl_memory_get_memory_desc(
    const_dnnl_memory_t memory,
    const dnnl_memory_desc_t** memory_desc
    );

dnnl_status_t DNNL_API dnnl_memory_get_engine(
    const_dnnl_memory_t memory,
    dnnl_engine_t* engine
    );

dnnl_status_t DNNL_API dnnl_memory_map_data(
    const_dnnl_memory_t memory,
    void** mapped_ptr
    );

dnnl_status_t DNNL_API dnnl_memory_unmap_data(
    const_dnnl_memory_t memory,
    void* mapped_ptr
    );

dnnl_status_t DNNL_API dnnl_memory_get_data_handle(
    const_dnnl_memory_t memory,
    void** handle
    );

dnnl_status_t DNNL_API dnnl_memory_set_data_handle(
    dnnl_memory_t memory,
    void* handle
    );

dnnl_status_t DNNL_API dnnl_memory_set_data_handle_v2(
    dnnl_memory_t memory,
    void* handle,
    dnnl_stream_t stream
    );

dnnl_status_t DNNL_API dnnl_memory_destroy(dnnl_memory_t memory);

// macros

#define DNNL_MAX_NDIMS
#define DNNL_MEMORY_ALLOCATE
#define DNNL_MEMORY_NONE
#define DNNL_RNN_MAX_N_PARTS
#define DNNL_RUNTIME_DIM_VAL
#define DNNL_RUNTIME_F32_VAL
#define DNNL_RUNTIME_S32_VAL
#define DNNL_RUNTIME_SIZE_VAL

Detailed Documentation¶

A container that describes and stores data.

Memory objects can contain data of various types and formats. There are two levels of abstraction:

Memory descriptor engine-agnostic logical description of data (number of dimensions, dimension sizes, and data type), and, optionally, the information about the physical format of data in memory. If this information is not known yet, a memory descriptor can be created with dnnl::memory::format_tag::any. This allows compute-intensive primitives to choose the best format for computation. The user is responsible for reordering the data into the chosen format when formats do not match.

A memory descriptor can be initialized either by specifying dimensions and a memory format tag or strides for each of them, or by manipulating the dnnl_memory_desc_t structure directly.

Warning

The latter approach requires understanding how the physical data representation is mapped to the structure and is discouraged. This topic is discussed in Understanding Memory Formats.

The user can query the amount of memory required by a memory descriptor using the dnnl::memory::desc::get_size() function. The size of data in general cannot be computed as the product of dimensions multiplied by the size of the data type. So users are required to use this function for better code portability.

Two memory descriptors can be compared using the equality and inequality operators. The comparison is especially useful when checking whether it is necessary to reorder data from the user’s data format to a primitive’s format.
Memory object an engine-specific object that handles the memory buffer and its description (a memory descriptor). For the CPU engine or with USM, the memory buffer handle is simply a pointer to void. The memory buffer can be queried using dnnl::memory::get_data_handle() and set using dnnl::memory::set_data_handle(). The underlying SYCL buffer, when used, can be queried using dnnl::sycl_interop::get_buffer and set using dnnl::sycl_interop::set_buffer. A memory object can also be queried for the underlying memory descriptor and for its engine using dnnl::memory::get_desc() and dnnl::memory::get_engine().

Along with ordinary memory descriptors with all dimensions being positive, the library supports zero-volume memory descriptors with one or more dimensions set to zero. This is used to support the NumPy* convention. If a zero-volume memory is passed to a primitive, the primitive typically does not perform any computations with this memory. For example:

A concatenation primitive would ignore all memory object with zeroes in the concat dimension / axis.
A forward convolution with a source memory object with zero in the minibatch dimension would always produce a destination memory object with a zero in the minibatch dimension and perform no computations.
However, a forward convolution with a zero in one of the weights dimensions is ill-defined and is considered to be an error by the library because there is no clear definition of what the output values should be.

Memory buffer of a zero-volume memory is never accessed.

Typedefs¶

typedef int64_t dnnl_dim_t

A type to describe tensor dimension.

typedef dnnl_dim_t dnnl_dims_t[DNNL_MAX_NDIMS]

A type to describe tensor dimensions.

typedef struct dnnl_memory* dnnl_memory_t

A memory handle.

typedef const struct dnnl_memory* const_dnnl_memory_t

A constant memory handle.

Global Functions¶

dnnl_status_t DNNL_API dnnl_memory_desc_init_by_strides(
    dnnl_memory_desc_t* memory_desc,
    int ndims,
    const dnnl_dims_t dims,
    dnnl_data_type_t data_type,
    const dnnl_dims_t strides
    )

Initializes a memory descriptor using dimensions and strides.

Note

As always, the logical order of dimensions corresponds to the abc... format tag, and the physical meaning of the dimensions depends on both the primitive that consumes the memory and the context of that consumption.

Parameters:

memory_desc	Output memory descriptor.
ndims	Number of dimensions
dims	Array of dimensions.
data_type	Elements data type.
strides	Strides in each dimension.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_memory_desc_init_by_tag(
    dnnl_memory_desc_t* memory_desc,
    int ndims,
    const dnnl_dims_t dims,
    dnnl_data_type_t data_type,
    dnnl_format_tag_t tag
    )

Initializes a memory descriptor using dimensions and memory format tag.

Note

Parameters:

memory_desc	Output memory descriptor.
ndims	Number of dimensions
dims	Array of dimensions.
data_type	Elements data type.
tag	Memory format tag. Can be dnnl_format_tag_any which would allow a primitive to chose the final memory format. In this case the format_kind field of the memory descriptor would be set to dnnl_format_kind_any.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_memory_desc_init_submemory(
    dnnl_memory_desc_t* memory_desc,
    const dnnl_memory_desc_t* parent_memory_desc,
    const dnnl_dims_t dims,
    const dnnl_dims_t offsets
    )

Initializes a memory descriptor for a region inside an area described by an existing memory descriptor.

Warning

Some combinations of physical memory layout and/or offsets or dims may result in a failure to create a submemory.

Parameters:

memory_desc	Output memory descriptor.
parent_memory_desc	An existing memory descriptor.
dims	Sizes of the region.
offsets	Offsets to the region from the encompassing memory object in each dimension

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_memory_desc_reshape(
    dnnl_memory_desc_t* out_memory_desc,
    const dnnl_memory_desc_t* in_memory_desc,
    int ndims,
    const dnnl_dims_t dims
    )

Initializes a memory descriptor by reshaping an existing one.

The new memory descriptor inherits the data type. This operation is valid only for memory descriptors that have format_kind set to dnnl_blocked or dnnl_format_kind_any.

The operation ensures the transformation of the physical memory format corresponds to the transformation of the logical dimensions. If such transformation is impossible, the function returns dnnl_invalid_arguments.

The reshape operation can be described as a combination of the following basic operations:

Add a dimension of size 1. This is always possible.
Remove a dimension of size 1. This is possible only if the dimension has no padding (i.e. padded_dims[dim] == dims[dim] && dims[dim] == 1).
Split a dimension into multiple ones. This is possible only if the size of the dimension is exactly equal to the product of the split ones and the dimension does not have padding (i.e. padded_dims[dim] = dims[dim]).
Joining multiple consecutive dimensions into a single one. As in the cases above, this requires that the dimensions do not have padding and that the memory format is such that in physical memory these dimensions are dense and have the same order as their logical counterparts. This also assumes that these dimensions are not blocked.
- Here, dense means: stride for dim[i] == (stride for dim[i + 1]) * dim[i + 1];
- And same order means: i < j if and only if stride for dim[j] <= stride for dim[i].

Warning

Some combinations of physical memory layout and/or offsets or dimensions may result in a failure to make a reshape.

Parameters:

out_memory_desc	Output memory descriptor.
in_memory_desc	An existing memory descriptor. Must have format_kind set to dnnl_blocked or dnnl_format_kind_any.
ndims	Number of dimensions for the output memory descriptor.
dims	Dimensions for the output memory descriptor.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_memory_desc_permute_axes(
    dnnl_memory_desc_t* out_memory_desc,
    const dnnl_memory_desc_t* in_memory_desc,
    const int* permutation
    )

Initializes a memory descriptor by permuting axes in an existing one.

The physical memory layout representation is adjusted accordingly to maintain the consistency between the logical and physical parts of the memory descriptor.

The new memory descriptor inherits the data type. This operation is valid only for memory descriptors that have format_kind set to dnnl_blocked or dnnl_format_kind_any.

The logical axes will be permuted in the following manner:

for (i: 0 .. in_memory_desc->ndims)
    out_memory_desc->dims[permutation[i]] = in_memory_desc->dims[i];

Example:

dnnl_memory_desc_t in_md, out_md, expect_out_md;

const int permutation[] = {1, 0}; // swap the first and the second axes

dnnl_dims_t in_dims = {2, 3}, out_dims = {3, 2};
dnnl_format_tag_t in_tag = dnnl_ab, out_tag = dnnl_ba;

dnnl_memory_desc_init_by_tag(
        &in_md, 2, in_dims, data_type, in_tag);
dnnl_memory_desc_init_by_tag(
        &expect_out_md, 2, out_dims, data_type, out_tag);

dnnl_memory_desc_permute_axes(&out_md, in_md, permutation);
assert(dnnl_memory_desc_equal(&out_md, &expect_out_md));

Parameters:

out_memory_desc	Output memory descriptor.
in_memory_desc	An existing memory descriptor. Must have format_kind set to dnnl_blocked or dnnl_format_kind_any.
permutation	Axes permutation (of size `in_memory_desc->ndims`).

Returns:

dnnl_success on success and a status describing the error otherwise.

int DNNL_API dnnl_memory_desc_equal(
    const dnnl_memory_desc_t* lhs,
    const dnnl_memory_desc_t* rhs
    )

Compares two memory descriptors.

Use this function to identify whether a reorder is required between the two memories

Parameters:

lhs	Left-hand side of the comparison.
rhs	Right-hand side of the comparison.

Returns:

1 if the descriptors are the same.

0 if the descriptors are different.

size_t DNNL_API dnnl_memory_desc_get_size(const dnnl_memory_desc_t* memory_desc)

Returns the size of a memory descriptor.

Parameters:

memory_desc

Memory descriptor.

Returns:

The number of bytes required for memory described by a memory descriptor.

size_t DNNL_API dnnl_data_type_size(dnnl_data_type_t data_type)

Returns the size of data type.

Parameters:

data_type

Data type.

Returns:

The number of bytes occupied by data type.

dnnl_status_t DNNL_API dnnl_memory_create(
    dnnl_memory_t* memory,
    const dnnl_memory_desc_t* memory_desc,
    dnnl_engine_t engine,
    void* handle
    )

Creates a memory object.

Unless handle is equal to DNNL_MEMORY_NONE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if dnnl_memory_set_data_handle() had been called.

Parameters:

memory	Output memory object.
memory_desc	Memory descriptor.
engine	Engine to use.
handle	Handle of the memory buffer to use as an underlying storage. A pointer to the user-allocated buffer. In this case the library doesn’t own the buffer. The DNNL_MEMORY_ALLOCATE special value. Instructs the library to allocate the buffer for the memory object. In this case the library owns the buffer. DNNL_MEMORY_NONE to create dnnl_memory without an underlying buffer.

Returns:

dnnl_success on success and a status describing the error otherwise.

Macros¶

#define DNNL_MAX_NDIMS

Maximum number of dimensions a tensor can have.

Only restricts the amount of space used for the tensor description. Individual computational primitives may support only tensors of certain dimensions.

#define DNNL_MEMORY_ALLOCATE

Special pointer value that indicates that the library needs to allocate an underlying buffer for a memory object.

#define DNNL_MEMORY_NONE

Special pointer value that indicates that a memory object should not have an underlying buffer.

#define DNNL_RNN_MAX_N_PARTS

Maximum number of parts of RNN weights tensor that require separate computation.

#define DNNL_RUNTIME_DIM_VAL

A wildcard value for dimensions that are unknown at a primitive creation time.

#define DNNL_RUNTIME_F32_VAL

A wildcard value for floating point values that are unknown at a primitive creation time.

#define DNNL_RUNTIME_S32_VAL

A wildcard value for int32_t values that are unknown at a primitive creation time.

#define DNNL_RUNTIME_SIZE_VAL

A size_t counterpart of the DNNL_RUNTIME_DIM_VAL.

For instance, this value is returned by dnnl_memory_desc_get_size() if either of the dimensions or strides equal to DNNL_RUNTIME_DIM_VAL.

struct dnnl::reduction::primitive_desc enum dnnl_data_type_t

memory	Memory object.
memory_desc	Output memory descriptor (a copy).

memory	Memory object.
mapped_ptr	Output pointer to the mapped buffer.

oneDNN v2.5.4 documentation

Memory¶

Overview¶

Detailed Documentation¶

Typedefs¶

Global Functions¶

Macros¶