repository

Contents

Compiled Partition

Overview

A compiled partition represents the generated kernels specialized for a partition on a target hardware (engine) with input and output information specified by the logical tensors. More…

// typedefs

typedef struct dnnl_graph_compiled_partition* dnnl_graph_compiled_partition_t;
typedef const struct dnnl_graph_compiled_partition* const_dnnl_graph_compiled_partition_t;

// structs

struct dnnl_graph_inplace_pair_t;

// classes

class dnnl::graph::compiled_partition;

// global functions

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_create(
    dnnl_graph_compiled_partition_t* compiled_partition,
    dnnl_graph_partition_t partition
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_execute(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_stream_t stream,
    size_t num_inputs,
    const_dnnl_graph_tensor_t* inputs,
    size_t num_outputs,
    const_dnnl_graph_tensor_t* outputs
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_destroy(dnnl_graph_compiled_partition_t compiled_partition);

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_query_logical_tensor(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t tid,
    dnnl_graph_logical_tensor_t* lt
    );

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_get_inplace_ports(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t* num_inplace_pairs,
    const dnnl_graph_inplace_pair_t** inplace_pairs
    );

Detailed Documentation

A compiled partition represents the generated kernels specialized for a partition on a target hardware (engine) with input and output information specified by the logical tensors.

Typedefs

typedef struct dnnl_graph_compiled_partition* dnnl_graph_compiled_partition_t

A compiled partition handle.

typedef const struct dnnl_graph_compiled_partition* const_dnnl_graph_compiled_partition_t

A constant compiled partition handle.

Global Functions

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_create(
    dnnl_graph_compiled_partition_t* compiled_partition,
    dnnl_graph_partition_t partition
    )

Creates a new compiled partition handle.

Parameters:

compiled_partition	The handle of output compiled partition.
partition	The handle of input partition.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_execute(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    dnnl_stream_t stream,
    size_t num_inputs,
    const_dnnl_graph_tensor_t* inputs,
    size_t num_outputs,
    const_dnnl_graph_tensor_t* outputs
    )

Executes a compiled partition.

Parameters:

compiled_partition	The handle of target compiled partition.
stream	The stream used for execution.
num_inputs	The number of input tensors.
inputs	A list of input tensors.
num_outputs	The number of output tensors.
outputs	A non-empty list of output tensors.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_destroy(dnnl_graph_compiled_partition_t compiled_partition)

Destroys a compiled partition.

Parameters:

compiled_partition

The compiled partition to be destroyed.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_query_logical_tensor(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t tid,
    dnnl_graph_logical_tensor_t* lt
    )

Queries an input or output logical tensor according to tensor ID.

If the tensor ID doesn’t belong to any input or output of the compiled partition, an error status dnnl_invalid_arguments will be returned by the API.

Parameters:

compiled_partition	The handle of target compiled_partition.
tid	The unique id of required tensor.
lt	The output logical tensor.

Returns:

dnnl_success on success or a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_graph_compiled_partition_get_inplace_ports(
    const_dnnl_graph_compiled_partition_t compiled_partition,
    size_t* num_inplace_pairs,
    const dnnl_graph_inplace_pair_t** inplace_pairs
    )

Returns the hint of in-place pairs from a compiled partition.

It indicates that an input and an output of the partition can share the same memory buffer for computation. In-place computation helps to reduce the memory footprint and improves cache locality. But since the library may not have a global view of user’s application, it’s possible that the tensor with input_id is used at other places in user’s computation graph. In this case, the user should take the in-place pair as a hint and pass a different memory buffer for output tensor to avoid overwriting the input memory buffer which will probably cause unexpected incorrect results.

Parameters:

compiled_partition	The handle of target compiled_partition.
num_inplace_pairs	The number of in-place pairs.
inplace_pairs	The handle of in-place pairs.

Returns:

dnnl_success on success or a status describing the error otherwise.

enum dnnl::graph::partition::policy struct dnnl_graph_inplace_pair_t