OpenCL interoperability API

Overview

API extensions to interact with the underlying OpenCL run-time. More…

// namespaces

namespace dnnl::ocl_interop;

// enums

enum dnnl_ocl_interop_memory_kind_t;

// global functions

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_create(
    dnnl_memory_t* memory,
    const_dnnl_memory_desc_t memory_desc,
    dnnl_engine_t engine,
    dnnl_ocl_interop_memory_kind_t memory_kind,
    void* handle
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_get_memory_kind(
    const_dnnl_memory_t memory,
    dnnl_ocl_interop_memory_kind_t* memory_kind
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_get_mem_object(
    const_dnnl_memory_t memory,
    cl_mem* mem_object
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_set_mem_object(
    dnnl_memory_t memory,
    cl_mem mem_object
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_get_cache_blob_id(
    cl_device_id device,
    size_t* size,
    uint8_t* cache_blob_id
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_get_cache_blob(
    dnnl_engine_t engine,
    size_t* size,
    uint8_t* cache_blob
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_create_from_cache_blob(
    dnnl_engine_t* engine,
    cl_device_id device,
    cl_context context,
    size_t size,
    const uint8_t* cache_blob
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_create(
    dnnl_engine_t* engine,
    cl_device_id device,
    cl_context context
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_get_context(
    dnnl_engine_t engine,
    cl_context* context
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_get_device(
    dnnl_engine_t engine,
    cl_device_id* device
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_stream_create(
    dnnl_stream_t* stream,
    dnnl_engine_t engine,
    cl_command_queue queue
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_stream_get_command_queue(
    dnnl_stream_t stream,
    cl_command_queue* queue
    );

dnnl_status_t DNNL_API dnnl_ocl_interop_primitive_execute(
    const_dnnl_primitive_t primitive,
    dnnl_stream_t stream,
    int nargs,
    const dnnl_exec_arg_t* args,
    const cl_event* deps,
    int ndeps,
    cl_event* return_event
    );

Detailed Documentation

API extensions to interact with the underlying OpenCL run-time.

See also:

OpenCL Interoperability in developer guide

Global Functions

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_create(
    dnnl_memory_t* memory,
    const_dnnl_memory_desc_t memory_desc,
    dnnl_engine_t engine,
    dnnl_ocl_interop_memory_kind_t memory_kind,
    void* handle
    )

Creates a memory object.

Unless handle is equal to DNNL_MEMORY_NONE or DNNL_MEMORY_ALLOCATE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if:

Parameters:

memory

Output memory object.

memory_desc

Memory descriptor.

engine

Engine to use.

memory_kind

Memory allocation kind to specify the type of handle.

handle

Handle of the memory buffer to use as an underlying storage.

  • A USM pointer to the user-allocated buffer. In this case the library doesn’t own the buffer. Requires memory_kind to be equal to dnnl_ocl_interop_usm.

  • An OpenCL buffer. In this case the library doesn’t own the buffer. Requires memory_kind be equal to be equal to dnnl_ocl_interop_buffer.

  • The DNNL_MEMORY_ALLOCATE special value. Instructs the library to allocate the buffer that corresponds to the memory allocation kind memory_kind for the memory object. In this case the library owns the buffer.

  • The DNNL_MEMORY_NONE specific value. Instructs the library to create memory object without an underlying buffer.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_get_memory_kind(
    const_dnnl_memory_t memory,
    dnnl_ocl_interop_memory_kind_t* memory_kind
    )

Returns the memory allocation kind associated with a memory object.

Parameters:

memory

Memory to query.

memory_kind

Output underlying memory allocation kind of the memory object.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_get_mem_object(
    const_dnnl_memory_t memory,
    cl_mem* mem_object
    )

Returns an OpenCL memory object associated with a memory object.

Parameters:

memory

Memory object.

mem_object

Output OpenCL memory object.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_memory_set_mem_object(
    dnnl_memory_t memory,
    cl_mem mem_object
    )

Sets OpenCL memory object associated with a memory object.

For behavioral details, see dnnl_memory_set_data_handle().

Parameters:

memory

Memory object.

mem_object

OpenCL memory object.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_get_cache_blob_id(
    cl_device_id device,
    size_t* size,
    uint8_t* cache_blob_id
    )

Retrieves a cache blob ID for the OpenCL device.

Warning

This API is intended to be used with dnnl_ocl_interop_engine_get_cache_blob() and dnnl_ocl_interop_engine_create_from_cache_blob(). The returned cache blob ID can only be used as an ID of the cache blob returned by dnnl_ocl_interop_engine_get_cache_blob().

Note

The cache blob ID can be empty (size will be 0 and cache_blob_id will be nullptr) if oneDNN doesn’t have anything to put in the cache blob. (dnnl_ocl_interop_engine_get_cache_blob will return an empty cache blob).

Parameters:

device

An OpenCL device.

size

Size of the cache blob ID in bytes.

cache_blob_id

Cache blob id of size size. If the cache_blob_id is nullptr then the size of the cache blob ID is returned in size.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_get_cache_blob(
    dnnl_engine_t engine,
    size_t* size,
    uint8_t* cache_blob
    )

Retrieves a cache blob associated with the given engine.

Note

The cache blob can be empty (size will be 0 and cache_blob will be nullptr) if oneDNN doesn’t have anything to put in the cache blob. It’s the user’s responsibility to check whether it’s empty prior to passing it to dnnl_ocl_interop_engine_create_from_cache_blob().

Parameters:

engine

Engine to query for the cache blob.

size

Size of the cache blob in bytes.

cache_blob

Cache blob of size size. If the cache_blob is nullptr then the size of the cache blob is returned in size.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_create_from_cache_blob(
    dnnl_engine_t* engine,
    cl_device_id device,
    cl_context context,
    size_t size,
    const uint8_t* cache_blob
    )

Creates an engine from the given cache blob.

Parameters:

engine

Output engine.

device

The OpenCL device that this engine will encapsulate.

context

The OpenCL context (containing the device) that this engine will use for all operations.

size

Size of the cache blob in bytes.

cache_blob

Cache blob of size size.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_create(
    dnnl_engine_t* engine,
    cl_device_id device,
    cl_context context
    )

Creates an engine associated with an OpenCL device and an OpenCL context.

Parameters:

engine

Output engine.

device

Underlying OpenCL device to use for the engine.

context

Underlying OpenCL context to use for the engine.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_engine_get_context(
    dnnl_engine_t engine,
    cl_context* context
    )

Returns the OpenCL context associated with an engine.

Parameters:

engine

Engine to query.

context

Output underlying OpenCL context of the engine.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_get_device(
    dnnl_engine_t engine,
    cl_device_id* device
    )

Returns the OpenCL device associated with an engine.

Parameters:

engine

Engine to query.

device

Output underlying OpenCL device of the engine.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_stream_create(
    dnnl_stream_t* stream,
    dnnl_engine_t engine,
    cl_command_queue queue
    )

Creates an execution stream for a given engine associated with an OpenCL command queue.

Parameters:

stream

Output execution stream.

engine

Engine to create the execution stream on.

queue

OpenCL command queue to use.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_stream_get_command_queue(
    dnnl_stream_t stream,
    cl_command_queue* queue
    )

Returns the OpenCL command queue associated with an execution stream.

Parameters:

stream

Execution stream to query.

queue

Output OpenCL command queue.

Returns:

dnnl_success on success and a status describing the error otherwise.

dnnl_status_t DNNL_API dnnl_ocl_interop_primitive_execute(
    const_dnnl_primitive_t primitive,
    dnnl_stream_t stream,
    int nargs,
    const dnnl_exec_arg_t* args,
    const cl_event* deps,
    int ndeps,
    cl_event* return_event
    )

Executes computations specified by the primitive in a specified stream and returns an OpenCL event.

Parameters:

primitive

Primitive to execute.

stream

Stream to use.

nargs

Number of arguments.

args

Array of arguments. Each argument is an <index, dnnl_memory_t> pair. The index is one of the DNNL_ARG_* values such as DNNL_ARG_SRC. Unless runtime shapes are used (see DNNL_RUNTIME_DIM_VAL), the memory object must have the same memory descriptor as that returned by dnnl_primitive_desc_query_md (dnnl_query_exec_arg_md, index).

deps

A pointer to a vector of size ndeps that contains dependencies.

ndeps

Number of dependencies.

return_event

Output event. It’s the user’s responsibility to manage lifetime of the event. Can be NULL. When stream is in-order NULL will be returned.

Returns:

dnnl_success on success and a status describing the error otherwise.