Integrating a Third-Party Library to oneAPI Math Kernel Library (oneMKL) Interfaces

Integrating a Third-Party Library to oneAPI Math Kernel Library (oneMKL) Interfaces#

This step-by-step tutorial provides examples for enabling new third-party libraries in oneMKL.

oneMKL has a header-based implementation of the interface layer (include directory) and a source-based implementation of the backend layer for each third-party library (src directory). To enable a third-party library, you must update both parts of oneMKL and integrate the new third-party library to the oneMKL build and test systems.

For the new backend library and header naming please use the following template:

onemkl_<domain>_<3rd-party library short name>[<wrapper for specific target>]

Where <wrapper for specific target> is required only if multiple wrappers are provided from the same 3rd-party library, e.g., wrappers with Intel oneMKL C API for CPU target onemkl_blas_mklcpu.so and wrappers with Intel oneMKL DPC++ API for GPU target onemkl_blas_mklgpu.so.

If there is no need for multiple wrappers only <domain> and <3rd-party library short name> are required, e.g. onemkl_rng_curand.so

1. Create Header Files

2. Integrate Header Files

3. Create Wrappers

4. Integrate Wrappers To the Build System

5. Update the Test System

1. Create Header Files#

For each new backend library, you should create the following two header files:

  • Header file with a declaration of entry points to the new third-party library wrappers

  • Compiler-time dispatching interface (see oneMKL Usage Models) for new third-party libraries

Header File Example: command to generate the header file with a declaration of BLAS entry points in the oneapi::mkl::newlib namespace

python scripts/generate_backend_api.py include/oneapi/mkl/blas.hpp \                                  # Base header file
                                       include/oneapi/mkl/blas/detail/newlib/onemkl_blas_newlib.hpp \ # Output header file
                                       oneapi::mkl::newlib                                            # Wrappers namespace

Code snippet of the generated header file include/oneapi/mkl/blas/detail/newlib/onemkl_blas_newlib.hpp

namespace oneapi {
namespace mkl {
namespace newlib {

void asum(sycl::queue &queue, std::int64_t n, sycl::buffer<float, 1> &x, std::int64_t incx,
          sycl::buffer<float, 1> &result);

Compile-time Dispatching Interface Example: command to generate the compile-time dispatching interface template instantiations for newlib and supported device newdevice

python scripts/generate_ct_instant.py   include/oneapi/mkl/blas/detail/blas_ct_templates.hpp \         # Base header file
                                        include/oneapi/mkl/blas/detail/newlib/blas_ct.hpp \            # Output header file
                                        include/oneapi/mkl/blas/detail/newlib/onemkl_blas_newlib.hpp \ # Header file with declaration of entry points to wrappers
                                        newlib \                                                       # Library name
                                        newdevice \                                                    # Backend name
                                        oneapi::mkl::newlib                                            # Wrappers namespace

Code snippet of the generated header file include/oneapi/mkl/blas/detail/newlib/blas_ct.hpp

namespace oneapi {
namespace mkl {
namespace blas {

template <>
void asum<library::newlib, backend::newdevice>(sycl::queue &queue, std::int64_t n,
                                               sycl::buffer<float, 1> &x, std::int64_t incx,
                                               sycl::buffer<float, 1> &result) {
    asum_precondition(queue, n, x, incx, result);
    oneapi::mkl::newlib::asum(queue, n, x, incx, result);
    asum_postcondition(queue, n, x, incx, result);
}

2. Integrate Header Files#

Below you can see structure of oneMKL top-level include directory:

include/
    oneapi/
        mkl/
            mkl.hpp -> oneMKL spec APIs
            types.hpp  -> oneMKL spec types
            blas.hpp   -> oneMKL BLAS APIs w/ pre-check/dispatching/post-check
            detail/    -> implementation specific header files
                exceptions.hpp        -> oneMKL exception classes
                backends.hpp          -> list of oneMKL backends
                backends_table.hpp    -> table of backend libraries for each domain and device
                get_device_id.hpp     -> function to query device information from queue for Run-time dispatching
            blas/
                predicates.hpp -> oneMKL BLAS pre-check post-check
                detail/        -> BLAS domain specific implementation details
                    blas_loader.hpp       -> oneMKL Run-time BLAS API
                    blas_ct_templates.hpp -> oneMKL Compile-time BLAS API general templates
                    cublas/
                        blas_ct.hpp            -> oneMKL Compile-time BLAS API template instantiations for <cublas>
                        onemkl_blas_cublas.hpp -> backend wrappers library API
                    mklcpu/
                        blas_ct.hpp            -> oneMKL Compile-time BLAS API template instantiations for <mklcpu>
                        onemkl_blas_mklcpu.hpp -> backend wrappers library API
                    <other backends>/
            <other domains>/

To integrate the new third-party library to a oneMKL header-based part, following files from this structure should be updated:

  • include/oneapi/mkl/detail/backends.hpp: add the new backend

    Example: add the newbackend backend

       enum class backend { mklcpu,
    +                       newbackend,
    
       static backendmap backend_map = { { backend::mklcpu, "mklcpu" },
    +                                    { backend::newbackend, "newbackend" },
    
  • include/oneapi/mkl/detail/backends_table.hpp: add new backend library for supported domain(s) and device(s)

    Example: enable newlib for blas domain and newdevice device

       enum class device : uint16_t { x86cpu,
                                      ...
    +                                 newdevice
                                    };
    
       static std::map<domain, std::map<device, std::vector<const char*>>> libraries = {
           { domain::blas,
             { { device::x86cpu,
                 {
       #ifdef ONEMKL_ENABLE_MKLCPU_BACKEND
                     LIB_NAME("blas_mklcpu")
       #endif
                  } },
    +          { device::newdevice,
    +            {
    +  #ifdef ONEMKL_ENABLE_NEWLIB_BACKEND
    +                 LIB_NAME("blas_newlib")
    +  #endif
    +             } },
    
  • include/oneapi/mkl/detail/get_device_id.hpp: add new device detection mechanism for Run-time dispatching

    Example: enable newdevice if the queue is targeted for the Host

       inline oneapi::mkl::device get_device_id(sycl::queue &queue) {
           oneapi::mkl::device device_id;
    +      if (queue.is_host())
    +          device_id=device::newdevice;
    
  • include/oneapi/mkl/blas.hpp: include the generated header file for the compile-time dispatching interface (see oneMKL Usage Models)

    Example: add include/oneapi/mkl/blas/detail/newlib/blas_ct.hpp generated at the 1. Create Header Files step

       #include "oneapi/mkl/blas/detail/mklcpu/blas_ct.hpp"
       #include "oneapi/mkl/blas/detail/mklgpu/blas_ct.hpp"
    +  #include "oneapi/mkl/blas/detail/newlib/blas_ct.hpp"
    

The new files generated at the 1. Create Header Files step result in the following updated structure of the BLAS domain header files.

include/
    oneapi/
        mkl/
            blas.hpp -> oneMKL BLAS APIs w/ pre-check/dispatching/post-check
            blas/
                predicates.hpp -> oneMKL BLAS pre-check post-check
                detail/        -> BLAS domain specific implementation details
                    blas_loader.hpp       -> oneMKL Run-time BLAS API
                    blas_ct_templates.hpp -> oneMKL Compile-time BLAS API general templates
                    cublas/
                        blas_ct.hpp            -> oneMKL Compile-time BLAS API template instantiations for <cublas>
                        onemkl_blas_cublas.hpp -> backend wrappers library API
                    mklcpu/
                        blas_ct.hpp            -> oneMKL Compile-time BLAS API template instantiations for <mklcpu>
                        onemkl_blas_mklcpu.hpp -> backend wrappers library API
    +              newlib/
    +                  blas_ct.hpp            -> oneMKL Compile-time BLAS API template instantiations for <newbackend>
    +                  onemkl_blas_newlib.hpp -> backend wrappers library API
                    <other backends>/
            <other domains>/

3. Create Wrappers#

Wrappers convert Data Parallel C++ (DPC++) input data types to third-party library data types and call corresponding implementation from the third-party library. Wrappers for each third-party library are built to separate oneMKL backend libraries. The libonemkl.so dispatcher library loads the wrappers at run-time if you are using the interface for run-time dispatching, or you will link with them directly in case you are using the interface for compile-time dispatching (for more information see oneMKL Usage Models).

All wrappers and dispatcher library implementations are in the src directory:

src/
    include/
        function_table_initializer.hpp -> general loader implementation w/ global libraries table
    blas/
        function_table.hpp -> loaded BLAS functions declaration
        blas_loader.cpp -> BLAS wrappers for loader
        backends/
            cublas/ -> cuBLAS wrappers
            mklcpu/ -> Intel oneMKL CPU wrappers
            mklgpu/ -> Intel oneMKL GPU wrappers
            <other backend libraries>/
    <other domains>/

Each backend library should contain a table of all functions from the chosen domain.

scripts/generate_wrappers.py can help to generate wrappers with the “Not implemented” exception for all functions based on the provided header file.

You can modify wrappers generated with this script to enable third-party library functionality.

Example: generate wrappers for newlib based on the header files generated and integrated previously, and enable only one asum function

The command below generates two new files:

  • src/blas/backends/newlib/newlib_wrappers.cpp - DPC++ wrappers for all functions from include/oneapi/mkl/blas/detail/newlib/onemkl_blas_newlib.hpp

  • src/blas/backends/newlib/newlib_wrappers_table_dyn.cpp - structure of symbols for run-time dispatcher (in the same location as wrappers), suffix _dyn indicates that this file is required for dynamic library only.

python scripts/generate_wrappers.py include/oneapi/mkl/blas/detail/newlib/onemkl_blas_newlib.hpp \ # Base header file
                                    src/blas/function_table.hpp \                                  # Declaration for structure of symbols
                                    src/blas/backends/newlib/newlib_wrappers.cpp \                 # Output wrappers
                                    newlib                                                         # Library name

You can then modify src/blas/backends/newlib/newlib_wrappers.cpp to enable the C function newlib_sasum from the third-party library libnewlib.so.

To enable this function:

  • Include the header file newlib.h with the newlib_sasum function declaration

  • Convert all DPC++ parameters to proper C types: use the get_access method for input and output DPC++ buffers to get row pointers

  • Submit the DPC++ kernel with a C function call to newlib as single_task

The following code snippet is updated for src/blas/backends/newlib/newlib_wrappers.cpp:

    #if __has_include(<sycl/sycl.hpp>)
    #include <sycl/sycl.hpp>
    #else
    #include <CL/sycl.hpp>
    #endif

    #include "oneapi/mkl/types.hpp"

    #include "oneapi/mkl/blas/detail/newlib/onemkl_blas_newlib.hpp"
+
+    #include "newlib.h"

    namespace oneapi {
    namespace mkl {
    namespace newlib {

    void asum(sycl::queue &queue, std::int64_t n, sycl::buffer<float, 1> &x, std::int64_t incx,
               sycl::buffer<float, 1> &result) {
-       throw std::runtime_error("Not implemented for newlib");
+       queue.submit([&](sycl::handler &cgh) {
+           auto accessor_x      = x.get_access<sycl::access::mode::read>(cgh);
+           auto accessor_result = result.get_access<sycl::access::mode::write>(cgh);
+           cgh.single_task<class newlib_sasum>([=]() {
+               accessor_result[0] = ::newlib_sasum((const int)n, accessor_x.get_pointer(), (const int)incx);
+           });
+       });
    }

    void asum(sycl::queue &queue, std::int64_t n, sycl::buffer<double, 1> &x, std::int64_t incx,
              sycl::buffer<double, 1> &result) {
        throw std::runtime_error("Not implemented for newlib");
    }

Updated structure of the src folder with the newlib wrappers:

src/
    blas/
        loader.hpp -> general loader implementation w/ global libraries table
        function_table.hpp -> loaded BLAS functions declaration
        blas_loader.cpp -> BLAS wrappers for loader
        backends/
            cublas/ -> cuBLAS wrappers
            mklcpu/ -> Intel oneMKL CPU wrappers
            mklgpu/ -> Intel oneMKL GPU wrappers
 +          newlib/
 +              newlib.h
 +              newlib_wrappers.cpp
 +              newlib_wrappers_table_dyn.cpp
            <other backend libraries>/
    <other domains>/

4. Integrate Wrappers to the Build System#

Here is the list of files that should be created/updated to integrate the new wrappers for the third-party library to the oneMKL build system:

  • Add the new option ENABLE_XXX_BACKEND for the new third-party library to the top of the CMakeList.txt file.

    Example: changes for newlib in the top of the CMakeList.txt file

        option(ENABLE_MKLCPU_BACKEND "" ON)
        option(ENABLE_MKLGPU_BACKEND "" ON)
    +   option(ENABLE_NEWLIB_BACKEND "" ON)
    
  • Add the new directory (src/<domain>/backends/<new_directory>) with the wrappers for the new third-party library under the ENABLE_XXX_BACKEND condition to the src/<domain>/backends/CMakeList.txt file.

    Example: changes for newlib in src/blas/backends/CMakeLists.txt

        if(ENABLE_MKLCPU_BACKEND)
            add_subdirectory(mklcpu)
        endif()
    +
    +   if(ENABLE_NEWLIB_BACKEND)
    +       add_subdirectory(newlib)
    +   endif()
    
  • Create the cmake/FindXXX.cmake cmake config file to find the new third-party library and its dependencies.

    Example: new config file cmake/FindNEWLIB.cmake for newlib

    include_guard()
    # Find library by name in NEWLIB_ROOT cmake variable or environment variable NEWLIBROOT
    find_library(NEWLIB_LIBRARY NAMES newlib
        HINTS ${NEWLIB_ROOT} $ENV{NEWLIBROOT}
        PATH_SUFFIXES "lib")
    # Make sure that the library was found
    include(FindPackageHandleStandardArgs)
    find_package_handle_standard_args(NEWLIB REQUIRED_VARS NEWLIB_LIBRARY)
    # Set cmake target for the library
    add_library(ONEMKL::NEWLIB::NEWLIB UNKNOWN IMPORTED)
    set_target_properties(ONEMKL::NEWLIB::NEWLIB PROPERTIES
        IMPORTED_LOCATION ${NEWLIB_LIBRARY})
    
  • Create the src/<domain>/backends/<new_directory>/CMakeList.txt cmake config file to specify how to build the backend layer for the new third-party library.

    scripts/generate_cmake.py can help to generate the initial src/<domain>/backends/<new_directory>/CMakeList.txt config file automatically for all files in the directory. Note: all source files with the _dyn suffix are added to build if the target is a dynamic library only.

    Example: command to generate the cmake config file for the src/blas/backends/newlib directory

    python scripts/generate_cmake.py src/blas/backends/newlib \ # Full path to the directory
                                     newlib                     # Library name
    

    You should manually update the generated config file with information about the new cmake/FindXXX.cmake file and instructions about how to link with the third-party library.

    Example: update the generated src/blas/backends/newlib/CMakeLists.txt file

        # Add third-party library
    -   # find_package(XXX REQUIRED)
    +   find_package(NEWLIB REQUIRED)
    
        target_link_libraries(${LIB_OBJ}
            PUBLIC ONEMKL::SYCL::SYCL
    -       # Add third-party library to link with here
    +       PUBLIC ONEMKL::NEWLIB::NEWLIB
        )
    

Now you can build the backend library for newlib to make sure the third-party library integration was completed successfully (for more information, see Build with cmake)

cd build/
cmake .. -DNEWLIB_ROOT=<path/to/newlib> \
    -DENABLE_MKLCPU_BACKEND=OFF \
    -DENABLE_MKLGPU_BACKEND=OFF \
    -DENABLE_NEWLIB_BACKEND=ON \           # Enable new third-party library backend
    -DBUILD_FUNCTIONAL_TESTS=OFF           # At this step we want build only
cmake --build . -j4

5. Update the Test System#

Update the following files to enable the new third-party library for unit tests:

  • src/config.hpp.in: add a cmake option for the new third-party library so this macro can be propagated to unit tests

    Example: add ENABLE_NEWLIB_BACKEND

       #cmakedefine ONEMKL_ENABLE_MKLCPU_BACKEND
    +  #cmakedefine ONEMKL_ENABLE_NEWLIB_BACKEND
    
  • tests/unit_tests/CMakeLists.txt: add instructions about how to link tests with the new backend library

    Example: add the newlib backend library

       if(ENABLE_MKLCPU_BACKEND)
           add_dependencies(test_main_ct onemkl_blas_mklcpu)
           if(BUILD_SHARED_LIBS)
               list(APPEND ONEMKL_LIBRARIES onemkl_blas_mklcpu)
           else()
               list(APPEND ONEMKL_LIBRARIES -foffload-static-lib=${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/libonemkl_blas_mklcpu.a)
               find_package(MKL REQUIRED)
               list(APPEND ONEMKL_LIBRARIES ${MKL_LINK_C})
           endif()
       endif()
    +
    +    if(ENABLE_NEWLIB_BACKEND)
    +       add_dependencies(test_main_ct onemkl_blas_newlib)
    +       if(BUILD_SHARED_LIBS)
    +           list(APPEND ONEMKL_LIBRARIES onemkl_blas_newlib)
    +       else()
    +           list(APPEND ONEMKL_LIBRARIES -foffload-static-lib=${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/libonemkl_blas_newlib.a)
    +           find_package(NEWLIB REQUIRED)
    +           list(APPEND ONEMKL_LIBRARIES ONEMKL::NEWLIB::NEWLIB)
    +       endif()
    +   endif()
    
  • tests/unit_tests/include/test_helper.hpp: add the helper function for the compile-time dispatching interface with the new backend, and specify the device for which it should be called

    Example: add the helper function for the newlib compile-time dispatching interface with newdevice if it is the Host

       #ifdef ONEMKL_ENABLE_MKLGPU_BACKEND
           #define TEST_RUN_INTELGPU(q, func, args) \
               func<oneapi::mkl::backend::mklgpu> args
       #else
           #define TEST_RUN_INTELGPU(q, func, args)
       #endif
    +
    +  #ifdef ONEMKL_ENABLE_NEWLIB_BACKEND
    +     #define TEST_RUN_NEWDEVICE(q, func, args) \
    +         func<oneapi::mkl::backend::newbackend> args
    +  #else
    +      #define TEST_RUN_NEWDEVICE(q, func, args)
    +  #endif
    
       #define TEST_RUN_CT(q, func, args)               \
           do {                                         \
    +          if (q.is_host())                         \
    +              TEST_RUN_NEWDEVICE(q, func, args);   \
    
  • tests/unit_tests/main_test.cpp: add the targeted device to the vector of devices to test

    Example: add the targeted device CPU for newlib

               }
           }
    +
    +  #ifdef ONEMKL_ENABLE_NEWLIB_BACKEND
    +      devices.push_back(sycl::device(sycl::host_selector()));
    +  #endif
    

Now you can build and run functional testing for enabled third-party libraries (for more information see Build with cmake).

cd build/
cmake .. -DNEWLIB_ROOT=<path/to/newlib> \
    -DENABLE_MKLCPU_BACKEND=OFF \
    -DENABLE_MKLGPU_BACKEND=OFF \
    -DENABLE_NEWLIB_BACKEND=ON  \
    -DBUILD_FUNCTIONAL_TESTS=ON
cmake --build . -j4
ctest