Library Setup, Exit, and Query Routines
Contents
Library Setup, Exit, and Query Routines¶
The library setup, exit, and query interfaces initialize and monitor the parallel environment of the PEs.
ISHMEM_INIT¶
-
void ishmem_init(void)¶
- Parameters
None.
- Returns
None.
Callable from the host.
Description:
Initializes the ishmem library.
ishmem_init and
ishmemx_init_attr are collective operations that
all PEs must call before any other ishmem routine may be called.
At the end of the ishmem program which it initialized, the call to
ishmem_init must be matched with a call to ishmem_finalize.
After the first call to ishmem_init, a subsequent call to
ishmem_init or ishmemx_init_attr in the same program results in
undefined behavior.
ISHMEMX_INIT_ATTR¶
Callable from the host.
Description: Indicates which runtime is used to initialize Intel® SHMEM: either OpenSHMEM, MPI, or PMI.
Important
As of Intel® SHMEM 1.5.2 only ISHMEM_RUNTIME_OPENSHMEM and ISHMEM_RUNTIME_MPI are supported.
Description:
Opaque unique ID used by ishmemx_init_attr when
ishmemx_attr_t.use_uid is enabled. The value is obtained using
ishmemx_get_uniqueid.
-
struct ishmemx_attr_t¶
-
ishmemx_runtime_type_t runtime¶
-
bool initialize_runtime = true¶
-
bool gpu = true¶
-
int device_idx = -1¶
-
bool use_uid = false¶
-
int nranks = -1¶
-
int rank = -1¶
-
ishmemx_uniqueid_t *uid = nullptr¶
-
void *mpi_comm¶
-
ishmemx_runtime_type_t runtime¶
Description:
A struct declaration describing attributes for initialization.
A valid runtime enumeration value must be set by the user and must
correspond to a runtime that is enabled within the build of the ishmem
library.
By default, the parallel runtime is initialized by Intel® SHMEM
(initialize_runtime default is true).
The gpu boolean indicates whether to use GPU memory for the symmetric
heap (default is true). device_idx selects the visible GPU ordinal to
use when multiple GPU devices are visible to a PE. The default value of -1
preserves the traditional single-visible-device behavior: Intel® SHMEM will
auto-select the device only when exactly one GPU is visible. mpi_comm is a
pointer to the corresponding
MPI communicator for representing ISHMEM_TEAM_WORLD when used with
ISHMEM_RUNTIME_MPI (default is MPI_COMM_WORLD).
use_uid enables UID-based initialization for launcher-independent MPI
workflows. This mode is only supported with ISHMEM_RUNTIME_MPI and requires
nranks > 0, 0 <= rank < nranks, and uid != nullptr.
When Intel® SHMEM initializes MPI itself, MPI_THREAD_MULTIPLE is required.
When MPI has already been initialized externally, Intel® SHMEM verifies the
existing MPI thread level and ignores rank/nranks if they are provided.
-
void ishmemx_init_attr(ishmemx_attr_t *attr)¶
- Parameters
attr – a struct of type ishmemx_attr_t specifying initialization attributes
- Returns
None.
Callable from the host.
Description:
Initializes the ishmem library.
ishmem_init and
ishmemx_init_attr are collective operations that
all PEs must call before any other ishmem routine may be called.
At the end of the ishmem program which it initialized, the call to
ishmem_init must be matched with a call to ishmem_finalize.
After the first call to ishmemx_init_attr, a subsequent call to
ishmemx_init_attr or ishmem_init in the same program results in
undefined behavior.
-
int ishmemx_get_uniqueid(ishmemx_uniqueid_t *uid)¶
- Parameters
uid – pointer to ishmemx_uniqueid_t that receives a unique ID
- Returns
0on success, nonzero on failure.
Callable from the host.
Description:
Builds a unique ID used with ishmemx_attr_t.use_uid.
The server host is taken from I_MPI_MPCP_SERVER_NAME when set; otherwise,
MASTER_ADDR is used.
I_MPI_MPCP_SERVER_PORT is used as the server port when set. If it is not
set, Intel® SHMEM uses 35555 by default.
When MASTER_PORT is set, I_MPI_MPCP_SERVER_PORT must be different from
MASTER_PORT.
ISHMEM_MY_PE¶
-
int ishmem_my_pe(void)¶
- Parameters
None.
- Returns
The PE number.
Callable from the host and device.
Description: This routine returns the PE number of the calling PE. The result is an integer between 0 and npes - 1, where npes is the total number of PEs executing the current program.
ISHMEM_N_PES¶
-
int ishmem_n_pes(void)¶
- Parameters
None.
- Returns
The number of total PEs running in the program.
Callable from the host and device.
Description: The routine returns the number of PEs running in the program.
ISHMEM_FINALIZE¶
-
void ishmem_finalize(void)¶
- Parameters
None.
- Returns
None.
Callable from the host.
Description:
ishmem_finalize is a collective operation that ends the ishmem
portion of a program previously initialized by
ishmem_init or
ishmemx_init_attr and releases all resources used
by the ishmem library.
This collective operation requires all PEs to participate in the call.
There is an implicit global barrier in ishmem_finalize to ensure that
pending communications are completed and that no resources are released until
all PEs have entered ishmem_finalize.
This routine destroys all teams created by the ishmem program.
ishmem_finalize must be the last ishmem library call encountered in
the ishmem portion of a program.
A call to ishmem_finalize will release all resources initialized by a
corresponding call to ishmem_init or ishmemx_init_attr. All
processes that represent the PEs will still exist after the call to
ishmem_finalize returns, but they will no longer have access to resources
that have been released.
Note
Because SYCL kernel execution is non-blocking on the host, all
kernels performing ishmem calls must first complete (for example, by
calling wait or wait_and_throw on the SYCL queue) before calling
ishmem_finalize.
ISHMEM_QUERY_INITIALIZED¶
-
void ishmem_query_initialized(int *initialized)¶
- Parameters
initialized – Nonzero if the library is in the initialized state. Zero otherwise.
- Returns
None.
Callable from the host.
Description:
The ishmem_query_initialized call returns the initialization status of the
ishmem library. If the application has called an initialization routine and
has not yet made the corresponding call to ishmem_finalize, this routine
returns nonzero. Otherwise, it returns zero.
This function may be called at any time, regardless of the thread safety level
or the current initialized state of the library.
ISHMEMX_MY_DEVICE¶
-
sycl::device ishmemx_my_device(void)¶
- Parameters
None.
- Returns
The SYCL device selected for the calling PE.
Callable from the host.
Description:
Returns the SYCL device chosen during ishmem initialization for the
calling PE.
This routine is useful for creating a matching sycl::queue for kernels and
on_queue APIs so that user-submitted work and ishmem operations
target the same device.
Calling ishmemx_my_device before initialization or after
ishmem_finalize is an error.
ISHMEM_PTR¶
-
void *ishmem_ptr(const void *dest, int pe)¶
- Parameters
dest – The symmetric address of the remotely accessible data object to be referenced
pe – An integer that indicates the PE number on which dest is to be accessed.
- Returns
A local pointer to the remotely accessible dest data object is returned when it can be accessed using memory loads and stores. Otherwise, a null pointer is returned.
Callable from the host and device.
Description:
ishmem_ptr returns a device address that may be used to directly
reference dest on the specified PE in the world team.
This address can be assigned to a pointer.
After that, ordinary loads and stores to dest may be performed from
within the device kernel.
The address returned by ishmem_ptr is a local address to a remotely
accessible data object.
Providing this address to an argument of a ishmem routine that requires
a symmetric address results in undefined behavior.
The ishmem_ptr routine can provide an efficient means to accomplish
communication, for example when a sequence of reads and writes to a data
object on a remote PE does not match the access pattern provided in a
ishmem data transfer routine like ishmem_put or
ishmem_iget.
ISHMEM_INFO_GET_VERSION¶
-
void ishmem_info_get_version(int *major, int *minor)¶
- Parameters
major – The major version of the
ishmemspecification in use.minor – The minor version of the
ishmemspecification in use.
- Returns
None.
Callable from the host and device.
Description:
Returns the major and minor version of the ishmem specification in use.
For a given library implementation, the major and minor version returned by
these calls are consistent with the library constants ISHMEM_MAJOR_VERSION
and ISHMEM_MINOR_VERSION.
ISHMEM_INFO_GET_NAME¶
-
void ishmem_info_get_name(char *name)¶
- Parameters
name – The vendor defined string.
- Returns
None.
Callable from the host and device.
Description: This routine returns the vendor defined name string of size defined by the library constant ISHMEM_MAX_NAME_LEN. The program calling this function provides the name memory buffer of at least size ISHMEM_MAX_NAME_LEN. The implementation copies the vendor defined string of size at most ISHMEM_MAX_NAME_LEN to name. The string is terminated by a null character. If the name memory buffer is provided with size less than ISHMEM_MAX_NAME_LEN, behavior is undefined. For a given library implementation, the vendor string returned is consistent with the library constant ISHMEM_VENDOR_STRING.