Skip to content

Commit ec8f4a3

Browse files
atmnpatelatmnpatel
committed
[OpenMP][Libomptarget] Introduce Remote Offloading Plugin
This introduces a remote offloading plugin for libomptarget. This implementation relies on gRPC and protobuf, so this library will only build if both libraries are available on the system. The corresponding server is compiled to `openmp-offloading-server`. This is a large change, but the only way to split this up is into RTL/server but I fear that could introduce an inconsistency amongst them. Ideally, tests for this should be added to the current ones that but that is problematic for at least one reason. Given that libomptarget registers plugin on a first-come-first-serve basis, if we wanted to offload onto a local x86 through a different process, then we'd have to either re-order the plugin list in `rtl.cpp` (which is what I did locally for testing) or find a better solution for runtime plugin registration in libomptarget. Differential Revision: https://reviews.llvm.org/D95314
1 parent 15313f6 commit ec8f4a3

File tree

17 files changed

+2506
-0
lines changed

17 files changed

+2506
-0
lines changed

‎openmp/docs/SupportAndFAQ.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,3 +109,12 @@ through a similar mechanism. It is worth noting that this support requires
109109
`extensions to the OpenMP begin/end declare variant context selector
110110
<https://clang.llvm.org/docs/AttributeReference.html#pragma-omp-declare-variant>`__
111111
that are exposed through LLVM/Clang to the user as well.
112+
113+
Q: What is a way to debug errors from mapping memory to a target device?
114+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
115+
116+
An experimental way to debug these errors is to use :ref:`remote process
117+
offloading <remote_offloading_plugin>`.
118+
By using ``libomptarget.rtl.rpc.so`` and ``openmp-offloading-server``, it is
119+
possible to explicitly perform memory transfers between processes on the host
120+
CPU and run sanitizers while doing so in order to catch these errors.

‎openmp/docs/design/Runtimes.rst‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,62 @@ LLVM/OpenMP Target Host Runtime Plugins (``libomptarget.rtl.XXXX``)
273273

274274
.. _device_runtime:
275275

276+
277+
.. _remote_offloading_plugin:
278+
279+
Remote Offloading Plugin:
280+
^^^^^^^^^^^^^^^^^^^^^^^^^
281+
282+
The remote offloading plugin permits the execution of OpenMP target regions
283+
on devices in remote hosts in addition to the devices connected to the local
284+
host. All target devices on the remote host will be exposed to the
285+
application as if they were local devices, that is, the remote host CPU or
286+
its GPUs can be offloaded to with the appropriate device number. If the
287+
server is running on the same host, each device may be identified twice:
288+
once through the device plugins and once through the device plugins that the
289+
server application has access to.
290+
291+
This plugin consists of ``libomptarget.rtl.rpc.so`` and
292+
``openmp-offloading-server`` which should be running on the (remote) host. The
293+
server application does not have to be running on a remote host, and can
294+
instead be used on the same host in order to debug memory mapping during offloading.
295+
These are implemented via gRPC/protobuf so these libraries are required to
296+
build and use this plugin. The server must also have access to the necessary
297+
target-specific plugins in order to perform the offloading.
298+
299+
Due to the experimental nature of this plugin, the CMake variable
300+
``LIBOMPTARGET_ENABLE_EXPERIMENTAL_REMOTE_PLUGIN`` must be set in order to
301+
build this plugin. For example, the rpc plugin is not designed to be
302+
thread-safe, the server cannot concurrently handle offloading from multiple
303+
applications at once (it is synchronous) and will terminate after a single
304+
execution. Note that ``openmp-offloading-server`` is unable to
305+
remote offload onto a remote host itself and will error out if this is attempted.
306+
307+
Remote offloading is configured via environment variables at runtime of the OpenMP application:
308+
* ``LIBOMPTARGET_RPC_ADDRESS=<Address>:<Port>``
309+
* ``LIBOMPTARGET_RPC_ALLOCATOR_MAX=<NumBytes>``
310+
* ``LIBOMPTARGET_BLOCK_SIZE=<NumBytes>``
311+
* ``LIBOMPTARGET_RPC_LATENCY=<Seconds>``
312+
313+
LIBOMPTARGET_RPC_ADDRESS
314+
""""""""""""""""""""""""
315+
The address and port at which the server is running. This needs to be set for
316+
the server and the application, the default is ``0.0.0.0:50051``. A single
317+
OpenMP executable can offload onto multiple remote hosts by setting this to
318+
comma-seperated values of the addresses.
319+
320+
LIBOMPTARGET_RPC_ALLOCATOR_MAX
321+
""""""""""""""""""""""""""""""
322+
After allocating this size, the protobuf allocator will clear. This can be set for both endpoints.
323+
324+
LIBOMPTARGET_BLOCK_SIZE
325+
"""""""""""""""""""""""
326+
This is the maximum size of a single message while streaming data transfers between the two endpoints and can be set for both endpoints.
327+
328+
LIBOMPTARGET_RPC_LATENCY
329+
""""""""""""""""""""""""
330+
This is the maximum amount of time the client will wait for a response from the server.
331+
276332
LLVM/OpenMP Target Device Runtime (``libomptarget-ARCH-SUBARCH.bc``)
277333
--------------------------------------------------------------------
278334

‎openmp/libomptarget/plugins/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ add_subdirectory(ppc64)
7474
add_subdirectory(ppc64le)
7575
add_subdirectory(ve)
7676
add_subdirectory(x86_64)
77+
add_subdirectory(remote)
7778

7879
# Make sure the parent scope can see the plugins that will be created.
7980
set(LIBOMPTARGET_SYSTEM_TARGETS "${LIBOMPTARGET_SYSTEM_TARGETS}" PARENT_SCOPE)

‎openmp/libomptarget/plugins/exports‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@ VERS1.0 {
1919
__tgt_rtl_run_target_region;
2020
__tgt_rtl_run_target_region_async;
2121
__tgt_rtl_synchronize;
22+
__tgt_rtl_register_lib;
23+
__tgt_rtl_unregister_lib;
2224
local:
2325
*;
2426
};

‎openmp/libomptarget/plugins/remote/CMakeLists.txt‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
##===----------------------------------------------------------------------===##
2+
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
3+
# See https://llvm.org/LICENSE.txt for license information.
4+
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
5+
#
6+
##===----------------------------------------------------------------------===##
7+
#
8+
# Build a plugin (client) and server for remote offloading.
9+
#
10+
##===----------------------------------------------------------------------===#
11+
12+
cmake_minimum_required(VERSION 3.15)
13+
14+
if (NOT(CMAKE_SYSTEM_NAME MATCHES "Linux"))
15+
libomptarget_say("Not building remote offloading plugin: only support Linux hosts.")
16+
return()
17+
endif()
18+
19+
if (NOT(LIBOMPTARGET_ENABLE_EXPERIMENTAL_REMOTE_PLUGIN))
20+
return()
21+
endif()
22+
23+
find_package(Protobuf)
24+
find_package(gRPC CONFIG)
25+
26+
find_program(PROTOC protoc)
27+
find_program(GRPC_CPP_PLUGIN grpc_cpp_plugin)
28+
29+
if (Protobuf_FOUND AND gRPC_FOUND AND PROTOC AND GRPC_CPP_PLUGIN)
30+
libomptarget_say("Building remote offloading plugin.")
31+
set(directory "${CMAKE_BINARY_DIR}/include/openmp/libomptarget/plugins/remote/")
32+
file(MAKE_DIRECTORY ${directory})
33+
execute_process(COMMAND ${CMAKE_COMMAND} -E make_directory ${directory})
34+
execute_process(
35+
COMMAND protoc --cpp_out=${directory} -I ${CMAKE_CURRENT_SOURCE_DIR}/include ${CMAKE_CURRENT_SOURCE_DIR}/include/openmp.proto
36+
COMMAND protoc --grpc_out=${directory} -I ${CMAKE_CURRENT_SOURCE_DIR}/include ${CMAKE_CURRENT_SOURCE_DIR}/include/openmp.proto --plugin=protoc-gen-grpc=${GRPC_CPP_PLUGIN}
37+
)
38+
39+
set(GRPC_SRC_FILES
40+
${directory}/openmp.grpc.pb.cc
41+
${directory}/openmp.pb.cc
42+
)
43+
44+
set(GRPC_INCLUDE_DIR
45+
${directory}
46+
)
47+
else()
48+
libomptarget_say("Not building remote offloading plugin: required libraries were not found.")
49+
endif()
50+
51+
set(RPC_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/include/)
52+
set(RPC_SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/lib/)
53+
54+
add_subdirectory(src)
55+
add_subdirectory(server)

‎openmp/libomptarget/plugins/remote/include/Utils.h‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
//===----------------- Utils.h - Utilities for Remote RTL -----------------===//
2+
//
3+
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4+
// See https://llvm.org/LICENSE.txt for license information.
5+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6+
//
7+
//===----------------------------------------------------------------------===//
8+
//
9+
// Utilities for data transfer through protobuf and debugging.
10+
//
11+
//===----------------------------------------------------------------------===//
12+
13+
#ifndef UTILS_H
14+
#define UTILS_H
15+
16+
#include "Debug.h"
17+
#include "omptarget.h"
18+
#include "openmp.grpc.pb.h"
19+
#include "openmp.pb.h"
20+
#include "rtl.h"
21+
#include <string>
22+
23+
#define CLIENT_DBG(...) \
24+
{ \
25+
if (DebugLevel > 0) { \
26+
fprintf(stderr, "[[Client]] --> "); \
27+
fprintf(stderr, __VA_ARGS__); \
28+
fprintf(stderr, "\n"); \
29+
} \
30+
}
31+
32+
#define SERVER_DBG(...) \
33+
{ \
34+
if (DebugLevel > 0) { \
35+
fprintf(stderr, "[[Server]] --> "); \
36+
fprintf(stderr, __VA_ARGS__); \
37+
fprintf(stderr, "\n"); \
38+
} \
39+
}
40+
41+
namespace RemoteOffloading {
42+
43+
using namespace openmp::libomptarget::remote;
44+
45+
using openmp::libomptarget::remote::DeviceOffloadEntry;
46+
using openmp::libomptarget::remote::TargetBinaryDescription;
47+
using openmp::libomptarget::remote::TargetOffloadEntry;
48+
using openmp::libomptarget::remote::TargetTable;
49+
50+
struct RPCConfig {
51+
std::vector<std::string> ServerAddresses;
52+
uint64_t MaxSize;
53+
uint64_t BlockSize;
54+
RPCConfig() {
55+
ServerAddresses = {"0.0.0.0:50051"};
56+
MaxSize = 1 << 30;
57+
BlockSize = 1 << 20;
58+
}
59+
};
60+
61+
/// Helper function to parse common environment variables between client/server
62+
void parseEnvironment(RPCConfig &Config);
63+
64+
/// Loads a target binary description into protobuf.
65+
void loadTargetBinaryDescription(const __tgt_bin_desc *Desc,
66+
TargetBinaryDescription &Request);
67+
68+
/// Unload a target binary description from protobuf. The map is used to keep
69+
/// track of already copied device images.
70+
void unloadTargetBinaryDescription(
71+
const TargetBinaryDescription *Request, __tgt_bin_desc *Desc,
72+
std::unordered_map<const void *, __tgt_device_image *>
73+
&HostToRemoteDeviceImage);
74+
75+
/// Frees argument as constructed by loadTargetBinaryDescription
76+
void freeTargetBinaryDescription(__tgt_bin_desc *Desc);
77+
78+
/// Copies from TargetOffloadEntry protobuf to a tgt_bin_desc during unloading.
79+
void copyOffloadEntry(const TargetOffloadEntry &EntryResponse,
80+
__tgt_offload_entry *Entry);
81+
82+
/// Copies from tgt_bin_desc into TargetOffloadEntry protobuf during loading.
83+
void copyOffloadEntry(const __tgt_offload_entry *Entry,
84+
TargetOffloadEntry *EntryResponse);
85+
86+
/// Shallow copy of offload entry from tgt_bin_desc to TargetOffloadEntry
87+
/// during loading.
88+
void shallowCopyOffloadEntry(const __tgt_offload_entry *Entry,
89+
TargetOffloadEntry *EntryResponse);
90+
91+
/// Copies DeviceOffloadEntries into table during unloading.
92+
void copyOffloadEntry(const DeviceOffloadEntry &EntryResponse,
93+
__tgt_offload_entry *Entry);
94+
95+
/// Loads tgt_target_table into a TargetTable protobuf message.
96+
void loadTargetTable(__tgt_target_table *Table, TargetTable &TableResponse,
97+
__tgt_device_image *Image);
98+
99+
/// Unloads from a target_table from protobuf.
100+
void unloadTargetTable(
101+
TargetTable &TableResponse, __tgt_target_table *Table,
102+
std::unordered_map<void *, void *> &HostToRemoteTargetTableMap);
103+
104+
/// Frees argument as constructed by unloadTargetTable
105+
void freeTargetTable(__tgt_target_table *Table);
106+
107+
void dump(const void *Start, const void *End);
108+
void dump(__tgt_offload_entry *Entry);
109+
void dump(TargetOffloadEntry Entry);
110+
void dump(__tgt_target_table *Table);
111+
void dump(__tgt_device_image *Image);
112+
} // namespace RemoteOffloading
113+
114+
#endif

0 commit comments

Comments
 (0)