Skip to content

Commit

Permalink
[Libomptarget] Fix block and thread limit environment variables not b…
Browse files Browse the repository at this point in the history
…eing respected

The next-gen plugins did not properly set the values from
`OMP_NUM_TEAMS` and `OMP_TEAMS_THREAD_LIMIT`. This is because these
maximum values are set by each plugin to its hardware maximum. This
happens *after* the previous initialization. Move it to the correct
place and then add a test.

Fixes #61082

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D145105
  • Loading branch information
jhuber6 committed Mar 1, 2023
1 parent c9843af commit 6563780
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 10 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -352,15 +352,7 @@ GenericDeviceTy::GenericDeviceTy(int32_t DeviceId, int32_t NumDevices,
OMPX_InitialNumEvents("LIBOMPTARGET_NUM_INITIAL_EVENTS", 32),
DeviceId(DeviceId), GridValues(OMPGridValues),
PeerAccesses(NumDevices, PeerAccessState::PENDING), PeerAccessesLock(),
PinnedAllocs(*this) {
if (OMP_NumTeams > 0)
GridValues.GV_Max_Teams =
std::min(GridValues.GV_Max_Teams, uint32_t(OMP_NumTeams));

if (OMP_TeamsThreadLimit > 0)
GridValues.GV_Max_WG_Size =
std::min(GridValues.GV_Max_WG_Size, uint32_t(OMP_TeamsThreadLimit));
}
PinnedAllocs(*this) {}

Error GenericDeviceTy::init(GenericPluginTy &Plugin) {
if (auto Err = initImpl(Plugin))
Expand All @@ -385,6 +377,16 @@ Error GenericDeviceTy::init(GenericPluginTy &Plugin) {
return HeapSizeEnvarOrErr.takeError();
OMPX_TargetHeapSize = std::move(*HeapSizeEnvarOrErr);

// Update the maximum number of teams and threads after the device
// initialization sets the corresponding hardware limit.
if (OMP_NumTeams > 0)
GridValues.GV_Max_Teams =
std::min(GridValues.GV_Max_Teams, uint32_t(OMP_NumTeams));

if (OMP_TeamsThreadLimit > 0)
GridValues.GV_Max_WG_Size =
std::min(GridValues.GV_Max_WG_Size, uint32_t(OMP_TeamsThreadLimit));

// Enable the memory manager if required.
auto [ThresholdMM, EnableMM] = MemoryManagerTy::getSizeThresholdFromEnv();
if (EnableMM)
Expand Down Expand Up @@ -1191,7 +1193,6 @@ __tgt_target_table *__tgt_rtl_load_binary(int32_t DeviceId,
GenericPluginTy &Plugin = Plugin::get();
GenericDeviceTy &Device = Plugin.getDevice(DeviceId);


auto TableOrErr = Device.loadBinary(Plugin, TgtImage);
if (!TableOrErr) {
auto Err = TableOrErr.takeError();
Expand Down
12 changes: 12 additions & 0 deletions openmp/libomptarget/test/api/omp_env_vars.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// RUN: %libomptarget-compile-generic
// RUN: env OMP_NUM_TEAMS=1 OMP_TEAMS_THREAD_LIMIT=1 LIBOMPTARGET_INFO=16 \
// RUN: %libomptarget-run-generic 2>&1 | %fcheck-generic

#define N 256

int main() {
// CHECK: Launching kernel [[KERNEL:.+_main_.+]] with 1 blocks and 1 threads
#pragma omp target teams
#pragma omp parallel
{}
}

0 comments on commit 6563780

Please sign in to comment.