Skip to content

Commit

Permalink
[libc] Change the starting port index to use the SMID (#79200)
Browse files Browse the repository at this point in the history
Summary:
The RPC interface uses several ports to provide parallel access. Right
now we begin the search at the beginning, which heavily contests the
early ports. Using the SMID allows us to stagger the starting index
based off of the cluster identifier that is executing the current warp.
Multiple warps can share an SM, but it will guaruntee that the
contention for the low indices is lower.

This also increases the maximum port size to around 4096, this is
because 512 isn't enough to cover the full hardare parallelism needed to
guarantee this doesdn't deadlock.
  • Loading branch information
jhuber6 committed Jan 30, 2024
1 parent f2a78e6 commit 5470ea4
Show file tree
Hide file tree
Showing 5 changed files with 17 additions and 3 deletions.
8 changes: 8 additions & 0 deletions libc/src/__support/GPU/amdgpu/utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,14 @@ LIBC_INLINE uint64_t fixed_frequency_clock() {
/// Terminates execution of the associated wavefront.
[[noreturn]] LIBC_INLINE void end_program() { __builtin_amdgcn_endpgm(); }

/// Returns a unique identifier for the process cluster the current wavefront is
/// executing on. Here we use the identifier for the compute unit (CU) and
/// shader engine.
/// FIXME: Currently unimplemented on AMDGPU until we have a simpler interface
/// than the one at
/// https://github.com/ROCm/clr/blob/develop/hipamd/include/hip/amd_detail/amd_device_functions.h#L899
LIBC_INLINE uint32_t get_cluster_id() { return 0; }

} // namespace gpu
} // namespace LIBC_NAMESPACE

Expand Down
2 changes: 2 additions & 0 deletions libc/src/__support/GPU/generic/utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ LIBC_INLINE uint64_t fixed_frequency_clock() { return 0; }

[[noreturn]] LIBC_INLINE void end_program() { __builtin_unreachable(); }

LIBC_INLINE uint32_t get_cluster_id() { return 0; }

} // namespace gpu
} // namespace LIBC_NAMESPACE

Expand Down
4 changes: 4 additions & 0 deletions libc/src/__support/GPU/nvptx/utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,10 @@ LIBC_INLINE uint64_t fixed_frequency_clock() {
/// Terminates execution of the calling thread.
[[noreturn]] LIBC_INLINE void end_program() { __nvvm_exit(); }

/// Returns a unique identifier for the process cluster the current warp is
/// executing on. Here we use the identifier for the symmetric multiprocessor.
LIBC_INLINE uint32_t get_cluster_id() { return __nvvm_read_ptx_sreg_smid(); }

} // namespace gpu
} // namespace LIBC_NAMESPACE

Expand Down
4 changes: 2 additions & 2 deletions libc/src/__support/RPC/rpc.h
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ template <uint32_t lane_size = gpu::LANE_SIZE> struct alignas(64) Packet {
};

/// The maximum number of parallel ports that the RPC interface can support.
constexpr uint64_t MAX_PORT_COUNT = 512;
constexpr uint64_t MAX_PORT_COUNT = 4096;

/// A common process used to synchronize communication between a client and a
/// server. The process contains a read-only inbox and a write-only outbox used
Expand Down Expand Up @@ -519,7 +519,7 @@ LIBC_INLINE void Port<T, S>::recv_n(void **dst, uint64_t *size, A &&alloc) {
template <uint16_t opcode> LIBC_INLINE Client::Port Client::open() {
// Repeatedly perform a naive linear scan for a port that can be opened to
// send data.
for (uint32_t index = 0;; ++index) {
for (uint32_t index = gpu::get_cluster_id();; ++index) {
// Start from the beginning if we run out of ports to check.
if (index >= process.port_count)
index = 0;
Expand Down
2 changes: 1 addition & 1 deletion libc/utils/gpu/server/rpc_server.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ extern "C" {
#endif

/// The maximum number of ports that can be opened for any server.
const uint64_t RPC_MAXIMUM_PORT_COUNT = 512;
const uint64_t RPC_MAXIMUM_PORT_COUNT = 4096;

/// The symbol name associated with the client for use with the LLVM C library
/// implementation.
Expand Down

0 comments on commit 5470ea4

Please sign in to comment.