Skip to content

Commit

Permalink
openib: Disable CUDA async by default
Browse files Browse the repository at this point in the history
Disable async receive for CUDA under OpenIB.  While a performance
optimization, it also causes incorrect results for transfers
larger than the GPUDirect RDMA limit.  This change has been validated
and approved by Akshay.

References #3972

Signed-off-by: Brian Barrett <bbarrett@amazon.com>
  • Loading branch information
bwbarrett committed Sep 20, 2018
1 parent dccf780 commit 9344afd
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion opal/mca/btl/openib/btl_openib_mca.c
Expand Up @@ -573,7 +573,7 @@ int btl_openib_register_mca_params(void)
CHECK(reg_bool("cuda_async_recv", NULL,
"Enable or disable CUDA async recv copies "
"(true = async; false = sync)",
true, &mca_btl_openib_component.cuda_async_recv));
false, &mca_btl_openib_component.cuda_async_recv));
/* Also make the max send size larger for better GPU buffer performance */
mca_btl_openib_module.super.btl_max_send_size = 128 * 1024;
/* Turn of message coalescing - not sure if it works with GPU buffers */
Expand Down

0 comments on commit 9344afd

Please sign in to comment.