power9: bus error with 4GB copy

[bureddy]

root@ibm-p9-012 gdrcopy]# ./copybw -s 4294967296 -c 4294967296 -d 0
GPU id:0 name:Tesla V100-SXM2-16GB PCI domain: 4 bus: 4 device: 0
GPU id:1 name:Tesla V100-SXM2-16GB PCI domain: 4 bus: 5 device: 0
GPU id:2 name:Tesla V100-SXM2-16GB PCI domain: 53 bus: 3 device: 0
GPU id:3 name:Tesla V100-SXM2-16GB PCI domain: 53 bus: 4 device: 0
selecting device 0
testing size: 4294967296
rounded size: 4294967296
device ptr: 7ffe40000000
bar_ptr: 0x7ffc3fff0000
info.va: 7ffe40000000
info.mapped_size: 4294967296
info.page_size: 65536
page offset: 0
user-space pointer:0x7ffc3fff0000
BAR writing test, size=4294967296 offset=0 num_iters=10000
Bus error (core dumped)
[root@ibm-p9-012 gdrcopy]#

[bureddy]

from @drossetti

1. a 4GB GPU mem allocation is not physically contiguous on my P9
2. that triggers a ugly work-around

[drossetti]

@bureddy please have a look at PR #40