Resource Exhausted Error

[ZahraAnam]

I tried to use your code and my data is huge, but not that huge to exhaust the resource. I am getting the following error, can you help??
2020-03-26 20:08:52.756851: I tensorflow/core/common_runtime/bfc_allocator.cc:964] total_region_allocated_bytes_: 7977644800 memory_limit_: 7977644852 available bytes: 52 curr_region_allocation_bytes_: 15955290112
2020-03-26 20:08:52.756880: I tensorflow/core/common_runtime/bfc_allocator.cc:970] Stats:
Limit: 7977644852
InUse: 33248000
MaxInUse: 49112832
NumAllocs: 23
MaxAllocSize: 23797504

2020-03-26 20:08:52.756913: W tensorflow/core/common_runtime/bfc_allocator.cc:429] *___________________________________________________________________________________________________
2020-03-26 20:08:52.756986: W tensorflow/core/framework/op_kernel.cc:1655] OP_REQUIRES failed at matmul_op.cc:480 : Resource exhausted: OOM when allocating tensor with shape[47350,991553] and type double on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
Traceback (most recent call last):
File "Registration_ICP.py", line 104, in
icp_scale, icp_offset = icp.icp(dgt_points.T, sgt_points.T, dim_offset='all', dim_scale='all', num_iter=50, use_gpu = True)
File "/home/anam/self_localization/self_localization/self_localization/Codes/Registration_Codes/icp/icp/core.py", line 46, in icp
dist = src_squ - 2 * tf.matmul(src, dst, transpose_b=True) + tf.transpose(dst_squ)
File "/home/anam/anaconda3/envs/my_tf_env/lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/home/anam/anaconda3/envs/my_tf_env/lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py", line 2798, in matmul
a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
File "/home/anam/anaconda3/envs/my_tf_env/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py", line 5616, in mat_mul
_ops.raise_from_not_ok_status(e, name)
File "/home/anam/anaconda3/envs/my_tf_env/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 6606, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "", line 3, in raise_from
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[47350,991553] and type double on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:MatMul] name: MatMul/

[VincentStimper]

Well, you need to store more than the data itself on your resource as the distances between every source and target point have to be computed. Just subsample your data if it is to large.