You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem: As highlighted in a discussion in MR #375, a lot of the tensors we create for influence calculation end up being very large. For example, the influence factors are stored in a matrix of size (N_test_points, N_model_parameters). Often, these cannot fit into RAM (be it ordinary or on GPU).
A simple fix would use generators. Instead of creating all the influence factors, we could calculate batch-by-batch within the compute_influences_up and compute_influences_pert methods. Same can be said of grads within these methods. However, this would imply re-calculating influence factors or grads several times, with massive computational overhead.
A better fix would cache the tensors to disk. We could create a CachedTensor class that saves the tensors into files and loads them whenever needed, thus freeing the VRAM. Even the output of the compute_influences method could be a CachedTensor: one would only need to remember to flush it when it is no longer needed.
The text was updated successfully, but these errors were encountered:
Problem: As highlighted in a discussion in MR #375, a lot of the tensors we create for influence calculation end up being very large. For example, the influence factors are stored in a matrix of size (N_test_points, N_model_parameters). Often, these cannot fit into RAM (be it ordinary or on GPU).
A simple fix would use generators. Instead of creating all the influence factors, we could calculate batch-by-batch within the compute_influences_up and compute_influences_pert methods. Same can be said of grads within these methods. However, this would imply re-calculating influence factors or grads several times, with massive computational overhead.
A better fix would cache the tensors to disk. We could create a CachedTensor class that saves the tensors into files and loads them whenever needed, thus freeing the VRAM. Even the output of the compute_influences method could be a CachedTensor: one would only need to remember to flush it when it is no longer needed.
The text was updated successfully, but these errors were encountered: