Fix large concatenations and stacks. #51
Merged
Conversation
hameerabbasi
force-pushed the
hameerabbasi:fix-large-concat-stack
branch
from
cf0c89f
to
1f04bd6
hameerabbasi
force-pushed the
hameerabbasi:fix-large-concat-stack
branch
from
1f04bd6
to
51a1360
|
This looks good to me. I had a small question, but please feel free to merge at will. |
| data = np.concatenate([x.data for x in arrays]) | ||
| coords = np.concatenate([x.coords for x in arrays], axis=1) | ||
| coords = np.concatenate([x.coords for x in arrays], axis=1).astype(coords_dtype) |
mrocklin
Dec 28, 2017
Collaborator
Is it better to call .astype(...) before concatenation on all of arrays or after as is done here?
Is it better to call .astype(...) before concatenation on all of arrays or after as is done here?
hameerabbasi
Dec 28, 2017
Author
Collaborator
I would think after. Because it would be applied to a bigger array and thus the overhead would be small. Memory would be better, too, as we are just passing already-allocated arrays to concatenate, otherwise we'd have to allocate for each temp array.
Incidentally, astype just returns the original array if it is already the correct dtype.
I would think after. Because it would be applied to a bigger array and thus the overhead would be small. Memory would be better, too, as we are just passing already-allocated arrays to concatenate, otherwise we'd have to allocate for each temp array.
Incidentally, astype just returns the original array if it is already the correct dtype.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Addresses #32.