-
Notifications
You must be signed in to change notification settings - Fork 900
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature request] pytorch like operations with arrays #377
Comments
Cool, we follow the NumPy API so the Unfold is something we could discuss if it's especially useful. We aren't opposed to including more than pure |
well.. from my view unford is mode efficient than np.lib.stride_tricks.as_strided() in two reasons:
technically nothing prevent us to make general manipulations using numpy or torch.. while mls use for modeling and training. |
I am facing the same issue when I try to implement the Moe block for Mixtral model. My understanding is that mx.where doesn't support using only condition blocks for vectorized computation in the selected experts(so we have to explicit eval inds and using np.where instead). |
Nice to have some pytorch-like operations very useful with tensor (arrays) manipulations.
e.g. tensor.unique().. which returns an array of unique numbers with or without counting them.
or - tensor.unfold()... much more simple and straightforward way to apply rolling window to an array than np.lib.stride_tricks.as_strided()
Also mx.where() function is missing important functionality when it only receives the condition as an argument and returns list of array indices. Current implementation always requires three arguments.
Both numpy and pytorch has such capability.
I only mean that have more compatible functions may simplify existing torch code migration at least for testing and verifications.
The text was updated successfully, but these errors were encountered: