-
Notifications
You must be signed in to change notification settings - Fork 6.8k
Conversation
@mxnet-label-bot add [Numpy] |
python/mxnet/numpy/multiarray.py
Outdated
Chained array operations. For more complicated contractions, speed ups | ||
might be achieved by repeatedly computing a 'greedy' path or pre-computing the | ||
'optimal' path and repeatedly applying it, using an | ||
`einsum_path` insertion (since version 1.12.0). Performance improvements can be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1.12.0 is the official numpy version. We'd better not include that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
`einsum_path` insertion (since version 1.12.0). Performance improvements can be | ||
particularly significant with larger arrays: | ||
|
||
>>> a = np.ones(64).reshape(2,4,8) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the benchmark from the mxnet.numpy.einsum or the official np.einsum?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't benchmarked yet. The benchmark here is from official np.einsum. I will remove them.
int newdim; | ||
|
||
const TShape& shape = op.shape_; | ||
TShape stride; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Combine this line and the next line into one line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
TShape* newshape, | ||
TShape* newstride) { | ||
using namespace mxnet_op; | ||
int idim, ndim, icombine, combineoffset; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define the variables where they are first used, i.e. narrow down the scope of variables as much as possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
src/operator/numpy/np_einsum_op.cc
Outdated
namespace mxnet { | ||
namespace op { | ||
|
||
inline bool NumpyEinsumShape(const nnvm::NodeAttrs& attrs, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: no need to inline
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
#define NPY_MAXDIMS 32 | ||
#define NPY_MAXARGS 32 | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one less blank line here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
return ret; | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one less blank line here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
src/operator/numpy/np_einsum_op.cc
Outdated
// std::cout << output_labels[i] << " "; | ||
// } | ||
// std::cout << std::endl; | ||
// std::cout << "max_broadcast = " << max_broadcast << std::endl; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove dead code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
4d2114e
to
cdfaf1c
Compare
924d327
to
b983ab4
Compare
cherry-picked from #15565. Tensordot optimization has been enabled. @reminisce |
10a03fa
to
a23120e
Compare
7bcdcb3
to
be06bd6
Compare
c301605
to
0114f6f
Compare
* Fix backward_clip num inputs and type of clip params * Clip test * Trigger CI * Changes to clip docs * Fix docstring * Trigger CI
* Modify ndarray slice to have numpy compatbile behaviou * Minor syntax fix * Fix slice inconsistency * Allow empty outputs after slicing ndarrays * Fix
* remove test images * add script and .gitignore * add test helper to download images * remove unlicensed pic * add license header
* Implements tensordot and dot. * Change tests. * Add spaces. * Reorganize codes. * Remove np_matrix_op.h
* Adding Large Index Support for slice operator * adding changes to fix py2 related error in CI/CD * fixing base.py * rearrange system call and slower Feature() call * refactoring c_api, c_symbolic_api, c_api_common * templatizing code * caching results of runtime features and minor refactoring * fixing local caching in ndarray shape
* Remove old cudnn support (<v7). * Simplify conv impl selection. * Remove comments justifying STATIC_ASSERT_CUDNN_VERSION_GE.
* Add squeeze/flatten/transpose/reshape Add more ops back Add new files Add unit tests Add unit test for broadcast_arrays Add more ops and tests Add arange * clean up
* test rdiv * floating_point exception handle * add 10 other ops * added rpow and made numpy consistent * attempt to solve memory issue * linting fix * Trigger notification * lint
* random ops * replace array with uniform * remove dtype * randn add * add multinomial * multi,randn small fix * add negative bino * fix memory issue - Failed to allocate CPU Memory * Trigger notification * linting fix * Trigger notification
* sequence_last, sequence_reverse, sequence_mask * working softmax_cross_entropy * fix linting, add index_copy * add softmax output * add leaky relu * add pooling * add layernorm * add dropout, activation, batchnorm and update layernorm * address comments to remove some comments * handling imports
* numpy-compatible concatenate upstream * extend ci deadline
* numpy-compatible concatenate upstream * extend ci deadline
* numpy-compatible concatenate upstream * extend ci deadline
* numpy-compatible concatenate upstream * extend ci deadline
* Fix ConcatType and add test * Remove return false * Change error message * Run RNN test only when CUDNN enabled * set default context for test_contrib_amp
PRed to the master branch. Closing this one. |
Description
numpy compatible einsum
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.
Changes
Comments
TODO
Thank @reminisce and @haojin2 for guidance and review.