Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Call for Contributions #426

Closed
houseroad opened this Issue Jan 17, 2018 · 19 comments

Comments

Projects
None yet
9 participants
@houseroad
Copy link
Member

houseroad commented Jan 17, 2018

Call for Contributions

ONNX is open source, and contributions from the community is welcome. Many tasks, (some are low-hanging fruits), are listed here. If you would like to contribute to the project, but don't know where to start, this list is worth to check.

Adding ONNX Backend Test Case

Ideally, we should have sample cases for every operator, which can generate backend test data and also serve as examples. However, many operators have no sample cases, and even not covered by ONNX backend test cases. So please contribute your useful cases to ONNX repository.

How to contribute ONNX backend test?

The operators which don't have examples:

  • Abs
  • Add
  • And
  • ArgMax
  • ArgMin
  • AveragePool
  • BatchNormalization
  • Cast
  • Ceil
  • Clip
  • Concat
  • Conv
  • ConvTranspose
  • DepthToSpace
  • Div
  • Dropout
  • Elu
  • Equal
  • Exp
  • Flatten
  • Floor
  • GRU
  • Gather
  • Gemm
  • GlobalAveragePool
  • GlobalLpPool
  • GlobalMaxPool
  • Greater
  • HardSigmoid
  • Hardmax
  • InstanceNormalization
  • LRN
  • LSTM
  • LeakyRelu
  • Less
  • Log
  • LogSoftmax
  • LpNormalization
  • LpPool
  • Max
  • MaxPool
  • MaxRoiPool
  • Mean
  • Min
  • Mul
  • Neg
  • Not
  • Or
  • PRelu
  • Pow
  • RNN
  • RandomNormal
  • RandomNormalLike
  • RandomUniform
  • RandomUniformLike
  • Reciprocal
  • ReduceL1
  • ReduceL2
  • ReduceLogSum
  • ReduceLogSumExp
  • ReduceMax
  • ReduceMean
  • ReduceMin
  • ReduceProd
  • ReduceSum
  • ReduceSumSquare
  • Reshape
  • Selu
  • Sigmoid
  • Softmax
  • Softplus
  • Softsign
  • SpaceToDepth
  • Split
  • Sqrt
  • Squeeze
  • Sub
  • Sum
  • Tanh
  • Tile
  • TopK
  • Transpose
  • Unsqueeze
  • Xor
  • experimental ATen
  • experimental Affine
  • experimental ConstantFill
  • experimental Crop
  • experimental FC
  • experimental GRUUnit
  • experimental GivenTensorFill
  • experimental Identity
  • experimental If
  • experimental ImageScaler
  • experimental Loop
  • experimental LoopIndexTensor
  • experimental MeanVarianceNormalization
  • experimental ParametricSoftplus
  • experimental Scale
  • experimental ScaledTanh
  • experimental ThresholdedRelu
  • experimental Upsample

Adding Export Support in PyTorch

Some simple operators are not supported by PyTorch exporter yet, call for contributions.

How to add ONNX support in PyTorch

How to generate ONNX backend test data in PyTorch operator export test

Here is a TO-DO list:

  • Tile (Repeat in PyTorch)

@houseroad houseroad changed the title Low-Hanging Fruits, Call for Contributions Call for Contributions Jan 17, 2018

@houseroad

This comment has been minimized.

Copy link
Member Author

houseroad commented Jan 19, 2018

@guoyuhong, thanks for contributing the node test of and, or, xor, and not!

@linkerzhang

This comment has been minimized.

Copy link
Member

linkerzhang commented Jan 20, 2018

To avoid duplicate work, ReduceMax, ReduceMin, ReduceSum, Conv, Flatten, and generator OPs are being taken care of by some folks now.

@tianleiwu

This comment has been minimized.

Copy link
Contributor

tianleiwu commented Jan 23, 2018

I am adding tests for some logical and math operators:

  • Equal, Greater, Less
  • Ceil, Div, Floor, LeakyRelu, Mul, Neg, Reciprocal, Selu, Sqrt, Sub
@kevin645

This comment has been minimized.

Copy link

kevin645 commented Jan 25, 2018

Working on ArgMax, and ArgMin. Will try to complete the list as I understand the source.

I was reading through the source of the backend stub that is test_backend_test.py, It's only checking the proto format. How can I setup the full ONNX reference implementation for the Ops? Do I need to set it up with a supported backend? (caffe2/pytorch)

@houseroad

This comment has been minimized.

Copy link
Member Author

houseroad commented Jan 25, 2018

@kevin645 Thanks a lot! I marked ArgMax and ArgMin as you picked them up. After you finish the case test, you can run cmd_tools.py to generate the data for backend test. The test data will generated in onnx/backend/test/data folder. test_backend_test.py only runs the checker on the test data. To have a full reference execution on the generated data, you need to install a backend, such as caffe2 with onnx-caffe2, or cntk. (Right now, PyTorch can only export the model, there is no importer yet.)

@linkerzhang

This comment has been minimized.

Copy link
Member

linkerzhang commented Jan 26, 2018

Cast Reshape Concat Split Transpose will be covered, @houseroad please mark them to avoid duplication.

#472 and #464 are covering math ops btw.

@bddppq

This comment has been minimized.

Copy link
Member

bddppq commented Feb 19, 2018

Gather added in #537 thanks to @huitseeker

@terrytangyuan

This comment has been minimized.

Copy link

terrytangyuan commented Mar 4, 2018

@houseroad I am putting together an R package for ONNX here. Would you like the ownership get transferred to ONNX Github organization? It would help with the exposure of the package later on.

Edit: let's discuss this in #581.

@anirudhacharya

This comment has been minimized.

Copy link
Collaborator

anirudhacharya commented Mar 6, 2018

Has the tests for the following operators - ReduceMax, ReduceMin, ReduceSum, Conv, Flatten, been merged? If no one is doing ReduceMean and ReduceProd, I can take it up. Please confirm.

@houseroad

This comment has been minimized.

Copy link
Member Author

houseroad commented Mar 6, 2018

As far as I know, no one is working on ReduceMax, ReduceMin, ReduceSum, ReduceMean, ReduceProd yet. Feel free to take them. @anirudhacharya :-)

@anirudhacharya

This comment has been minimized.

Copy link
Collaborator

anirudhacharya commented Mar 6, 2018

@houseroad This comment - #426 (comment) says that someone is working on ReduceMax, ReduceMin and ReduceSum. And the checklist in the description section of this Issue has these three operators marked as done.

I will work on the ReduceProd and ReduceMean operator for now.

@houseroad

This comment has been minimized.

Copy link
Member Author

houseroad commented Mar 7, 2018

@anirudhacharya sounds good to me. Thanks!

ruimashita added a commit to ruimashita/onnx that referenced this issue Mar 15, 2018

bddppq added a commit that referenced this issue Mar 23, 2018

Add DepthToSpace test cases (#619)
* Add DepthToSpace test cases #426

* Split the two DepthToSpace test cases into two functions

* Remove multiple spaces after comma in depthtospace test case
@anirudhacharya

This comment has been minimized.

Copy link
Collaborator

anirudhacharya commented May 3, 2018

@onnx/facebook-onnx-contributors Can we revisit this list and verify if it is up to date? I have made a few updates but it might still be a bit out of sync.
Also when do we tick off an operator in the list -

  • When someone starts working on it or when the PR gets merged?
  • If a particular operator has tests in pytorch-converted/pytorch-operator but does not have node tests do we consider it as full test coverage?
@houseroad

This comment has been minimized.

Copy link
Member Author

houseroad commented May 3, 2018

A1: When someone starts working on it, we will mark the op

A2: No, we still need node test, which serves as doc purpose.

@anirudhacharya

This comment has been minimized.

Copy link
Collaborator

anirudhacharya commented May 3, 2018

Then what is the purpose of pytorch-converted/pytorch-operator tests?

@houseroad

This comment has been minimized.

Copy link
Member Author

houseroad commented May 3, 2018

Providing more cases to check the backend's correctness. :-)

@anirudhacharya

This comment has been minimized.

Copy link
Collaborator

anirudhacharya commented May 4, 2018

I am working on BatchNormalization, GEMM, and LRN operator tests.

@houseroad houseroad referenced this issue Jun 13, 2018

Merged

Add stats on ONNX node tests #1110

3 of 3 tasks complete

Ac2zoom pushed a commit to Ac2zoom/onnx that referenced this issue Jun 21, 2018

Add DepthToSpace test cases (onnx#619)
* Add DepthToSpace test cases onnx#426

* Split the two DepthToSpace test cases into two functions

* Remove multiple spaces after comma in depthtospace test case
@prasanthpul

This comment has been minimized.

Copy link
Member

prasanthpul commented Mar 21, 2019

@houseroad Suggest we close this old issue and open a new one if needed with what remains

@houseroad

This comment has been minimized.

Copy link
Member Author

houseroad commented Mar 21, 2019

@prasanthpul okay, close it

@houseroad houseroad closed this Mar 21, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.