Scala Package for v1.0 TODO List

[yzhliu]

MXNet is going to release a stable version #2944. So it's time to make Scala package a stable and complete release. Here I make a TODO list, please point out if I miss anything @Ldpe2G.

1. Support NNVM refactor.
2. Complete operators for Symbol and NDArray. So far we add operators by hand in Scala, while the ops have changed a lot since the last version the scala pkg picks.
   1. Introduce code generation to make it up-to-date.
   2. Test major operators on GPU.
3. Add more optimizers (support lr_mult and wd_mult) and metrics. And make sure Executor acts the same as the one in current python version.
4. Fully support DType for NDArray and type inferring in Symbol.
5. Add module api.
6. More examples, e.g., RNN with mask (and seq2seq), E2E Neural Art, RCNN, Multi-task Learning, etc.
7. Discuss the production Spark APIs. If we cannot make it stable before v1.0 releases, mark them as @Experimental. @liyuance
8. More documents.
9. Fix compiling and runtime issues, e.g., fail to compile scala package test  #2184

Here I also refer some issues that listed before but haven't solved yet, with lower priority: #1591 #2268

[Ldpe2G]

@Javelinjs , I am wondering how much of the Scala Api will be changed, for example, if using code generation to get the operator, will it break the compability with the old version ?

[yzhliu]

@Ldpe2G Sure, we need to keep it compatible with the old.

[wellner]

@Javelinjs Would the plans here include support for custom operators (that could be implemented in Scala)?

[yzhliu]

@wellner Mind giving an example?

[Ldpe2G]

@Javelinjs I think what wellner means is to implement the CustomOp in scala like python.

[yzhliu]

@wellner @Ldpe2G I see. Not in the roadmap for this coming version.
@Ldpe2G I sent you an email, could you check your mailbox once you have time?

[wellner]

Yes, CustomOp was what I meant. I couldn't argue to prioritize it higher than what you've outlined, but wanted to check!

[Ldpe2G]

@Javelinjs ok

[yzhliu]

@wellner We'll support it once the TODOs here are finished. Do you heavily rely on this feature?

[wellner]

@Javelinjs Thanks for asking. No, definitely not heavily but some future/potential plans might rely on it, see #3118

[yzhliu]

@wellner We'll look into and try to support it once we finish the issues above.