Additional backends

[justusschock]

This PR will add additional backends (TorchScript in #98 and TF Eager Execution in #101 ) as well as the move to the new apex.amp API (#97 ).

Currently this PR is just open to track conversations and the Progress of the different Parts of it. It will be merged, after #66 , #97 , #98 and #101 are merged.

Progress so far:

• Trainer Refactoring (Trainer refactoring #66) merged into master
• new apex.amp API enabled (Move to new APEX.amp API #97 )
• TorchScript Backend (including new dependency resolution) (TorchScript backend #98 ) merged
• TF Eager Execution Backend (Add TF Eager Execution Backend #101 ) merged
• SkLearn Backend (Scikit-Learn Compatibility #109 ) merged
• chainer Backend (Chainer backend #110 ) merged
• Move backends to separate files -> 1 File / backend in delira.training including train-utils, trainer and experiment; 1 File / backend in delira.models including base network and other necessary stuff (like data parallel for chainer)
• Refactor tests
• Add defaults for closure and prepare_batch to each abstract network
• Register all backends in init
• Register all backends in setup

The following changes will be done in subsequent PRs.

• MXNet Backend (WIP: Mxnet backend #119 ) merged
• Provide examples for all backends
• Add All backend-specific files to docs
• Add all backends to README
• Add all backends to Docs/Getting Started

Other possible backends include

• Theano: Could be easy - quite similar to TF
• Caffe2: Probably won't integrate due to incorporation into PyTorch
• CNTK: Probably won't integrate because no longer actively developed and not that commonly used
• PaddlePaddle (by Baidu)

If single backends of this list will be included, this should be done in separate issues/prs afterwards, since it is not yet clear which backends we want to integrate from here

[justusschock]

@ORippler Tests are passing now. It would be great if you could make some time for a review

[ORippler]

• change

  delira/delira/training/base_trainer.py

  Line 207 in cdee72f

  self.save_state(os.path.join(self.save_path, "checkpoint_epoch_0"))

  to make use of self.start_epoch.
• Look at the docstrings. Oftentimes, e.g. for basetrainer, they are wrong (still wrongly states to raise NotImplementedError. I think this still comes from before the Trainer merger.)

In general, tfeager and tfgraph seem to look quite nice. One note, we don't need to actually do the tf.convert_to_tensor, as tf accepts np.ndarrays. But that is nitpicking.

I also remarked some points for discussion

[justusschock]

I'll have a look at all of them and either address them here or in separate PRs:

Regarding the conversation to tensor type: I added this on purpose, since explicit is better than implicit and one can do some optimization (and maybe even quantization) after converting to tensors. Also this shows where the conversion is done, as tensors and numpy arrays are not completely identical regarding their syntax. Also it is important to notice, that for tf eager one will probably start gradient taping right after this. But if you prefer removing this part it's fine for me.