Half float checkpoint

[dcslin]

added half cpp backend, half tensor conversion, half tensor math operations, added half to linear, softmax layer, cnn layers, opt module. tested examples naive, module, train_cnn on mlp and cnn, refactored broadcast, added benchmark for fp16 vs fp32

[dcslin]

Usage example:

root@1c6aaef3db53:~/singa-hpmerge# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py cnn mnist -m3 -pfloat16 -i 7
Starting Epoch 0:
Training loss = 0.061211, training accuracy = 0.752235
Evaluation accuracy = 0.944912, Elapsed Time = 6.867703s
Starting Epoch 1:
Training loss = 237.194122, training accuracy = 0.920124
Evaluation accuracy = 0.957131, Elapsed Time = 6.740274s
Starting Epoch 2:
Training loss = 167.028595, training accuracy = 0.944337
Evaluation accuracy = 0.970353, Elapsed Time = 6.833484s
root@1c6aaef3db53:~/singa-hpmerge# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py cnn mnist -m3 -pfloat32 -i 7
Starting Epoch 0:
Training loss = 596.964600, training accuracy = 0.789421
Evaluation accuracy = 0.943209, Elapsed Time = 5.502895s
Starting Epoch 1:
Training loss = 234.664322, training accuracy = 0.920758
Evaluation accuracy = 0.960036, Elapsed Time = 5.506354s
Starting Epoch 2:
Training loss = 165.501694, training accuracy = 0.944454
Evaluation accuracy = 0.971254, Elapsed Time = 5.482749s

[codecov]

Codecov Report

Merging #797 into dev will decrease coverage by 6.30%.
The diff coverage is 20.00%.

@@            Coverage Diff             @@
##              dev     #797      +/-   ##
==========================================
- Coverage   70.05%   63.74%   -6.31%     
==========================================
  Files         100       87      -13     
  Lines       11573     4904    -6669     
==========================================
- Hits         8107     3126    -4981     
+ Misses       3466     1778    -1688     

Flag	Coverage Δ
#singa-cpp	63.74% <20.00%> (-1.35%)	⬇️
#singa-python	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
include/half.hpp	0.00% <ø> (ø)
include/singa/core/common.h	100.00% <ø> (ø)
include/singa/core/tensor.h	73.21% <ø> (ø)
src/core/tensor/tensor_math.h	5.03% <0.00%> (-0.08%)	⬇️
src/model/operation/convolution.cc	90.00% <ø> (ø)
src/core/tensor/tensor_math_cpp.h	63.04% <20.68%> (-3.02%)	⬇️
src/core/tensor/tensor.cc	75.75% <25.00%> (-0.80%)	⬇️
sonnx.py
opt.py
autograd.py
... and 12 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95f7b39...b439aac. Read the comment docs.

[nudles]

put it under singa/core/ ?