Roadmap

[coreylowman]

0.9.0 - nightly conv nets & transformers

• Transformers mega issue #34
• Add reshape function #55
• Convolution mega issue #1
• If param is not used in loss calculation, Optimizer::update() causes runtime panic #67
• Make nightly features appear in docs.rs #123
• Make docstrings consistent with comparisons to other framework apis #115
• Tensor with 2-max-values in Max Last stores wrong gradients #111
• Residual Connection with R(x) = F(x) + G(x) instead of R(x) = F(x) + x #96

Comparison against pytorch (patch version bump)

• Plot benchmark speed against pytorch #20
• Document differences from pytorch & other rust DL crates #22

Misc other generic const exprs functions (patch version bump)

• Add concatenate function #43
• Add flatten layer/functions #14

---

Released v0.5.1 - Mnist example with linear MLP

• add hard_cross_entropy #19
• Rename tape.add_operation to tape.add_backward_op #23
• Rename tape_holder to tape. #24
• Fix docstrings in macros #27
• Add Batch sampler utility class #26
• Add example training on mnist #7

Released v0.5.2 - RL examples & save/load

• Add dropout layers #13
• Add max_last_dim() #29
• Add gather_last_dim() #30
• Add DQN Example code (on random data) #31
• Save/load from numpy file #6
• Add example ppo #32

Released v0.6.0 - transformers prep & other additions

• Add Repeated<T, N> for repeating a module N times. #36
• (minor breaking change) Randomize parameters based on parameter size #41
• Add Binary Cross Entropy loss #40
• Add nn::DropoutOneIn<N> #38
• (patch breaking change) Clone for UniqueId should produce a different id #35
• Change Linear.weight to be transposed (for compatibility with torch) #33
• Add nn::LayerNorm #37
• Add *_broadcast_rhs_first functions #51
• (breaking) broadcast_inner_* should take NoTape as lhs, similar to other arith functions #44
• (breaking) Remove std::ops::Mul for matmul - matmul should always be explicit #46
• (breaking) broadcast_inner_sub should take ownership of both arguments  #47
• (breaking) Randomize parameters based on parameter size #41
• Add nn::Residual<T> #53

[coreylowman]

Closing this issue in favor of per release issues