about transformer framework

[chenweihua91]

Is there a plan to support the transformer framework?

[jiaxiang-wu]

Yes, we have plans to support the Transformer model (or RNN/LSTM-based models), in the next few months. If you are interested, you may also join us and extend PocketFlow to compress the Transformer model.

[jiaxiang-wu]

Enhancement required: add support for Transformer and RNN/LSTM-based models for NLP application.

[GaofengCheng]

@jiaxiang-wu any progress in supporting LSTM/GRU? I'm from ASR filed and very interested in seeing the effect of pocketflow in LSTM/GRU.

[jiaxiang-wu]

@GaofengCheng We are currently working on object detection models, and after that, we will switch to RNN-like models (including LSTM & GRU). If interested, your contribution is also welcomed.