DeltaFormers

Part 1: The State CLEVR dataset

---

In order to pretrain the model, the state only CLEVR dataset is used

Part 2: The MultiModal Embedder

---

In order to mix modalities the MultiModal Embedding Module is Implemented

Part 3: Stack of Bert Encoders + NLL task

---

The task is pretrained with only one of the available outputs.

TODO

---

Implementation of the "Select and Move Transformer"