TranformerTest

A quick and dirty training script to test (at least) one new variant of transformer block.

Todo

• Add option for n_program_tokens to be zero
  • Works out of the box.
• Add option for cross-attention lengths
• Try training on toy lm task
• Add option to run the compute block in recursion
• Create arithmatic task