Example: inference OPT-13B in kaggle with benchmarks

[justheuristic]

Model: https://huggingface.co/facebook/opt-13b
Smaller model for experiments: https://huggingface.co/facebook/opt-350m

It would be great to allow inference for this model

Steps:

• write a config in ./tensor_parallel/slicing_configs.py
  • make sure you split / gather KV similarly to BloomModel
• find a way to shard a checkpoint such that the model is fully loaded with kaggle's limited ram
  • please ping @me or @BlackSamorez on t5 checkpoint example
• benchmark performance
  • baseline: model.parallelize
  • inference, batch size = 1 sequence
  • forward no_grad, batch = 4x 64 tokens (if it fits!)
  • forward+backward batch = 4x 64 tokens (if it fits!), make sure you model.gradient_checkpointing_enable() in both cases