[spike] evaluate + potentially prototype out-of-tree backend registration

[Titus-von-Koeller]

review:

• Challenges and Efforts in PyTorch Multi-Device Integration: Compatibility, Portability, and Integration Efficiencies
• Autoloading Out-of-Tree Extension
• vLLM as Ascend NPU and IBM Spyre as out-of-tree community supported plugins.
  • https://vllm-ascend.readthedocs.io/en/latest/

Matthew:

• "The way pytorch uses the package entry points in that blog is more or less what I saw adopted by vLLM."
• "And, as far as I am aware, as long as our custom ops are defined first and then we load a custom backend package, the entry point for that would be perfect capable of just registering its implementations for those ops. I think even it can override a backend implementation we provide, e.g. let's say a bitsandbytes-ipex has cpu optimizations. It would just have to be loaded after ours."
• "Regarding the out of tree topic on certain backends, this is also something I see on vLLM, just for reference. They have Ascend NPU and IBM Spyre as out-of-tree community supported plugins."

[Titus-von-Koeller]

https://github.com/bitsandbytes-foundation/bnb_initel