A tutorial by the Runhouse team to help you get started with multimodal conversational models using the open-source LLaVA-1.5 model and the open-source Runhouse library.
For more information please reference our blog post
Update - Llava was recently released in Hugging Face Transformers, lending to much simpler code. You can find an example in llava_chat_transformers.py.