Skip to content

Conversation

@thirdmartini
Copy link
Contributor

@thirdmartini thirdmartini commented May 8, 2025

Adds support for returning image content to ollama models. I'm not sure what the best long term solution for this is, but this works with multimodal models like mistral-small3.1 .

./mcphost --config ./mcphost.json -mollama:mistral-small3.1
2025/05/08 08:29:17 INFO Model loaded provider=ollama model=mistral-small3.1
2025/05/08 08:29:17 INFO Initializing server... name=example
2025/05/08 08:29:17 INFO Server connected name=example
2025/05/08 08:29:17 INFO Tools loaded server=example count=1

  You: describe whats on my front door camera
2025/05/08 08:29:26 INFO 🔧 Using tool name=example__getCameraImage

  Assistant:                                                                                                                                                                                                                                                            


  The image from the front door camera shows a person walking on a concrete path towards the door. The person is wearing a red sleeveless shirt and black shorts. There is greenery on both sides of the path and the person's shadow is cast on the path due to        
  sunlight. The sky appears to be clear.            

@ezynda3 ezynda3 merged commit 009beb5 into mark3labs:main May 13, 2025
@corani
Copy link
Contributor

corani commented May 14, 2025

@thirdmartini nice use-case! I guess it should be possible to give the LLM a reference image as well so it can describe the difference, rather than needlessly describing the "background".

@thirdmartini
Copy link
Contributor Author

@corani I've been unable to get any of the Ollama supported visual models to give me anything usable as a diff. But... it is quite good at being able to tell you that there is a "delivery person with a package" at the door, which makes it useful for smart alerts using dumb cameras.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants