.Net: Bug: Llama3.2-vision and Ollama Connector not working anymore.

[tennaito]

Describe the bug
The code below worked fine with llama3.2-vision in 1.31 versions (and the Ollama version from that time), but when upgraded to latest version it is not working anymore. Any directions?

I´ve have tested again on 1.31 and it did not worked now in the future... I am lost.

	public static async Task<string> AnalyzeImage(string text, ReadOnlyMemory<byte> image, string? mimeType, OllamaPromptExecutionSettings settings = null, CancellationToken cancellationToken = default)
	{
		settings ??= new OllamaPromptExecutionSettings();
		var kernel = CreateKernel(settings.ModelId ?? "llama3.2-vision");
		var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();
		ChatHistory input = new ChatHistory();
		ChatMessageContent content = new ChatMessageContent();  
		ChatMessageContentItemCollection items = new ChatMessageContentItemCollection();
		items.Add(new TextContent(text));
		items.Add(new ImageContent(image, mimeType));
		input.AddMessage(AuthorRole.User, items);
		ChatMessageContent result = await chatCompletionService.GetChatMessageContentAsync(input, settings, kernel);
		return result.Items.OfType<TextContent>().First().Text;
	}

	public static Kernel CreateKernel(string model = "llama3.1")
	{
		var builder = Kernel.CreateBuilder();
		builder.Services.AddTransient(s =>
		{
			var client = new HttpClient(new HttpClientHandler());
			client.Timeout = TimeSpan.FromMinutes(2);
			return client;
		});
		
		builder.Services.AddOllamaChatCompletion(model, new Uri("http://localhost:11434/"));
		return builder.Build();
	}

To Reproduce
Execute method AnalyzeImage.

Expected behavior
No error on version upgrade.

On the 1.31 version it returned the analysis of the image with the text context from user input (nowdays same error).
On the 1.56 version it returns:

Response status code does not indicate success: 500 (Internal Server Error).

at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at OllamaSharp.OllamaApiClient.EnsureSuccessStatusCodeAsync(HttpResponseMessage response)
at OllamaSharp.OllamaApiClient.SendToOllamaAsync(HttpRequestMessage requestMessage, OllamaRequest ollamaRequest, HttpCompletionOption completionOption, CancellationToken cancellationToken)
at OllamaSharp.OllamaApiClient.ChatAsync(ChatRequest request, CancellationToken cancellationToken)+MoveNext()
at OllamaSharp.OllamaApiClient.ChatAsync(ChatRequest request, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at OllamaSharp.IAsyncEnumerableExtensions.StreamToEndAsync[Tin,Tout](IAsyncEnumerable`1 stream, IAppender`2 appender, Action`1 itemCallback)
at OllamaSharp.IAsyncEnumerableExtensions.StreamToEndAsync[Tin,Tout](IAsyncEnumerable`1 stream, IAppender`2 appender, Action`1 itemCallback)
at OllamaSharp.OllamaApiClient.Microsoft.Extensions.AI.IChatClient.GetResponseAsync(IEnumerable`1 messages, ChatOptions options, CancellationToken cancellationToken)
at Microsoft.Extensions.AI.FunctionInvokingChatClient.GetResponseAsync(IEnumerable`1 messages, ChatOptions options, CancellationToken cancellationToken)
at Microsoft.SemanticKernel.ChatCompletion.ChatClientChatCompletionService.GetChatMessageContentsAsync(ChatHistory chatHistory, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)
at Microsoft.SemanticKernel.ChatCompletion.ChatCompletionServiceExtensions.GetChatMessageContentAsync(IChatCompletionService chatCompletionService, ChatHistory chatHistory, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)
at MyApplication.Util.OpenAIHelper.AnalyzeVision(String text, ReadOnlyMemory`1 image, String mimeType, OllamaPromptExecutionSettings settings, CancellationToken cancellationToken)

Platform that works

• Language: C#
• Source:
  Microsoft.SemanticKernel" Version="1.31.0"
  Microsoft.SemanticKernel.Connectors.Ollama" Version="1.31.0-alpha"
• AI model: llama3.2-vision
• IDE: Visual Studio
• OS: Windows

Platform that does not work

• Language: C#
• Source:
  Microsoft.SemanticKernel" Version="1.56.0"
  Microsoft.SemanticKernel.Connectors.Ollama" Version="1.56.0-alpha"
• AI model: llama3.2-vision
• IDE: Visual Studio
• OS: Windows

[rogeriorfp]

I tested your code and it worked. Which packages are you referencing?
Did you try cleaning the solution and rebuilding it?

[tennaito]

@rogeriorfp Did you test your code with version 1.56?

I have already cleaned, rebuilt, and locally deleted cached libraries.

[rogeriorfp]

@rogeriorfp Did you test your code with version 1.56?

I have already cleaned, rebuilt, and locally deleted cached libraries.

Yes @tennaito , I tested it with version 1.56.

<PackageReference Include="Microsoft.SemanticKernel" Version="1.56.0" />
<PackageReference Include="Microsoft.SemanticKernel.Connectors.Ollama" Version="1.56.0-alpha" />

[tennaito]

@rogeriorfp Maybe my server is too slow for the ChatClient, If you see my old code I´ve setted a timespan, but that HttpClient is unreachable now.
Did you know how can we set timeout for OllamaClient through SemanticKernel?

[tennaito]

I think I´ve reached the source of the error:

I changed the code a bit to insert the HttpClient:

  var httpClient = new HttpClient(new DebugMessageHandler(new HttpClientHandler()));
  httpClient.Timeout = TimeSpan.FromMinutes(2);
  httpClient.BaseAddress = new Uri("http://localhost:11434/");
  var client = new OllamaApiClient(httpClient, model);
  builder.Services.AddOllamaChatCompletion(client);

And I created the DebugMessageHandler to catch the request.

After I catch the request and sent directly to ollama it gets:

{
    "error": "'llama3.2-vision' is no longer compatible with your version of Ollama and has been replaced by a newer version. To re-download, run 'ollama pull llama3.2-vision'"
}

So, I have to probably reinstall the llama3.2-vision model on Ollama.

Thank you so much @rogeriorfp, I was really lost with that.

[tennaito]

But It is strange that the OllamaClient is not getting the HttpClient from Services.
And it is not propagating the error too.

That can be a bug, maybe?

[tennaito]

https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/Connectors/Connectors.Ollama/Extensions/OllamaServiceCollectionExtensions.DependencyInjection.cs

Line 40, It should use serviceProvider to get developer HttpClient.

[rogeriorfp]

@rogeriorfp Maybe my server is too slow for the ChatClient, If you see my old code I´ve setted a timespan, but that HttpClient is unreachable now. Did you know how can we set timeout for OllamaClient through SemanticKernel?

public static Kernel CreateKernel(string model = "llama3.1")
{
    var builder = Kernel.CreateBuilder();
    builder.Services.AddTransient(s =>
    {
        var client = new HttpClient(new HttpClientHandler());
        client.BaseAddress = new Uri("http://localhost:11434/");
        client.Timeout = TimeSpan.FromMinutes(2);
        return client;
    });

    builder.Services.AddOllamaChatCompletion(model);
    return builder.Build();
}

[rogeriorfp]

I think I´ve reached the source of the error:

I changed the code a bit to insert the HttpClient:

var httpClient = new HttpClient(new DebugMessageHandler(new HttpClientHandler()));
httpClient.Timeout = TimeSpan.FromMinutes(2);
httpClient.BaseAddress = new Uri("http://localhost:11434/");
var client = new OllamaApiClient(httpClient, model);
builder.Services.AddOllamaChatCompletion(client);
And I created the DebugMessageHandler to catch the request.

After I catch the request and sent directly to ollama it gets:

{
"error": "'llama3.2-vision' is no longer compatible with your version of Ollama and has been replaced by a newer version. To re-download, run 'ollama pull llama3.2-vision'"
}
So, I have to probably reinstall the llama3.2-vision model on Ollama.

Thank you so much @rogeriorfp, I was really lost with that.

No problem at all! Happy to help.