update message processing

[mxyng]

this change changes the way messages are processed before handing off to the llm. there are a few areas worth mentioning:

1. messages are now a first class component of the template. template rendering will only falling back to the previous iterative template if messages is unsupported by the template. however, new models should implement the previous prompt/response template for compatibility with older ollama versions

2. the generate endpoint has been updated to use messages for prompt templating but the end result should be the same

3. the chat endpoint has been updated to preprocess incoming messages

   • continuous messages of the same role are joined into a single message, separated with two newlines
   • content and image data can be interleaved by sending messages with alternating fields, e.g.

     [
         {"role": "user", "content": "Consider the following images:"},
         {"role": "user", "images": ["<base64 image data>", "<base64 image data>"]},
         {"role": "user", "content": "What is the difference between the two images?"}
     ]
     

   • system messages are aggregated and prepended to the last user message

[bmizerany]

Very nice! Thank you.

[BruceMacD]

Last comment about the response code in an error case, but otherwise this is looking ready to me.

[BruceMacD]

This looks good, excited for this one

[jmorganca]

Passing in caps is weird here – shouldn't the caller do that before scheduling the runner?

[mxyng]

the caller doesn't have access to the model so it can't check capabilities

[jmorganca]

It did before this change

Could we load the model and check the capabilities before deciding to schedule the model here? and then pass in the Model? It seems like a precondition the handler should check (ideally this function goes away and we do something like schduler.Schedule(model, opts) from the handler which would be a lot clearer

[mxyng]

maybe the function is misnamed. this doesn't schedule anything until some check including capabilities. it aggregates the various model and options related lines that chat, generate, and embeddings all have to call before scheduling a runner

[jmorganca]

This is looking good. Love that messages are becoming a first-class part of the templating.

[jmorganca]

Overall looks great- small comment on the templating so we don't overcommit to helpers yet unless we really have to

[jmorganca]

LGTM. Will test this a bunch this evening (no need to block merging) but it looks great