Generic AI client and models + open-ai client lib

[raulraja]

Delegates the open ai client runtime to https://github.com/aallam/ and reuses previous models as generic classes for xef core.

Adds a new hierarchy for LLM Models to be typed based on their capabilities. This prevents sending the wrong kind of model to an operation that does not support that capability.

For example the prompt function can only be invoked with a model that has serialization capabilities through functions.

@AiDsl
suspend inline fun <reified A> AIScope.prompt(
  question: String,
  ...
  model: LLM.ChatWithFunctions = this.defaultSerializationModel
  ...
): A

I plan to eliminate the AIClient in the next PR and implement the capabilities in the models directly to remove the client dependency from the scope. The client should not be a scoped object because each call can be to different models with different client implementations inside the AI block.

[raulraja]

@xebia-functional/team-ai

[diesalbla]

Would that be https://github.com/aallam/openai-kotlin?

[diesalbla]

To my limited understanding, seems ok...

[rafaparadela]

Besides a minor comment, the changes look good to me.

Note: I'm wondering if OpenAI-Kotlin implements Embeddings in a way that we can use in OpenAIEmbeddings.

[diesalbla]

Note: I'm wondering if OpenAI-Kotlin implements Embeddings in a way that we can use in OpenAIEmbeddings.

There is indeed a class called Embedding in that package, namely com.aallam.openai.api.embedding.Embedding, and it has a cosine similarity extension method in the package com.aallam.openai.client.extension, so some can delegate there.

[raulraja]

@nomisRev I adapted these changes to the latest from main after the Java stuff has been merged.

[nomisRev]

A couple of questions, and suggestions. Looks good overall. I am not sure I understand the dependency cycle between OpenAI and Core. It seems the dependencies need to be grouped in the top-level language specific modules if we want to provide top-level defaults rather than couple them in core.

[raulraja]

Besides a minor comment, the changes look good to me.

Note: I'm wondering if OpenAI-Kotlin implements Embeddings in a way that we can use in OpenAIEmbeddings.

This is how it is reflected now, the AIClient impl for OpenAI uses the library embeddings. Our embeddings delegate to the client embedding request, which delegates to the library.