Migration of the AzureOpenAiChatModel to use the Azure OpenAI SDK

[jdubois]

This PR fixes #325

• Migrate AzureOpenAiChatModel
• Migrate AzureOpenAiEmbeddingModel
• Migrate AzureOpenAiLanguageModel
• Migrate AzureOpenAiStreamingChatModel
• Migrate AzureOpenAiStreamingLanguageModel
• Add a full suite of tests

[clun]

wow if even rockstars start contributing here......

[jdubois]

Ahah I've had a good look at your work here @clun and I'll try to do the same for Azure Cognitive Search...

[geoand]

Really nice!

The only thing I can think of with this one is that it breaks integrations using dev.ai4j.openai4j.spi.OpenAiClientBuilderFactory because with this that client is swapped out and never comes into play.
From an integrator's perspective, the ideal thing here would be to have an Azure specific SPI that would allow us to use the previous client.
If this is not an option, we'll have to pretty much duplicate the functionality of this module in the quarkus-langchain4j extension.

[jdubois]

Thank you @geoand but why do you need this client? For Azure OpenAI, it's better to use the official client (and I'm not saying the client is better at all, just that it's the official one)

[geoand]

There are a few reasons:

• It reuses the network stack we have in Quarkus - this important for us for keeping runtime footprint (across various metrics) to a minimum
• It makes compilation to native-image without any necessary configuration
• It provides us out of the box observability features

[jdubois]

Thanks for insights, I had no idea this client was using some Quarkus-specific APIs. As the Azure SDK usually works very well with Spring, I'm guessing there will be the opposite argument for Spring users.
I believe it would be better to use the official Azure SDK here, just because it's official (again, not because it's better - and to be honest I don't believe it is!), and then have framework-specific optimisations in each framework (like quarkus-langchain4j). Then I'm concerned (for Quarkus and for Azure), if you need to re-implement the whole SDK just for this. Let me ping some friends inside Microsoft who work on both Quarkus and the Azure SDK, maybe they will have some ideas (hey @agoncal !!).

[geoand]

👍🏼

Just to clarify, it would work in Quarkus too, it just won't have all the bells and whistles I mentioned above.

[langchain4j]

@jdubois pardon me for the oversight, @geoand brought good points.

Existing client does not use/depend on any Quarkus-specific APIs, but it has an SPI that allows Quarkus to use their network stack and have all the benefits mentioned above. It does not mean that users of Spring will see any downsides.

I would propose to put this PR on hold for the moment and allow some time to explore options. Thanks for understanding!

[jdubois]

No it's indeed the same issue with Spring:

• With "ai4j" you're using Retrofit + OKHttp, and I'm guessing this works fine with Mutiny, so it's all good for Quarkus.
• With the Azure SDK, you're using Netty + Reactor, which means it's very good for Spring.

Both will work with the other framework, but they'll add more dependencies (and I'm guessing a second event loop), so it'll be less efficient for the "secondary" framework than for the "primary" framework.

Now I don't want to enter the framework wars, here the issue for me is whether you should be using the official Azure SDK, from the Azure team, to connect to Azure OpenAI. Or use the client from https://github.com/ai-for-java which is a totally unknown organization (their org page is empty).

[langchain4j]

@jdubois fair point. openai4j is ours, so I think I can adapt it to use Netty+Reactor by default, then both Spring and Quarkus users will benefit. What is the problem with using non-official library?

[geoand]

so I think I can adapt it to use Netty+Reactor by default

I think this is a good way to go

[jdubois]

@langchain4j it's weird, so you're 1 person and coded everything by yourself? You should clarify it somewhere, because I didn't understand at all that LangChain4J and ai-for-java belonged to the same person - you're not using the same GitHub ID for both, and there's no name anywhere, so there's no way to know.

Concerning using the official library:

1. It's first an issue of trust of your software supply chain, like here I have no idea who codes what and how things are maintained, when as an Azure customer people are looking for something they can trust. They are sending their secret keys and credentials through those librairies. They are running this code in production, and don't want any license issue.
2. Microsoft guarantees the maintenance and an SLA, and works on non-public stuff in advance, so by using the official library you should have more features and less bugs. I'm not saying it's better coded (It's not!!! As most of it is automatically generated), then that's also why I'm interested in LangChain4J (you provide a better quality, neutral abstraction layer).
3. If you're a user of Azure OpenAI, chances are that you're also using other parts of Microsoft cloud. Maybe some Cognitive Search (for the RAG pattern), Cosmos DB or Blob storage (for storing your files). All of those work using the same Azure SDK.
4. It's a mix of the 3 points above: if you want authentication with Active Directory, then you'll need to go use the Azure SDK. And many people will request something like this, they're not going to have their Azure OpenAI service open on the Internet or easily accessible, that's one of the main points of using Azure.

Anyway, I get your point in having something specific for Quarkus, so I have 2 other proposals for you:

• We keep both implementations here, in different directories. We should share the integration tests, and then it'll be easy for people to switch.
• I put this "langchain-azure-open-ai-official" library in a different repo, hopefully under the Azure organization. This would require some paperwork, and I'm guessing some collaboration on your side.

[geoand]

Thanks for elaborating @jdubois.

Just to be clear, this repo does not have anything specific to Quarkus. All it has is an SPI that can be used by code to swap out various parts - this is what the Quarkus extension uses.

[langchain4j]

@jdubois good point, I would indeed prefer using oficial SDK if I were Azure user. I think we can indeed use official Azure SDK for langchain4j-azure-open-ai module and keep using openai4j for langchain4j-open-ai. langchain4j-azure-open-ai in current state (using openai4j) will be moved as-is to langchain4j-quarkus and everyone will be happy.

[jdubois]

Oh yes totally agree @langchain4j !

[jdubois]

Thank you so much for the detailed review @langchain4j !!! I fixed most of the comments, except 2 discussions that are still opened, could you check them?

[jdubois]

Argh I'm seeing the license compliance with Logback. That's really bad, I didn't know about this! Let me check.

[jdubois]

So I believe the Logback license issue is a false positive, as it is dual licensed ( https://logback.qos.ch/license.html ) and as many projects (including Spring Boot) use it by default.
Let me check if I can fix this, otherwise we'll migrate to Log4J2, as this is only for testing this isn't really important.

[langchain4j]

I was not sure about using logback due to unclear license and I am no legal expert to risk it, so I just use tinylog which is apache 2:

            <dependency>
                <groupId>org.tinylog</groupId>
                <artifactId>tinylog-impl</artifactId>
                <scope>test</scope>
            </dependency>

            <dependency>
                <groupId>org.tinylog</groupId>
                <artifactId>slf4j-tinylog</artifactId>
                <scope>test</scope>
            </dependency>

But log4j also sounds good

[langchain4j]

To fix it you need to add an exclusion to the license plugin, see example in langchain4j-weaviate pom, but this is probably a bad idea. I would better keep compliance on and use another logger

[jdubois]

I migrated to Log4J2, as it's the other supported framework by Spring Boot. Anyway, it shouldn't be very important.

[langchain4j]

@jdubois awesome, thank you so much for your work and patience!

[jdubois]

Wonderful, thank you so much @langchain4j !!