Skip to content

Commit

Permalink
Refactoring of ChatClient to add fluent API and introduce Model as de…
Browse files Browse the repository at this point in the history
…pendent object

* Rename the ModelClient class hierarchy into Model:
  - Rename ModelClient into Model. Update all code and doc references.
  - Rename ChatClient to ChatModel. Update all ChatClient suffixes and chatClient fields and variables in code and doc.
  - Rename EmbeddingClient into EmbeddingModel. Update the XxxEmbeddingClient class and variable suffixes and embeddingClient variables and fields in code and docs.
  - Rename ImageClient into ImageModel.
  - Rename SpeechClient into SpeechModel.
  - Rename TranscriptionClient into TranscriptionModel.
  - Update all javadocs and antora pages. Update the related diagrams.

* Create fluent API in ChatClient interface that now includes streaming support
* Add OpenAI FunctionCallbackWrapper2IT auto-config tests.
* Add ChatClientTest mockito testing.
* Add ChatModel#getDefaultOptions(), and remove @FunctionalInterface

* ChatModel enums extend the new ModelDescription interface.
* Implement fromOptions copy method in every ChatOptions implementation.
* Extend ChatClient to use the model default options if not provided explicitly.

* Update readme to provide guidance on how to adapt to breaking changes.

Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
Co-authored-by: Mark Pollack <mpollack@vmware.com>
  • Loading branch information
3 people committed May 22, 2024
1 parent bce45c2 commit fbfc87e
Show file tree
Hide file tree
Showing 426 changed files with 4,825 additions and 2,922 deletions.
144 changes: 141 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,147 @@ Let's make your `@Beans` intelligent!

For further information go to our [Spring AI reference documentation](https://docs.spring.io/spring-ai/reference/).

### Breaking changes
(15.05.2024)
On our march to release 1.0 M1 we have made several breaking changes. Apologies, it is for the best!
## Breaking changes

On our march to release 1.0.0 M1 we have made several breaking changes. Apologies, it is for the best!

**(22.05.2024)**

A major change was made that took the 'old' `ChatClient` and moved the functionality into `ChatModel`. The 'new' `ChatClient` now takes an instance of `ChatModel`. This was done do support a fluent API for creating and executing prompts in a style similar to other client classes in the Spring ecosystem, such as `RestClient`, `WebClient`, and `JdbcClient`. Refer to the [JavaDoc](https://docs.spring.io/spring-ai/docs/1.0.0-SNAPSHOT/api/) for more information on the Fluent API, proper reference documentation is coming shortly.

We renamed the 'old' `ModelClient` to `Model` and renamed implementing classes, for example `ImageClient` was renamed to `ImageModel`. The `Model` implementation represent the portability layer that converts between the Spring AI API and the underlying AI Model API.

### Adapting to the changes

#### Approach 1

Now, instead of getting an Autoconfigured `ChatClient` instance, you will get a `ChatModel` instance. The `call` method signatures after renaming remain the same.
To adapt your code should refactor you code to change use of the type `ChatClient` to `ChatModel`
Here is an example of existing code before the change

```java
@RestController
public class OldSimpleAiController {

private final ChatClient chatClient;

@Autowired
public OldSimpleAiController(ChatClient chatClient) {
this.chatClient = chatClient;
}

@GetMapping("/ai/simple")
public Map<String, String> completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of("generation", chatClient.call(message));
}
}
```

Now after the changes this will be

```java
@RestController
public class SimpleAiController {

private final ChatModel chatModel;

@Autowired
public SimpleAiController(ChatModel chatModel) {
this.chatModel = chatModel;
}

@GetMapping("/ai/simple")
public Map<String, String> completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of("generation", chatModel.call(message));
}
}
```

NOTE: The renaming also applies to the classes
* `StreamingChatClient` -> `StreamingChatModel`
* `EmbeddingClient` -> `EmbeddingModel`
* `ImageClient` -> `ImageModel`
* `SpeechClient` -> `SpeechModel`
* and similar for other `<XYZ>Client` classes

#### Approach 2

In this approach you will use the new fluent API available on the 'new' `ChatClient`

Here is an example of existing code before the change

```java
@RestController
public class OldSimpleAiController {

private final ChatClient chatClient;

@Autowired
public OldSimpleAiController(ChatClient chatClient) {
this.chatClient = chatClient;
}

@GetMapping("/ai/simple")
public Map<String, String> completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of(
"generation",
chatClient.call(message)
);
}
}
```


Now after the changes this will be

```java
@RestController
public class SimpleAiController {

private final ChatClient chatClient;

@Autowired
public SimpleAiController(ChatClient chatClient) {
this.chatClient = chatClient;
}

@GetMapping("/ai/simple")
public Map<String, String> completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of(
"generation",
chatClient.prompt().user(message).call().content()
);
}
}
```

and in your `@Configuration` class you need to define an `@Bean` as shown below

```java
@Configuration
public class ApplicationConfiguration {

@Bean
ChatClient chatClient(ChatModel chatModel) {
return ChatClient.builder(chatModel).build();
}
}
```

NOTE: The `ChatModel` instance is made available to you through autoconfiguration.

#### Approach 3

There is a tag in the GitHub repository called [v1.0.0-SNAPSHOT-before-chatclient-changes](https://github.com/spring-projects/spring-ai/tree/v1.0.0-SNAPSHOT-before-chatclient-changes) that you can checkout and do a local build to avoid updating any of your code until you are ready to migrate your code base.

```bash
git checkout tags/v1.0.0-SNAPSHOT-before-chatclient-changes

./mvnw clean install -DskipTests
```


**(15.05.2024)**

Renamed POM artifact names:
- spring-ai-qdrant -> spring-ai-qdrant-store
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.ChatModel;
import reactor.core.publisher.Flux;

import org.springframework.ai.anthropic.api.AnthropicApi;
Expand All @@ -38,10 +39,9 @@
import org.springframework.ai.anthropic.api.AnthropicApi.StreamResponse;
import org.springframework.ai.anthropic.api.AnthropicApi.Usage;
import org.springframework.ai.anthropic.metadata.AnthropicChatResponseMetadata;
import org.springframework.ai.chat.ChatClient;
import org.springframework.ai.chat.ChatResponse;
import org.springframework.ai.chat.Generation;
import org.springframework.ai.chat.StreamingChatClient;
import org.springframework.ai.chat.StreamingChatModel;
import org.springframework.ai.chat.messages.MessageType;
import org.springframework.ai.chat.metadata.ChatGenerationMetadata;
import org.springframework.ai.chat.prompt.ChatOptions;
Expand All @@ -56,16 +56,16 @@
import org.springframework.util.CollectionUtils;

/**
* The {@link ChatClient} implementation for the Anthropic service.
* The {@link ChatModel} implementation for the Anthropic service.
*
* @author Christian Tzolov
* @since 1.0.0
*/
public class AnthropicChatClient extends
public class AnthropicChatModel extends
AbstractFunctionCallSupport<AnthropicApi.RequestMessage, AnthropicApi.ChatCompletionRequest, ResponseEntity<AnthropicApi.ChatCompletion>>
implements ChatClient, StreamingChatClient {
implements ChatModel, StreamingChatModel {

private static final Logger logger = LoggerFactory.getLogger(AnthropicChatClient.class);
private static final Logger logger = LoggerFactory.getLogger(AnthropicChatModel.class);

public static final String DEFAULT_MODEL_NAME = AnthropicApi.ChatModel.CLAUDE_3_OPUS.getValue();

Expand All @@ -89,10 +89,10 @@ public class AnthropicChatClient extends
public final RetryTemplate retryTemplate;

/**
* Construct a new {@link AnthropicChatClient} instance.
* Construct a new {@link AnthropicChatModel} instance.
* @param anthropicApi the lower-level API for the Anthropic service.
*/
public AnthropicChatClient(AnthropicApi anthropicApi) {
public AnthropicChatModel(AnthropicApi anthropicApi) {
this(anthropicApi,
AnthropicChatOptions.builder()
.withModel(DEFAULT_MODEL_NAME)
Expand All @@ -102,34 +102,34 @@ public AnthropicChatClient(AnthropicApi anthropicApi) {
}

/**
* Construct a new {@link AnthropicChatClient} instance.
* Construct a new {@link AnthropicChatModel} instance.
* @param anthropicApi the lower-level API for the Anthropic service.
* @param defaultOptions the default options used for the chat completion requests.
*/
public AnthropicChatClient(AnthropicApi anthropicApi, AnthropicChatOptions defaultOptions) {
public AnthropicChatModel(AnthropicApi anthropicApi, AnthropicChatOptions defaultOptions) {
this(anthropicApi, defaultOptions, RetryUtils.DEFAULT_RETRY_TEMPLATE);
}

/**
* Construct a new {@link AnthropicChatClient} instance.
* Construct a new {@link AnthropicChatModel} instance.
* @param anthropicApi the lower-level API for the Anthropic service.
* @param defaultOptions the default options used for the chat completion requests.
* @param retryTemplate the retry template used to retry the Anthropic API calls.
*/
public AnthropicChatClient(AnthropicApi anthropicApi, AnthropicChatOptions defaultOptions,
public AnthropicChatModel(AnthropicApi anthropicApi, AnthropicChatOptions defaultOptions,
RetryTemplate retryTemplate) {
this(anthropicApi, defaultOptions, retryTemplate, null);
}

/**
* Construct a new {@link AnthropicChatClient} instance.
* Construct a new {@link AnthropicChatModel} instance.
* @param anthropicApi the lower-level API for the Anthropic service.
* @param defaultOptions the default options used for the chat completion requests.
* @param retryTemplate the retry template used to retry the Anthropic API calls.
* @param functionCallbackContext the function callback context used to store the
* state of the function calls.
*/
public AnthropicChatClient(AnthropicApi anthropicApi, AnthropicChatOptions defaultOptions,
public AnthropicChatModel(AnthropicApi anthropicApi, AnthropicChatOptions defaultOptions,
RetryTemplate retryTemplate, FunctionCallbackContext functionCallbackContext) {

super(functionCallbackContext);
Expand Down Expand Up @@ -457,4 +457,9 @@ protected Flux<ResponseEntity<ChatCompletion>> doChatCompletionStream(ChatComple
"Streaming (stream=true) is not yet supported. We plan to add streaming support in a future beta version.");
}

@Override
public ChatOptions getDefaultOptions() {
return AnthropicChatOptions.fromOptions(this.defaultOptions);
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,11 @@ public class AnthropicChatOptions implements ChatOptions, FunctionCallingOptions
private @JsonProperty("top_k") Integer topK;

/**
* Tool Function Callbacks to register with the ChatClient. For Prompt
* Tool Function Callbacks to register with the ChatModel. For Prompt
* Options the functionCallbacks are automatically enabled for the duration of the
* prompt execution. For Default Options the functionCallbacks are registered but
* disabled by default. Use the enableFunctions to set the functions from the registry
* to be used by the ChatClient chat completion requests.
* to be used by the ChatModel chat completion requests.
*/
@NestedConfigurationProperty
@JsonIgnore
Expand Down Expand Up @@ -223,4 +223,17 @@ public void setFunctions(Set<String> functions) {
this.functions = functions;
}

public static AnthropicChatOptions fromOptions(AnthropicChatOptions fromOptions) {
return builder().withModel(fromOptions.getModel())
.withMaxTokens(fromOptions.getMaxTokens())
.withMetadata(fromOptions.getMetadata())
.withStopSequences(fromOptions.getStopSequences())
.withTemperature(fromOptions.getTemperature())
.withTopP(fromOptions.getTopP())
.withTopK(fromOptions.getTopK())
.withFunctionCallbacks(fromOptions.getFunctionCallbacks())
.withFunctions(fromOptions.getFunctions())
.build();
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import org.springframework.ai.model.ModelDescription;
import org.springframework.ai.model.ModelOptionsUtils;
import org.springframework.ai.retry.RetryUtils;
import org.springframework.http.HttpHeaders;
Expand Down Expand Up @@ -116,7 +117,7 @@ public AnthropicApi(String baseUrl, String anthropicApiKey, String anthropicVers
* "https://docs.anthropic.com/claude/docs/models-overview#model-comparison">model
* comparison</a> for additional details and options.
*/
public enum ChatModel {
public enum ChatModel implements ModelDescription {

// @formatter:off
CLAUDE_3_OPUS("claude-3-opus-20240229"),
Expand All @@ -140,6 +141,11 @@ public String getValue() {
return this.value;
}

@Override
public String getModelName() {
return this.value;
}

}

/**
Expand Down
Loading

0 comments on commit fbfc87e

Please sign in to comment.