Skip to content

Commit

Permalink
refactor!: Remove deprecated generate and predict APIs (#335)
Browse files Browse the repository at this point in the history
  • Loading branch information
davidmigloz committed Feb 17, 2024
1 parent ab3ab57 commit c55fe50
Show file tree
Hide file tree
Showing 30 changed files with 172 additions and 329 deletions.
3 changes: 1 addition & 2 deletions docs/modules/model_io/model_io.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Model I/O

The core element of any language model application is... the model. LangChain
gives you the building blocks to interface with any language model.
The core element of any language model application is...the model. LangChain gives you the building blocks to interface with any language model.

- [Prompts](/modules/model_io/prompts/prompts.md): templatize, dynamically
select, and manage model inputs.
Expand Down
38 changes: 15 additions & 23 deletions docs/modules/model_io/models/chat_models/chat_models.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ import 'package:langchain_openai/langchain_openai.dart';

We can then instantiate the chat model:
```dart
final chat = final chat = ChatOpenAI(
final chatModel = final chat = ChatOpenAI(
apiKey: openaiApiKey,
defaultOptions: const ChatOpenAIOptions(
temperature: 0,
Expand All @@ -48,38 +48,39 @@ types of messages currently supported in LangChain are `AIChatMessage`,
you’ll just be dealing with `HumanChatMessage`, `AIChatMessage`, and
`SystemChatMessage`.

### `call`: messages in -> message out
### LCEL

You can get chat completions by passing one or more messages to the chat model.
The response will be a message.
LLMs implement the `Runnable` interface, the basic building block of the LangChain Expression Language (LCEL). This means they support `invoke`, `stream`, and `batch` calls.

```dart
final messages = [
ChatMessage.humanText(
'Translate this sentence from English to French. I love programming.',
),
];
final chatRes = await chat(messages);
// -> AIChatMessage{content: J'aime programmer., example: false}
final prompt = PromptValue.chat(messages);
final chatRes = await chatModel.invoke(prompt);
// -> [ChatGeneration{
// output: AIChatMessage{content: J'adore la programmation., example: false},
// generationInfo: {index: 0, finish_reason: stop}}]
```

OpenAI’s chat model supports multiple messages as input.
See [here](https://platform.openai.com/docs/guides/gpt/chat-completions-vs-completions) for more
information. Here is an example of sending a system and user message to the chat model:
OpenAI’s chat model supports multiple messages as input. See [here](https://platform.openai.com/docs/guides/gpt/chat-completions-vs-completions) for more information. Here is an example of sending a system and user message to the chat model:

```dart
final messages = [
ChatMessage.system('You are a helpful assistant that translates English to French.'),
ChatMessage.humanText('I love programming.')
];
final chatRes = await chat(messages);
final prompt = PromptValue.chat(messages);
final chatRes = await chatModel.invoke(prompt);
print(chatRes);
// -> AIChatMessage{content: J'adore la programmation., example: false}
// -> [ChatGeneration{
// output: AIChatMessage{content: J'adore la programmation., example: false},
// generationInfo: {index: 0, finish_reason: stop}}]
```

### `generate`: richer outputs

The `generate` APIs return an `ChatResult` which contains a `ChatGeneration`
The `invoke` API return an `ChatResult` which contains a `ChatGeneration`
object with the `output` messages and some metadata about the generation. It
also contains some additional information like `usage` and `modelOutput`.

Expand All @@ -94,12 +95,3 @@ print(chatRes1.usage?.totalTokens);
print(chatRes1.modelOutput);
// -> {id: chatcmpl-7QHTjpTCELFuGbxRaazFqvYtepXOc, created: 2023-06-11 17:41:11.000, model: gpt-3.5-turbo}
```

### `generatePrompt`: generate from a `PromptValue`

```dart
final chatRes2 = await chat.generatePrompt(ChatPromptValue(messages));
print(chatRes2.generations);
print(chatRes2.usage);
print(chatRes2.modelOutput);
```
2 changes: 2 additions & 0 deletions docs/modules/model_io/models/chat_models/how_to/llm_chain.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# LLMChain

> DEPRECATED: `LLMChain` is deprecated in favour of LangChain Expression Language (LCEL).
You can use the existing `LLMChain` in a very similar way as
[with LLMs](/modules/chains/chains?id=get-started) - provide a prompt and a
model.
Expand Down
3 changes: 0 additions & 3 deletions docs/modules/model_io/models/chat_models/how_to/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@

Some chat models provide a streaming response. This means that instead of waiting for the entire response to be returned, you can start processing it as soon as it's available. This is useful if you want to display the response to the user as it's being generated, or if you want to process the response as it's being generated.

Currently, it is supported for the following chat models:
- `ChatOpenAI`

Example usage:

```dart
Expand Down
3 changes: 0 additions & 3 deletions docs/modules/model_io/models/llms/how_to/llm_streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@

Some LLMs provide a streaming response. This means that instead of waiting for the entire response to be returned, you can start processing it as soon as it's available. This is useful if you want to display the response to the user as it's being generated, or if you want to process the response as it's being generated.

Currently, it is supported for the following LLMs:
- `OpenAI`

Example usage:

```dart
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ final openai = OpenAI(
apiKey: openaiApiKey,
defaultOptions: const OpenAIOptions(temperature: 0.9),
);
final result = await openai.generate('Tell me a joke');
final prompt = PromptValue.string('Tell me a joke');
final result = await llm.invoke(prompt);
final usage = result.usage;
print(usage?.promptTokens); // 4
print(usage?.responseTokens); // 20
Expand Down
33 changes: 7 additions & 26 deletions docs/modules/model_io/models/llms/llms.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,33 +49,20 @@ import 'package:langchain_openai/langchain_openai.dart';
final llm = OpenAI(apiKey: openaiApiKey);
```

### `call`: string in -> string out
### LCEL

The simplest way to use an LLM is a callable: pass in a string, get a string
completion.
LLMs implement the `Runnable` interface, the basic building block of the LangChain Expression Language (LCEL). This means they support `invoke`, `stream`, and `batch` calls.

```dart
final llmRes = await llm('Tell me a joke');
print(llmRes); // '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'
final prompt = PromptValue.string('Tell me a joke');
final result = await llm.invoke(prompt);
// LLMGeneration(output='\n\nWhy did the chicken cross the road?\n\nTo get to the other side!')
```

### `generate`: batch calls, richer outputs

`generate` lets you can call the model with a list of strings, getting back a
more complete response than just the text. This complete response can include
things like multiple top responses and other LLM provider-specific information:

```dart
final llmRes = await llm.generate('Tell me a joke');
print(llmRes.generations.first);
// [LLMGeneration(output='\n\nWhy did the chicken cross the road?\n\nTo get to the other side!')]
```

`usage` field contains the amount of tokens used for the generation. This is useful for
tracking usage and billing.
The response is a `LLMGeneration` object, which contains the output of the model, as well as other useful information like the amount of tokens used for the generation.

```dart
print(llmRes.usage?.totalUsage); // 641
print(result.usage?.totalUsage); // 641
```

You can also access provider specific information that is returned.
Expand All @@ -89,9 +76,3 @@ print(llmRes.modelOutput);
// model: text-ada-001
// }
```

### `generatePrompt`: generate from a `PromptValue`

```dart
final llmRes = await llm.generatePrompt(const StringPromptValue('Tell me a joke.'));
```
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
import 'dart:io';

import 'package:langchain/langchain.dart';
import 'package:langchain_chroma/langchain_chroma.dart';
import 'package:langchain_openai/langchain_openai.dart';

void main(final List<String> arguments) async {
Expand Down Expand Up @@ -76,7 +75,8 @@ Future<void> _ragSearch() async {
documents: [
const Document(pageContent: 'LangChain was created by Harrison'),
const Document(
pageContent: 'David ported LangChain to Dart in LangChain.dart'),
pageContent: 'David ported LangChain to Dart in LangChain.dart',
),
],
);

Expand Down
2 changes: 1 addition & 1 deletion packages/langchain/lib/src/chains/llm_chain.dart
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ class LLMChain<
Future<ChainValues> callInternal(final ChainValues inputs) async {
final promptValue = prompt.formatPrompt(inputs);

final response = await llm.generatePrompt(promptValue, options: llmOptions);
final response = await llm.invoke(promptValue, options: llmOptions);

final res = outputParser == null
? response.generations.firstOrNull?.output
Expand Down
85 changes: 5 additions & 80 deletions packages/langchain/lib/src/model_io/chat_models/base.dart
Original file line number Diff line number Diff line change
Expand Up @@ -28,45 +28,9 @@ abstract class BaseChatModel<Options extends ChatModelOptions>
Future<ChatResult> invoke(
final PromptValue input, {
final Options? options,
}) async {
return generatePrompt(input, options: options);
}

/// Runs the chat model on the given messages.
///
/// - [messages] The messages to pass into the model.
/// - [options] Generation options to pass into the Chat Model.
///
/// Example:
/// ```dart
/// final result = await chat.generate([ChatMessage.humanText('say hi!')]);
/// ```
@override
Future<ChatResult> generate(
final List<ChatMessage> messages, {
final Options? options,
});

/// Runs the chat model on the given prompt value.
///
/// - [promptValue] The prompt value to pass into the model.
/// - [options] Generation options to pass into the Chat Model.
///
/// Example:
/// ```dart
/// final result = await chat.generatePrompt(
/// PromptValue.chat([ChatMessage.humanText('say hi!')]),
/// );
/// ```
@override
Future<ChatResult> generatePrompt(
final PromptValue promptValue, {
final Options? options,
}) {
return generate(promptValue.toChatMessages(), options: options);
}

/// Runs the chat model on the given messages.
/// Runs the chat model on the given messages and returns a chat message.
///
/// - [messages] The messages to pass into the model.
/// - [options] Generation options to pass into the Chat Model.
Expand All @@ -80,48 +44,9 @@ abstract class BaseChatModel<Options extends ChatModelOptions>
final List<ChatMessage> messages, {
final Options? options,
}) async {
final result = await generate(messages, options: options);
final result = await invoke(PromptValue.chat(messages), options: options);
return result.generations[0].output;
}

/// Runs the chat model on the given text as a human message and returns the
/// content of the AI message.
///
/// - [text] The text to pass into the model.
/// - [options] Generation options to pass into the Chat Model.
///
/// Example:
/// ```dart
/// final result = await predict('say hi!');
/// ```
@override
Future<String> predict(
final String text, {
final Options? options,
}) async {
final res = await call(
[ChatMessage.humanText(text)],
options: options,
);
return res.content;
}

/// Runs the chat model on the given messages (same as [call] method).
///
/// - [messages] The messages to pass into the model.
/// - [options] Generation options to pass into the Chat Model.
///
/// Example:
/// ```dart
/// final result = await chat.predictMessages([ChatMessage.humanText('say hi!')]);
/// );
@override
Future<ChatMessage> predictMessages(
final List<ChatMessage> messages, {
final Options? options,
}) async {
return call(messages, options: options);
}
}

/// {@template simple_chat_model}
Expand All @@ -135,11 +60,11 @@ abstract class SimpleChatModel<Options extends ChatModelOptions>
const SimpleChatModel();

@override
Future<ChatResult> generate(
final List<ChatMessage> messages, {
Future<ChatResult> invoke(
final PromptValue input, {
final Options? options,
}) async {
final text = await callInternal(messages, options: options);
final text = await callInternal(input.toChatMessages(), options: options);
final message = AIChatMessage(content: text);
return ChatResult(
generations: [ChatGeneration(message)],
Expand Down
32 changes: 10 additions & 22 deletions packages/langchain/lib/src/model_io/language_models/base.dart
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import 'package:meta/meta.dart';

import '../../core/core.dart';
import '../chat_models/models/models.dart';
import '../prompts/models/models.dart';
import 'models/models.dart';

Expand All @@ -22,36 +21,25 @@ abstract class BaseLanguageModel<Input extends Object,
/// Return type of language model.
String get modelType;

/// Runs the language model on the given input.
Future<LanguageModelResult<Output>> generate(
final Input input, {
final Options? options,
});

/// Runs the language model on the given prompt value.
Future<LanguageModelResult<Output>> generatePrompt(
final PromptValue promptValue, {
/// Runs the Language Model on the given prompt value.
///
/// - [input] The prompt value to pass into the model.
/// - [options] Generation options to pass into the model.
@override
Future<LanguageModelResult<Output>> invoke(
final PromptValue input, {
final Options? options,
});

/// Runs the language model on the given input.
///
/// - [input] The prompt to pass into the model.
/// - [options] Generation options to pass into the model.
Future<Output> call(
final Input input, {
final Options? options,
});

/// Predicts text from text.
Future<String> predict(
final String text, {
final Options? options,
});

/// Predicts a chat message from chat messages.
Future<ChatMessage> predictMessages(
final List<ChatMessage> messages, {
final Options? options,
});

/// Tokenizes the given prompt using the encoding used by the language
/// model.
///
Expand Down
Loading

0 comments on commit c55fe50

Please sign in to comment.