-
Notifications
You must be signed in to change notification settings - Fork 1.3k
docs: Added Amazon Nova provider documentation #1696
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+371
−0
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
c656303
Added Amazon Nova provider documentation
5herlocked 9491ea8
some finetuning
5herlocked f8e0e4e
Merge branch 'main' into main
5herlocked 5443216
Update src/oss/python/integrations/chat/amazon_nova.mdx
5herlocked 0897b54
additional fine tuning
5herlocked 97e55d1
Merge branch 'main' into main
5herlocked 3bc8b46
nits
ccurme File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,358 @@ | ||
| --- | ||
| title: ChatAmazonNova | ||
| description: Get started using Amazon Nova [chat models](/oss/langchain/models) in LangChain. | ||
| --- | ||
|
|
||
| This guide provides a quick overview for getting started with Amazon Nova chat models. Amazon Nova models are OpenAI-compatible and accessed via the OpenAI SDK pointed at Nova's endpoint, providing seamless integration with LangChain's standard interfaces. | ||
|
|
||
| You can find information about Amazon Nova's models, their features, and API details in the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation). | ||
|
|
||
| <Tip> | ||
| **API Reference** | ||
|
|
||
| For detailed documentation of all @[`ChatAmazonNova`] features and configuration options, head to the @[`ChatAmazonNova`] API reference. | ||
|
|
||
| For Amazon Nova model details and capabilities, see the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation). | ||
| </Tip> | ||
|
|
||
| ## Overview | ||
|
|
||
| ### Integration details | ||
|
|
||
| | Class | Package | <Tooltip tip="Can run on local hardware" cta="Learn more" href="/oss/langchain/models#local-models">Local</Tooltip> | Serializable | JS/TS Support | Downloads | Latest Version | | ||
| | :--- | :--- | :---: | :---: | :---: | :---: | :---: | | ||
| | @[`ChatAmazonNova`] | @[`langchain-amazon-nova`] | ❌ | beta | ❌ |  |  | | ||
|
|
||
| ### Model features | ||
|
|
||
| | [Tool calling](/oss/langchain/tools) | [Structured output](/oss/langchain/structured-output) | JSON mode | [Image input](/oss/langchain/messages#multimodal) | Audio input | Video input | [Token-level streaming](/oss/langchain/streaming/) | Native async | [Token usage](/oss/langchain/models#token-usage) | [Logprobs](/oss/langchain/models#log-probabilities) | | ||
| | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | ||
| | ✅ | ❌ | ❌ | Model-dependent | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | | ||
|
|
||
| ## Setup | ||
|
|
||
| To access Amazon Nova models, you'll need to obtain API credentials and install the @[`langchain-amazon-nova`] integration package. | ||
|
|
||
| ### Installation | ||
|
|
||
| <CodeGroup> | ||
| ```bash pip | ||
| pip install -U langchain-amazon-nova | ||
| ``` | ||
| ```bash uv | ||
| uv add langchain-amazon-nova | ||
| ``` | ||
| </CodeGroup> | ||
|
|
||
| ### Credentials | ||
|
|
||
| Set your Nova API credentials as environment variables: | ||
|
|
||
| ```python | ||
| import getpass | ||
| import os | ||
|
|
||
| if "NOVA_API_KEY" not in os.environ: | ||
| os.environ["NOVA_API_KEY"] = getpass.getpass("Enter your Nova API key: ") | ||
|
|
||
| if "NOVA_BASE_URL" not in os.environ: | ||
| os.environ["NOVA_BASE_URL"] = getpass.getpass("Enter your Nova base URL: ") | ||
| ``` | ||
|
|
||
| To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key: | ||
|
|
||
| ```python | ||
| os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ") | ||
| os.environ["LANGSMITH_TRACING"] = "true" | ||
| ``` | ||
|
|
||
| ## Instantiation | ||
|
|
||
| Now we can instantiate our model object and generate chat completions: | ||
|
|
||
| ```python | ||
| from langchain_amazon_nova import ChatAmazonNova | ||
|
|
||
| model = ChatAmazonNova( | ||
| model="nova-2-lite-v1", | ||
| temperature=0.7, | ||
| max_tokens=2048, | ||
| timeout=None, | ||
| max_retries=2, | ||
| # other params... | ||
| ) | ||
| ``` | ||
|
|
||
| <Info> | ||
| For a complete list of supported parameters and their descriptions, see the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation). | ||
| </Info> | ||
|
|
||
| ## Invocation | ||
|
|
||
| ```python | ||
| messages = [ | ||
| ( | ||
| "system", | ||
| "You are a helpful assistant that translates English to French. Translate the user sentence.", | ||
| ), | ||
| ("human", "I love programming."), | ||
| ] | ||
| ai_msg = model.invoke(messages) | ||
| ai_msg | ||
| ``` | ||
|
|
||
| ```output | ||
| AIMessage(content="J'adore la programmation.", response_metadata={'model': 'nova-2-lite-v1', 'finish_reason': 'stop'}, id='run-12345678-1234-1234-1234-123456789abc', usage_metadata={'input_tokens': 29, 'output_tokens': 8, 'total_tokens': 37}) | ||
| ``` | ||
|
|
||
| ```python | ||
| print(ai_msg.content) | ||
| ``` | ||
|
|
||
| ```output | ||
| J'adore la programmation. | ||
| ``` | ||
|
|
||
| ## Content blocks | ||
|
|
||
| Amazon Nova messages can contain either a single string or a list of content blocks. You can access standardized content blocks using the `content_blocks` property: | ||
|
|
||
| ```python | ||
| ai_msg.content_blocks | ||
| ``` | ||
|
|
||
| Using `content_blocks` will render the content in a standard format that is consistent across other model providers. Read more about [content blocks](/oss/langchain/messages#standard-content-blocks). | ||
|
|
||
| ## Streaming | ||
|
|
||
| Amazon Nova supports token-level streaming for real-time response generation: | ||
|
|
||
| ```python | ||
| for chunk in model.stream(messages): | ||
| print(chunk.content, end="", flush=True) | ||
| ``` | ||
|
|
||
| ```output | ||
| J'adore la programmation. | ||
| ``` | ||
|
|
||
| ### Async streaming | ||
|
|
||
| For async applications, use `astream`: | ||
|
|
||
| ```python | ||
| import asyncio | ||
|
|
||
| async def main(): | ||
| async for chunk in model.astream(messages): | ||
| print(chunk.content, end="", flush=True) | ||
|
|
||
| asyncio.run(main()) | ||
| ``` | ||
|
|
||
| ## Tool calling | ||
|
|
||
| Amazon Nova supports tool calling (function calling) on compatible models. You can check if a model supports tool calling using LangChain model profiles. | ||
|
|
||
| <Info> | ||
| For details on Nova's tool calling implementation and available parameters, see the [tool calling documentation](https://nova.amazon.com/dev/documentation). | ||
| </Info> | ||
|
|
||
| ### Basic tool usage | ||
|
|
||
| Bind tools to the model using Pydantic models or LangChain @[`@tool`]: | ||
|
|
||
| ```python | ||
| from pydantic import BaseModel, Field | ||
|
|
||
| class GetWeather(BaseModel): | ||
| """Get the weather for a location.""" | ||
|
|
||
| location: str = Field(..., description="City name") | ||
|
|
||
| model_with_tools = model.bind_tools([GetWeather]) | ||
| response = model_with_tools.invoke("What's the weather in Paris?") | ||
| print(response.tool_calls) | ||
| ``` | ||
|
|
||
| ```output | ||
| [{'name': 'GetWeather', 'args': {'location': 'Paris'}, 'id': 'call_abc123', 'type': 'tool_call'}] | ||
| ``` | ||
|
|
||
| You can also access tool calls specifically in a standard format using the `tool_calls` attribute: | ||
|
|
||
| ```python | ||
| response.tool_calls | ||
| ``` | ||
|
|
||
| ```output | ||
| [{'name': 'GetWeather', | ||
| 'args': {'location': 'Paris'}, | ||
| 'id': 'call_abc123', | ||
| 'type': 'tool_call'}] | ||
| ``` | ||
|
|
||
| ### Using LangChain tools | ||
|
|
||
| You can also use standard LangChain tools: | ||
|
|
||
| ```python | ||
| from langchain.tools import tool | ||
|
|
||
| @tool | ||
| def get_weather(location: str) -> str: | ||
| """Get the weather for a location.""" | ||
| return f"Weather in {location}: Sunny, 72°F" | ||
|
|
||
| model_with_tools = model.bind_tools([get_weather]) | ||
| response = model_with_tools.invoke("What's the weather in San Francisco?") | ||
| ``` | ||
|
|
||
| ### Strict tool binding | ||
|
|
||
| By default, @[`BaseChatModel.bind_tools`] validates that the model supports tool calling. To disable this validation: | ||
|
|
||
| ```python | ||
| model_with_tools = model.bind_tools([GetWeather], strict=False) | ||
| ``` | ||
|
|
||
| ## System tools | ||
|
|
||
| Amazon Nova provides built-in system tools that can be enabled by passing them to the model initialization. See the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation) for available system tools and their capabilities. | ||
|
|
||
| ```python | ||
| from langchain_amazon_nova import ChatAmazonNova | ||
|
|
||
| model = ChatAmazonNova( | ||
| model="nova-2-lite-v1", | ||
| system_tools=["nova_grounding", "nova_code_interpreter"], | ||
| ) | ||
| ``` | ||
|
|
||
| <Info> | ||
| **System tools** | ||
|
|
||
| System tools like `nova_grounding` and `nova_code_interpreter` provide built-in capabilities. For details on available system tools and their usage, see the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation). | ||
| </Info> | ||
|
|
||
| ## Model Profile | ||
|
|
||
| Amazon Nova provides different models with varying capabilities. It includes support for LangChain [model profiles](/oss/langchain/models#model-profiles). | ||
|
|
||
| <Info> | ||
| **Model capabilities vary by model** | ||
|
|
||
| Some Amazon Nova models support vision inputs while others do not. Always check model capabilities before using multimodal features. | ||
|
|
||
| For a complete list of available models and their capabilities, see the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation). | ||
| </Info> | ||
|
|
||
| ## Async operations | ||
|
|
||
| For production applications requiring high throughput, use native async operations: | ||
|
|
||
| ```python | ||
| import asyncio | ||
|
|
||
| async def main(): | ||
| messages = [ | ||
| ("system", "You are a helpful assistant."), | ||
| ("human", "What is the capital of France?"), | ||
| ] | ||
| response = await model.ainvoke(messages) | ||
| print(response.content) | ||
|
|
||
| asyncio.run(main()) | ||
| ``` | ||
|
|
||
| ```output | ||
| The capital of France is Paris. | ||
| ``` | ||
|
|
||
| ## Chaining | ||
|
|
||
| Amazon Nova models work seamlessly with LangChain's LCEL (LangChain Expression Language) for building chains: | ||
|
|
||
| ```python | ||
| from langchain_core.output_parsers import StrOutputParser | ||
| from langchain_core.prompts import ChatPromptTemplate | ||
|
|
||
| prompt = ChatPromptTemplate.from_messages([ | ||
| ("system", "You are a helpful assistant that translates {input_language} to {output_language}."), | ||
| ("human", "{text}"), | ||
| ]) | ||
|
|
||
| chain = prompt | model | StrOutputParser() | ||
|
|
||
| result = chain.invoke({ | ||
| "input_language": "English", | ||
| "output_language": "Spanish", | ||
| "text": "Hello, how are you?" | ||
| }) | ||
| print(result) | ||
| ``` | ||
|
|
||
| ```output | ||
| Hola, ¿cómo estás? | ||
| ``` | ||
|
|
||
| ## Error handling | ||
|
|
||
| The model includes built-in retry logic with configurable parameters: | ||
|
|
||
| ```python | ||
| model = ChatAmazonNova( | ||
| model="nova-2-lite-v1", | ||
| max_retries=3, # Number of retries on failure | ||
| timeout=60.0, # Request timeout in seconds | ||
| ) | ||
| ``` | ||
|
|
||
| For additional control over retries, use the `with_retry` method: | ||
|
|
||
| ```python | ||
| model_with_custom_retry = model.with_retry( | ||
| stop_after_attempt=5, | ||
| wait_exponential_jitter=True, | ||
| ) | ||
| ``` | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Connection issues | ||
|
|
||
| If you encounter connection errors, verify your environment variables are set correctly: | ||
|
|
||
| ```python | ||
| import os | ||
| print(f"API Key set: {'NOVA_API_KEY' in os.environ}") | ||
| print(f"Base URL: {os.environ.get('NOVA_BASE_URL', 'Not set')}") | ||
| ``` | ||
|
|
||
| For authentication and connection issues, refer to the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation). | ||
|
|
||
| ### Compression errors | ||
|
|
||
| <Note> | ||
| The @[`ChatAmazonNova`] client automatically disables compression to avoid potential decompression issues. | ||
| </Note> | ||
|
|
||
| If you need to customize HTTP client behavior, you can access the underlying OpenAI client: | ||
|
|
||
| ```python | ||
| # The client is automatically configured with no compression | ||
| model = ChatAmazonNova(model="nova-2-lite-v1") | ||
| # model.client is the configured OpenAI client | ||
| ``` | ||
|
|
||
| ### Tool calling validation errors | ||
|
|
||
| If you receive a validation error when binding tools, ensure the model supports tool calling. | ||
|
|
||
| --- | ||
|
|
||
| ## API reference | ||
|
|
||
| For detailed documentation of all @[`ChatAmazonNova`] features and configurations, head to the @[`ChatAmazonNova`] API reference. | ||
|
|
||
| For Amazon Nova-specific features, model details, and API specifications, see the [Amazon Nova documentation](https://nova.amazon.com/dev/documentation). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,6 +20,7 @@ mode: wide | |
| | [`ChatGoogleGenerativeAI`](/oss/integrations/chat/google_generative_ai) | ✅ | ✅ | ✅ | ❌ | ✅ | | ||
| | [`ChatGroq`](/oss/integrations/chat/groq) | ✅ | ✅ | ✅ | ❌ | ❌ | | ||
| | [`ChatBedrock`](/oss/integrations/chat/bedrock) | ✅ | ✅ | ❌ | ❌ | ❌ | | ||
| | [`ChatAmazonNova`](/oss/integrations/chat/amazon_nova) | ✅ | ❌ | ❌ | ❌ | ✅ | | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. structured output may be supported already with the base implementation if you support the |
||
| | [`ChatHuggingFace`](/oss/integrations/chat/huggingface) | ✅ | ✅ | ❌ | ✅ | ❌ | | ||
| | [`ChatOllama`](/oss/integrations/chat/ollama) | ✅ | ✅ | ✅ | ✅ | ❌ | | ||
| | [`ChatWatsonx`](/oss/integrations/chat/ibm_watsonx) | ✅ | ✅ | ✅ | ❌ | ✅ | | ||
|
|
@@ -106,6 +107,14 @@ Certain model providers offer endpoints that are compatible with OpenAI's (legac | |
| cta="View guide" | ||
| /> | ||
|
|
||
| <Card | ||
| title="Amazon Nova" | ||
| icon="link" | ||
| href="/oss/integrations/chat/amazon_nova" | ||
| arrow="true" | ||
| cta="View guide" | ||
| /> | ||
|
|
||
| <Card | ||
| title="Anthropic" | ||
| icon="link" | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I see in the implementation,
strictis propagated as a kwarg, so it ends up in invocation params. Is it true that it validates model support for tool calling? Ifstrictis False and the model does not support tool calling, do we get a different error than we otherwise would?Typically
strictinbind_toolsengages schema adherence for tool calls (see docs here).