diff --git a/docs/docs/get_started/quickstart.mdx b/docs/docs/get_started/quickstart.mdx index 1367a6cf6fff0e..509b1fe6b440b3 100644 --- a/docs/docs/get_started/quickstart.mdx +++ b/docs/docs/get_started/quickstart.mdx @@ -11,6 +11,13 @@ In this quickstart we'll show you how to: That's a fair amount to cover! Let's dive in. ## Setup + +### Jupyter Notebook + +This guide (and most of the other guides in the documentation) use [Jupyter notebooks](https://jupyter.org/) and assume the reader is as well. Jupyter notebooks are perfect for learning how to work with LLM systems because often times things can go wrong (unexpected output, API down, etc) and going through guides in an interactive environment is a great way to better understand them. + +You do not NEED to go through the guide in a Jupyter Notebook, but it is recommended. See [here](https://jupyter.org/install) for instructions on how to install. + ### Installation To install LangChain run: @@ -31,22 +38,58 @@ import CodeBlock from "@theme/CodeBlock"; For more details, see our [Installation guide](/docs/get_started/installation). -### Environment +### LangSmith + +Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. +As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. +The best way to do this is with [LangSmith](https://smith.langchain.com). + +Note that LangSmith is not needed, but it is helpful. +If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces: + +```shell +export LANGCHAIN_TRACING_V2="true" +export LANGCHAIN_API_KEY="..." +``` + +## Building with LangChain + +LangChain enables building application that connect external sources of data and computation to LLMs. +In this quickstart, we will walk through a few different ways of doing that. +We will start with a simple LLM chain, which just relies on information in the prompt template to respond. +Next, we will build a retrieval chain, which fetches data from a separate database and passes that into the prompt template. +We will then add in chat history, to create a conversation retrieval chain. This allows you interact in a chat manner with this LLM, so it remembers previous questions. +Finally, we will build an agent - which utilizes and LLM to determine whether or not it needs to fetch data to answer questions. +We will cover these at a high level, but there are lot of details to all of these! +We will link to relevant docs. -Using LangChain will usually require integrations with one or more model providers, data stores, APIs, etc. For this example, we'll use OpenAI's model APIs. +## LLM Chain + +For this getting started guide, we will provide two options: using OpenAI (a popular model available via API) or using a local open source model. + + + First we'll need to install their Python package: -```bash +```shell pip install openai ``` Accessing the API requires an API key, which you can get by creating an account and heading [here](https://platform.openai.com/account/api-keys). Once we have a key we'll want to set it as an environment variable by running: -```bash +```shell export OPENAI_API_KEY="..." ``` +We can then initialize the model: + +```python +from langchain.chat_models import ChatOpenAI + +llm = ChatOpenAI() +``` + If you'd prefer not to set an environment variable you can pass the key in directly via the `openai_api_key` named parameter when initiating the OpenAI LLM class: ```python @@ -55,257 +98,367 @@ from langchain.chat_models import ChatOpenAI llm = ChatOpenAI(openai_api_key="...") ``` -### LangSmith + + -Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. -As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. -The best way to do this is with [LangSmith](https://smith.langchain.com). +[Ollama](https://ollama.ai/) allows you to run open-source large language models, such as Llama 2, locally. -Note that LangSmith is not needed, but it is helpful. -If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces: +First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance: -```shell -export LANGCHAIN_TRACING_V2="true" -export LANGCHAIN_API_KEY="..." +* [Download](https://ollama.ai/download) +* Fetch a model via `ollama pull llama2` + +Then, make sure the Ollama server is running. After that, you can do: +```python +from langchain.llms import Ollama +llm = Ollama(model="llama2") ``` -### LangServe + + -LangServe helps developers deploy LangChain chains as a REST API. You do not need to use LangServe to use LangChain, but in this guide we'll show how you can deploy your app with LangServe. +Once you've installed and initialized the LLM of your choice, we can try using it! +Let's ask it what LangSmith is - this is something that wasn't present in the training data so it shouldn't have a very good response. -Install with: -```bash -pip install "langserve[all]" +```python +llm.invoke("how can langsmith help with testing?") ``` -## Building with LangChain +We can also guide it's response with a prompt template. +Prompt templates are used to convert raw user input to a better input to the LLM. + +```python +from langchain.prompts import ChatPromptTemplate +prompt = ChatPromptTemplate.from_messages([ + ("system", "You are world class technical documentation writer."), + ("user", "{input}") +]) +``` -LangChain provides many modules that can be used to build language model applications. -Modules can be used as standalones in simple applications and they can be composed for more complex use cases. -Composition is powered by **LangChain Expression Language** (LCEL), which defines a unified `Runnable` interface that many modules implement, making it possible to seamlessly chain components. +We can now combine these into a simple LLM chain: -The simplest and most common chain contains three things: -- LLM/Chat Model: The language model is the core reasoning engine here. In order to work with LangChain, you need to understand the different types of language models and how to work with them. -- Prompt Template: This provides instructions to the language model. This controls what the language model outputs, so understanding how to construct prompts and different prompting strategies is crucial. -- Output Parser: These translate the raw response from the language model to a more workable format, making it easy to use the output downstream. +```python +chain = prompt | llm +``` + +We can now invoke it and ask the same question. It still won't know the answer, but it should respond in a more proper tone for a technical writer! -In this guide we'll cover those three components individually, and then go over how to combine them. -Understanding these concepts will set you up well for being able to use and customize LangChain applications. -Most LangChain applications allow you to configure the model and/or the prompt, so knowing how to take advantage of this will be a big enabler. +The output of a ChatModel (and therefore, of this chain) is a message. However, it's often much more convenient to work with strings. Let's add a simple output parser to convert the chat message to a string. + +```python +from langchain_core.output_parsers import StrOutputParser -### LLM / Chat Model +output_parser = StrOutputParser() +``` -There are two types of language models: +We can now add this to the previous chain: -- `LLM`: underlying model takes a string as input and returns a string -- `ChatModel`: underlying model takes a list of messages as input and returns a message +```python +chain = prompt | llm | output_parser +``` -Strings are simple, but what exactly are messages? The base message interface is defined by `BaseMessage`, which has two required attributes: +We can now invoke it and ask the same question. The answer will now be a string (rather than a ChatMessage). -- `content`: The content of the message. Usually a string. -- `role`: The entity from which the `BaseMessage` is coming. +```python +chain.invoke({"input": "how can langsmith help with testing?"}) +``` -LangChain provides several objects to easily distinguish between different roles: +### Diving Deeper -- `HumanMessage`: A `BaseMessage` coming from a human/user. -- `AIMessage`: A `BaseMessage` coming from an AI/assistant. -- `SystemMessage`: A `BaseMessage` coming from the system. -- `FunctionMessage` / `ToolMessage`: A `BaseMessage` containing the output of a function or tool call. +We've now successfully set up a basic LLM chain. We only touched on the basics of prompts, models, and output parsers - for a deeper dive into everything mentioned here, see [this section of documentation](/docs/modules/model_io). -If none of those roles sound right, there is also a `ChatMessage` class where you can specify the role manually. -LangChain provides a common interface that's shared by both `LLM`s and `ChatModel`s. -However it's useful to understand the difference in order to most effectively construct prompts for a given language model. +## Retrieval Chain -The simplest way to call an `LLM` or `ChatModel` is using `.invoke()`, the universal synchronous call method for all LangChain Expression Language (LCEL) objects: -- `LLM.invoke`: Takes in a string, returns a string. -- `ChatModel.invoke`: Takes in a list of `BaseMessage`, returns a `BaseMessage`. +In order to properly answer the original question ("how can langsmith help with testing?"), we need to provide additional context to the LLM. +We can do this via *retrieval*. +Retrieval is useful when you have **too much data** to pass to the LLM directly. +You can then use a retriever to fetch only the most relevant pieces and pass those in. -The input types for these methods are actually more general than this, but for simplicity here we can assume LLMs only take strings and Chat models only takes lists of messages. -Check out the "Go deeper" section below to learn more about model invocation. +In this process, we will look up relevant documents from a *Retriever* and then pass them into the prompt. +A Retriever can be backed by anything - a SQL table, the internet, etc - but in this instance we will populate a vector store and use that as a retriever. For more information on vectorstores, see [this documentation](/docs/modules/data_connection/vectorstores). + +First, we need to load the data that we want to index: -Let's see how to work with these different types of models and these different types of inputs. -First, let's import an LLM and a ChatModel. ```python -from langchain.llms import OpenAI -from langchain.chat_models import ChatOpenAI +from langchain_community.document_loaders import WebBaseLoader +loader = WebBaseLoader("https://docs.smith.langchain.com/overview") + +docs = loader.load() +``` + +Next, we need to index it into a vectorstore. This requires a few components, namely an [embedding model](/docs/modules/data_connection/text_embedding) and a [vectorstore](/docs/modules/data_connection/vectorstores). + +For embedding models, we once again provide examples for accessing via OpenAI or via local models. + + + + +Make sure you have the openai package installed an the appropriate environment variables set (these are the same as needed for the LLM). + +```python +from langchain_community.embeddings import OpenAIEmbeddings -llm = OpenAI() -chat_model = ChatOpenAI() +embeddings = OpenAIEmbeddings() ``` -`LLM` and `ChatModel` objects are effectively configuration objects. -You can initialize them with parameters like `temperature` and others, and pass them around. + + + +Make sure you have Ollama running (same set up as with the LLM). ```python -from langchain.schema import HumanMessage +from langchain_community.embeddings import OllamaEmbeddings + +embeddings = OllamaEmbeddings() +``` + + + +Now, we can use this embedding model to ingest documents into a vectorstore. +We will use a simple local vectorstore, [DocArray InMemorySearch](/docs/integrations/vectorstores/docarray_in_memory), for simplicity's sake. + +First we need to install the required packages for that: + +```shell +pip install docarray +``` + +Then we can build our index: -text = "What would be a good company name for a company that makes colorful socks?" -messages = [HumanMessage(content=text)] +```python +from langchain_community.vectorstores import DocArrayInMemorySearch +from langchain.text_splitter import RecursiveCharacterTextSplitter -llm.invoke(text) -# >> Feetful of Fun -chat_model.invoke(messages) -# >> AIMessage(content="Socks O'Color") +text_splitter = RecursiveCharacterTextSplitter() +documents = text_splitter.split_documents(docs) +vector = DocArrayInMemorySearch.from_documents(documents, embeddings) ``` -
Go deeper +Now that we have this data indexed in a vectorstore, we will create a retrieval chain. +This chain will take an incoming question, look up relevant documents, then pass those documents along with the original question into an LLM and ask it to answer the original question. -`LLM.invoke` and `ChatModel.invoke` actually both support as input any of `Union[str, List[BaseMessage], PromptValue]`. -`PromptValue` is an object that defines its own custom logic for returning its inputs either as a string or as messages. -`LLM`s have logic for coercing any of these into a string, and `ChatModel`s have logic for coercing any of these to messages. -The fact that `LLM` and `ChatModel` accept the same inputs means that you can directly swap them for one another in most chains without breaking anything, -though it's of course important to think about how inputs are being coerced and how that may affect model performance. -To dive deeper on models head to the [Language models](/docs/modules/model_io/models) section. +First, let's set up the chain that takes a question and the retrieved documents and generates an answer. + +```python +from langchain.chains.combine_documents import create_stuff_documents_chain -
+prompt = ChatPromptTemplate.from_template("""Answer the following question based only on the provided context: -### Prompt templates + +{context} + -Most LLM applications do not pass user input directly into an LLM. Usually they will add the user input to a larger piece of text, called a prompt template, that provides additional context on the specific task at hand. +Question: {input}""") -In the previous example, the text we passed to the model contained instructions to generate a company name. For our application, it would be great if the user only had to provide the description of a company/product without worrying about giving the model instructions. +document_chain = create_stuff_documents_chain(llm, prompt) +``` -PromptTemplates help with exactly this! -They bundle up all the logic for going from user input into a fully formatted prompt. -This can start off very simple - for example, a prompt to produce the above string would just be: +If we wanted to, we could run this ourselves by passing in documents directly: ```python -from langchain.prompts import PromptTemplate +from langchain_core.documents import Document -prompt = PromptTemplate.from_template("What is a good name for a company that makes {product}?") -prompt.format(product="colorful socks") +document_chain.invoke({ + "input": "how can langsmith help with testing?", + "context": [Document(page_content="langsmith can let you visualize test results")] +}) ``` +However, we want the documents to first come from the retriever we just set up. +That way, for a given question we can use the retriever to dynamically select the most relevant documents and pass those in. + ```python -What is a good name for a company that makes colorful socks? +from langchain.chains import create_retrieval_chain + +retriever = vector.as_retriever() +retrieval_chain = create_retrieval_chain(retriever, document_chain) ``` -However, the advantages of using these over raw string formatting are several. -You can "partial" out variables - e.g. you can format only some of the variables at a time. -You can compose them together, easily combining different templates into a single prompt. -For explanations of these functionalities, see the [section on prompts](/docs/modules/model_io/prompts) for more detail. +We can now invoke this chain. This returns a dictionary - the response from the LLM is in the `answer` key -`PromptTemplate`s can also be used to produce a list of messages. -In this case, the prompt not only contains information about the content, but also each message (its role, its position in the list, etc.). -Here, what happens most often is a `ChatPromptTemplate` is a list of `ChatMessageTemplates`. -Each `ChatMessageTemplate` contains instructions for how to format that `ChatMessage` - its role, and then also its content. -Let's take a look at this below: +```python +response = retrieval_chain.invoke({"input": "how can langsmith help with testing?"}) +print(response["answer"]) + +// LangSmith offers several features that can help with testing:... +``` + +This answer should be much more accurate! + +### Diving Deeper + +We've now successfully set up a basic retrieval chain. We only touched on the basics of retrieval - for a deeper dive into everything mentioned here, see [this section of documentation](/docs/modules/data_connection). + +## Conversation Retrieval Chain + +The chain we've created so far can only answer single questions. One of the main types of LLM applications that people are building are chat bots. So how do we turn this chain into one that can answer follow up questions? + +We can still use the `create_retrieval_chain` function, but we need to change two things: + +1. The retrieval method should now not just work on the most recent input, but rather should take the whole history into account. +2. The final LLM chain should likewise take the whole history into account + +**Updating Retrieval** + +In order to update retrieval, we will create a new chain. This chain will take in the most recent input (`input`) and the conversation history (`chat_history`) and use an LLM to generate a search query. ```python -from langchain.prompts.chat import ChatPromptTemplate +from langchain.chains import create_history_aware_retriever +from langchain_core.prompts import MessagesPlaceholder -template = "You are a helpful assistant that translates {input_language} to {output_language}." -human_template = "{text}" +# First we need a prompt that we can pass into an LLM to generate this search query -chat_prompt = ChatPromptTemplate.from_messages([ - ("system", template), - ("human", human_template), +prompt = ChatPromptTemplate.from_messages([ + MessagesPlaceholder(variable_name="chat_history"), + ("user", "{input}") + ("user", "Given the above conversation, generate a search query to look up in order to get information relevant to the conversation") ]) +retriever_chain = create_history_aware_retriever(llm, retriever, prompt) +``` + +We can test this out by passing in an instance where the user is asking a follow up question. -chat_prompt.format_messages(input_language="English", output_language="French", text="I love programming.") +```python +from langchain_core.messages import HumanMessage, AIMessage + +chat_history = [HumanMessage(content="Can LangSmith help test my LLM applications?"), AIMessage(content="Yes!")] +retrieval_chain.invoke({ + "chat_history": chat_history, + "input": "Tell me how" +}) ``` +You should see that this returns documents about testing in LangSmith. This is because the LLM generated a new query, combining the chat history with the follow up question. + +Now that we have this new retriever, we can create a new chain to continue the conversation with these retrieved documents in mind. -```pycon -[ - SystemMessage(content="You are a helpful assistant that translates English to French.", additional_kwargs={}), - HumanMessage(content="I love programming.") -] +```python +prompt = ChatPromptTemplate.from_messages([ + ("system", "Answer the user's questions based on the below context:\n\n{context}"), + MessagesPlaceholder(variable_name="chat_history"), + ("user", "{input}"), +]) +document_chain = create_stuff_documents_chain(llm, prompt) + +retrieval_chain = create_retrieval_chain(retriever_chain, document_chain) ``` +We can now test this out end-to-end: -ChatPromptTemplates can also be constructed in other ways - see the [section on prompts](/docs/modules/model_io/prompts) for more detail. +```python +chat_history = [HumanMessage(content="Can LangSmith help test my LLM applications?"), AIMessage(content="Yes!")] +retrieval_chain.invoke({ + "chat_history": chat_history, + "input": "Tell me how" +}) +``` +We can see that this gives a coherent answer - we've successfully turned our retrieval chain into a chatbot! + +## Agent -### Output parsers +We've so far create examples of chains - where each step is known ahead of time. +The final thing we will create is an agent - where the LLM decides what steps to take. -`OutputParser`s convert the raw output of a language model into a format that can be used downstream. -There are a few main types of `OutputParser`s, including: +**NOTE: for this example we will only show how to create an agent using OpenAI models, as local models are not reliable enough yet.** -- Convert text from `LLM` into structured information (e.g. JSON) -- Convert a `ChatMessage` into just a string -- Convert the extra information returned from a call besides the message (like OpenAI function invocation) into a string. +One of the first things to do when building an agent is to decide what tools it should have access to. +For this example, we will give the agent access two tools: -For full information on this, see the [section on output parsers](/docs/modules/model_io/output_parsers). +1. The retriever we just created. This will let it easily answer questions about LangSmith +2. A search tool. This will let it easily answer questions that require up to date information. -In this getting started guide, we will write our own output parser - one that converts a comma separated list into a list. +First, let's set up a tool for the retriever we just created: ```python -from langchain.schema import BaseOutputParser +from langchain.tools.retriever import create_retriever_tool -class CommaSeparatedListOutputParser(BaseOutputParser): - """Parse the output of an LLM call to a comma-separated list.""" +retriever_tool = create_retriever_tool( + retriever, + "langsmith_search", + "Search for information about LangSmith. For any questions about LangSmith, you must use this tool!", +) +``` - def parse(self, text: str): - """Parse the output of an LLM call.""" - return text.strip().split(", ") +The search tool that we will use is [Tavily](/docs/integrations/retrievers/tavily). This will require an API key (they have generous free tier). After creating it on their platform, you need to set it as an environment variable: -CommaSeparatedListOutputParser().parse("hi, bye") -# >> ['hi', 'bye'] +```shell +export TAVILY_API_KEY=... ``` +If you do not want to set up an API key, you can skip creating this tool. -### Composing with LCEL +```python +from langchain_community.tools.tavily_search import TavilySearchResults + +search = TavilySearchResults() +``` -We can now combine all these into one chain. -This chain will take input variables, pass those to a prompt template to create a prompt, pass the prompt to a language model, and then pass the output through an (optional) output parser. -This is a convenient way to bundle up a modular piece of logic. -Let's see it in action! +We can now create a list of the tools we want to work with: ```python -from typing import List +tools = [retriever_tool, search] +``` -from langchain.chat_models import ChatOpenAI -from langchain.prompts import ChatPromptTemplate -from langchain.schema import BaseOutputParser +Now that we have the tools, we can create an agent to use them. We will go over this pretty quickly - for a deeper dive into what exactly is going on, check out the [Agent's Getting Started documentation](/docs/modules/agents) -class CommaSeparatedListOutputParser(BaseOutputParser[List[str]]): - """Parse the output of an LLM call to a comma-separated list.""" +```python +from langchain.chat_models import ChatOpenAI +from langchain import hub +from langchain.agents import create_openai_functions_agent +from langchain.agents import AgentExecutor + +# Get the prompt to use - you can modify this! +prompt = hub.pull("hwchase17/openai-functions-agent") +llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) +agent = create_openai_functions_agent(llm, tools, prompt) +agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True) +``` +We can now invoke the agent and see how it responds! We can ask it questions about LangSmith: - def parse(self, text: str) -> List[str]: - """Parse the output of an LLM call.""" - return text.strip().split(", ") +```python +agent_executor.invoke({"input": "how can langsmith help with testing?"}) +``` -template = """You are a helpful assistant who generates comma separated lists. -A user will pass in a category, and you should generate 5 objects in that category in a comma separated list. -ONLY return a comma separated list, and nothing more.""" -human_template = "{text}" +We can ask it about the weather: -chat_prompt = ChatPromptTemplate.from_messages([ - ("system", template), - ("human", human_template), -]) -chain = chat_prompt | ChatOpenAI() | CommaSeparatedListOutputParser() -chain.invoke({"text": "colors"}) -# >> ['red', 'blue', 'green', 'yellow', 'orange'] +```python +agent_executor.invoke({"input": "what is the weather in SF?"}) ``` -Note that we are using the `|` syntax to join these components together. -This `|` syntax is powered by the LangChain Expression Language (LCEL) and relies on the universal `Runnable` interface that all of these objects implement. -To learn more about LCEL, read the documentation [here](/docs/expression_language). +We can have conversations with it: -## Tracing with LangSmith +```python +chat_history = [HumanMessage(content="Can LangSmith help test my LLM applications?"), AIMessage(content="Yes!")] +agent_executor.invoke({ + "chat_history": chat_history, + "input": "Tell me how" +}) +``` -Assuming we've set our environment variables as shown in the beginning, all of the model and chain calls we've been making will have been automatically logged to LangSmith. -Once there, we can use LangSmith to debug and annotate our application traces, then turn them into datasets for evaluating future iterations of the application. +### Diving Deeper -Check out what the trace for the above chain would look like: -https://smith.langchain.com/public/09370280-4330-4eb4-a7e8-c91817f6aa13/r +We've now successfully set up a basic agent. We only touched on the basics of agents - for a deeper dive into everything mentioned here, see [this section of documentation](/docs/modules/agents). -For more on LangSmith [head here](/docs/langsmith/). ## Serving with LangServe Now that we've built an application, we need to serve it. That's where LangServe comes in. -LangServe helps developers deploy LCEL chains as a REST API. -The library is integrated with FastAPI and uses pydantic for data validation. +LangServe helps developers deploy LangChain chains as a REST API. You do not need to use LangServe to use LangChain, but in this guide we'll show how you can deploy your app with LangServe. + +While the first part of this guide was intended to be run in a Jupyter Notebook, we will now move out of that. We will be creating a Python file and then interacting with it from the command line. + +Install with: +```bash +pip install "langserve[all]" +``` ### Server -To create a server for our application we'll make a `serve.py` file with three things: -1. The definition of our chain (same as above) +To create a server for our application we'll make a `serve.py` file. This will contain our logic for serving our application. It consists of three things: +1. The definition of our chain that we just built above 2. Our FastAPI app 3. A definition of a route from which to serve the chain, which is done with `langserve.add_routes` @@ -316,42 +469,73 @@ from typing import List from fastapi import FastAPI from langchain.prompts import ChatPromptTemplate from langchain.chat_models import ChatOpenAI -from langchain.schema import BaseOutputParser +from langchain_community.document_loaders import WebBaseLoader +from langchain_community.embeddings import OpenAIEmbeddings +from langchain_community.vectorstores import DocArrayInMemorySearch +from langchain.text_splitter import RecursiveCharacterTextSplitter +from langchain.tools.retriever import create_retriever_tool +from langchain_community.tools.tavily_search import TavilySearchResults +from langchain.chat_models import ChatOpenAI +from langchain import hub +from langchain.agents import create_openai_functions_agent +from langchain.agents import AgentExecutor +from langchain.pydantic_v1 import BaseModel, Field +from langchain_core.messages import BaseMessage from langserve import add_routes -# 1. Chain definition - -class CommaSeparatedListOutputParser(BaseOutputParser[List[str]]): - """Parse the output of an LLM call to a comma-separated list.""" +# 1. Load Retriever +loader = WebBaseLoader("https://docs.smith.langchain.com/overview") +docs = loader.load() +text_splitter = RecursiveCharacterTextSplitter() +documents = text_splitter.split_documents(docs) +embeddings = OpenAIEmbeddings() +vector = DocArrayInMemorySearch.from_documents(documents, embeddings) +retriever = vector.as_retriever() + +# 2. Create Tools +retriever_tool = create_retriever_tool( + retriever, + "langsmith_search", + "Search for information about LangSmith. For any questions about LangSmith, you must use this tool!", +) +search = TavilySearchResults() +tools = [retriever_tool, search] - def parse(self, text: str) -> List[str]: - """Parse the output of an LLM call.""" - return text.strip().split(", ") +# 3. Create Agent +prompt = hub.pull("hwchase17/openai-functions-agent") +llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) +agent = create_openai_functions_agent(llm, tools, prompt) +agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True) -template = """You are a helpful assistant who generates comma separated lists. -A user will pass in a category, and you should generate 5 objects in that category in a comma separated list. -ONLY return a comma separated list, and nothing more.""" -human_template = "{text}" -chat_prompt = ChatPromptTemplate.from_messages([ - ("system", template), - ("human", human_template), -]) -category_chain = chat_prompt | ChatOpenAI() | CommaSeparatedListOutputParser() - -# 2. App definition +# 4. App definition app = FastAPI( title="LangChain Server", version="1.0", description="A simple API server using LangChain's Runnable interfaces", ) -# 3. Adding chain route +# 5. Adding chain route + +# We need to add these input/output schemas because the current AgentExecutor +# is lacking in schemas. + +class Input(BaseModel): + input: str + chat_history: List[BaseMessage] = Field( + ..., + extra={"widget": {"type": "chat", "input": "location"}}, + ) + + +class Output(BaseModel): + output: str + add_routes( app, - category_chain, - path="/category_chain", + agent_executor.with_types(input_type=Input, output_type=Output), + path="/agent", ) if __name__ == "__main__": @@ -369,19 +553,18 @@ we should see our chain being served at localhost:8000. ### Playground Every LangServe service comes with a simple built-in UI for configuring and invoking the application with streaming output and visibility into intermediate steps. -Head to http://localhost:8000/category_chain/playground/ to try it out! +Head to http://localhost:8000/agent/playground/ to try it out! Pass in the same question as before - "how can langsmith help with testing?" - and it should respond same as before. ### Client -Now let's set up a client for programmatically interacting with our service. We can easily do this with the `langserve.RemoteRunnable`. +Now let's set up a client for programmatically interacting with our service. We can easily do this with the `[langserve.RemoteRunnable](/docs/langserve#client)`. Using this, we can interact with the served chain as if it were running client-side. ```python from langserve import RemoteRunnable -remote_chain = RemoteRunnable("http://localhost:8000/category_chain/") -remote_chain.invoke({"text": "colors"}) -# >> ['red', 'blue', 'green', 'yellow', 'orange'] +remote_chain = RemoteRunnable("http://localhost:8000/agent/") +remote_chain.invoke({"input": "how can langsmith help with testing?"}) ``` To learn more about the many other features of LangServe [head here](/docs/langserve). @@ -390,10 +573,12 @@ To learn more about the many other features of LangServe [head here](/docs/langs We've touched on how to build an application with LangChain, how to trace it with LangSmith, and how to serve it with LangServe. There are a lot more features in all three of these than we can cover here. -To continue on your journey: +To continue on your journey, we recommend you read the following (in order): -- Read up on [LangChain Expression Language (LCEL)](/docs/expression_language) to learn how to chain these components together -- [Dive deeper](/docs/modules/model_io) into LLMs, prompts, and output parsers and learn the other [key components](/docs/modules) +- All of these features are backed by [LangChain Expression Language (LCEL)](/docs/expression_language) - a way to chain these components together. Check out that documentation to better understand how to create custom chains. +- [Model IO](/docs/modules/model_io) covers more details of prompts, LLMs, and output parsers. +- [Retrieval](/docs/modules/data_connection) covers more details of everything related to retrieval +- [Agents](/docs/modules/agents) covers details of everything related to agents - Explore common [end-to-end use cases](/docs/use_cases/qa_structured/sql) and [template applications](/docs/templates) - [Read up on LangSmith](/docs/langsmith/), the platform for debugging, testing, monitoring and more - Learn more about serving your applications with [LangServe](/docs/langserve) diff --git a/docs/docs/integrations/llms/ollama.ipynb b/docs/docs/integrations/llms/ollama.ipynb index adbf4eccac8ead..96a7cb7e9904d1 100644 --- a/docs/docs/integrations/llms/ollama.ipynb +++ b/docs/docs/integrations/llms/ollama.ipynb @@ -220,7 +220,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.16" + "version": "3.10.1" } }, "nbformat": 4,