Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pipeline/preprocessors/link_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -437,6 +437,7 @@ class LinkMap(TypedDict):
"toolRetryMiddleware": "functions/langchain.index.toolRetryMiddleware.html",
"modelRetryMiddleware": "functions/langchain.index.modelRetryMiddleware.html",
"systemPrompt": "types/langchain.index.CreateAgentParams.html#systemprompt",
"openAIModerationMiddleware": "classes/_langchain_openai.middleware.OpenAIModerationMiddleware.html",
},
},
]
Expand Down
182 changes: 181 additions & 1 deletion src/oss/langchain/middleware/built-in.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,18 @@ const agent = createAgent({
For complete examples, configuration options, and integration patterns, see the [Human-in-the-loop documentation](/oss/langchain/human-in-the-loop).
</Tip>

:::python
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=SpfT6-YAVPk) demonstrating Human-in-the-loop middleware behavior.
</Callout>
:::

:::js
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=tdOeUVERukA) demonstrating Human-in-the-loop middleware behavior.
</Callout>
:::

### Model call limit

Limit the number of model calls to prevent infinite loops or excessive costs. Model call limit is useful for the following:
Expand Down Expand Up @@ -454,6 +466,18 @@ const agent = createAgent({
```
:::

:::python
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=nJEER0uaNkE) demonstrating Model Call Limit middleware behavior.
</Callout>
:::

:::js
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=x5jLQTFXR0Y) demonstrating Model Call Limit middleware behavior.
</Callout>
:::

<Accordion title="Configuration options">

:::python
Expand Down Expand Up @@ -539,6 +563,18 @@ const agent = createAgent({
```
:::

:::python
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=6gYlaJJ8t0w) demonstrating Tool Call Limit middleware behavior.
</Callout>
:::

:::js
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=oL6am5UqODY) demonstrating Tool Call Limit middleware behavior.
</Callout>
:::

<Accordion title="Configuration options">

:::python
Expand Down Expand Up @@ -686,6 +722,12 @@ const agent = createAgent({
```
:::

:::python
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=8rCRO0DUeIM) demonstrating Model Fallback middleware behavior.
</Callout>
:::

<Accordion title="Configuration options">

:::python
Expand Down Expand Up @@ -1051,6 +1093,18 @@ const agent = createAgent({
```
:::

:::python
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=yTWocbVKQxw) demonstrating To-do List middleware behavior.
</Callout>
:::

:::js
<Callout icon="circle-play" iconType="solid">
Watch this [video guide](https://www.youtube.com/watch?v=dwvhZ1z_Pas) demonstrating To-do List middleware behavior.
</Callout>
:::

<Accordion title="Configuration options">

:::python
Expand Down Expand Up @@ -2876,6 +2930,28 @@ agent = create_agent(
)
```
:::
:::js
**API reference:** @[`openAIModerationMiddleware`]
Comment on lines +2933 to +2934
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't hurt to call out that:

  1. this only works with openai models
  2. this won't work with model strings

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. this only works with openai models

The section starts with Middleware specifically designed for OpenAI models.

2. this won't work with model strings

It does. The middleware model option can be a string too.


```typescript
import { createAgent, openAIModerationMiddleware } from "langchain";
import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({ model: "gpt-4o" });

const agent = createAgent({
model,
tools: [search_tool, customer_data_tool],
middleware: [openAIModerationMiddleware({
model,
moderationModel: "omni-moderation-latest",
checkInput: true,
checkOutput: true,
exitBehavior: "end",
})],
});
```
:::

<Accordion title="Configuration options">

Expand Down Expand Up @@ -2923,12 +2999,52 @@ agent = create_agent(
</ParamField>
:::

:::js
<ParamField body="model" type="ChatOpenAI">
OpenAI model to use.
</ParamField>

<ParamField body="moderationModel" type="string">
OpenAI moderation model to use. Options: `'omni-moderation-latest'`, `'omni-moderation-2024-09-26'`, `'text-moderation-latest'`, `'text-moderation-stable'`
</ParamField>

<ParamField body="checkInput" type="boolean" default="true">
Whether to check user input messages before the model is called
</ParamField>

<ParamField body="checkOutput" type="boolean" default="true">
Whether to check model output messages after the model is called
</ParamField>

<ParamField body="checkToolResults" type="boolean" default="false">
Whether to check tool result messages before the model is called
</ParamField>

<ParamField body="exitBehavior" type="string" default="end">
How to handle violations when content is flagged. Options:

- `'end'` - End agent execution immediately with a violation message
- `'error'` - Raise `OpenAIModerationError` exception
- `'replace'` - Replace the flagged content with the violation message and continue
</ParamField>

<ParamField body="violationMessage" type="string | null">
Custom template for violation messages. Supports template variables:

- `{categories}` - Comma-separated list of flagged categories
- `{category_scores}` - JSON string of category scores
- `{original_content}` - The original flagged content

Default: `"I'm sorry, but I can't comply with that request. It was flagged for {categories}."`
</ParamField>
:::
</Accordion>

<Accordion title="Full example">

The middleware integrates OpenAI's moderation endpoint to check content at different stages:

:::python
**Moderation stages:**
- `check_input` - User messages before model call
- `check_output` - AI messages after model call
Expand All @@ -2939,7 +3055,6 @@ The middleware integrates OpenAI's moderation endpoint to check content at diffe
- `'error'` - Raise exception for application handling
- `'replace'` - Replace flagged content and continue

:::python
```python
from langchain_openai import ChatOpenAI
from langchain_openai.middleware import OpenAIModerationMiddleware
Expand Down Expand Up @@ -2992,5 +3107,70 @@ agent_replace = create_agent(
)
```
:::
:::js
**Moderation stages:**
- `checkInput` - User messages before model call
- `checkOutput` - AI messages after model call
- `checkToolResults` - Tool outputs before model call

**Exit behaviors:**
- `'end'` (default) - Stop execution with violation message
- `'error'` - Raise exception for application handling
- `'replace'` - Replace flagged content and continue

```typescript
import { createAgent, openAIModerationMiddleware } from "langchain";
import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({ model: "gpt-4o" });

// Basic moderation
const agent = createAgent({
model,
tools: [searchTool, customerDataTool],
middleware: [
openAIModerationMiddleware({
model,
moderationModel: "omni-moderation-latest",
checkInput: true,
checkOutput: true,
}),
],
});

// Strict moderation with custom message
const agentStrict = createAgent({
model,
tools: [searchTool, customerDataTool],
middleware: [
openAIModerationMiddleware({
model,
moderationModel: "omni-moderation-latest",
checkInput: true,
checkOutput: true,
checkToolResults: true,
exitBehavior: "error",
violationMessage:
"Content policy violation detected: {categories}. " +
"Please rephrase your request.",
}),
],
});

// Moderation with replacement behavior
const agentReplace = createAgent({
model,
tools: [searchTool],
middleware: [
openAIModerationMiddleware({
model,
checkInput: true,
exitBehavior: "replace",
violationMessage: "[Content removed due to safety policies]",
}),
],
});
```

:::
</Accordion>
Loading