Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support claude ai model #969

Merged
merged 8 commits into from
Jun 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 149 additions & 17 deletions plugins/wasm-go/extensions/ai-proxy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ description: AI 代理插件配置参考

`provider`的配置字段说明如下:


| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------------- | --------------- | -------- | ------ | ------------------------------------------------------------ |
| `type` | string | 必填 | - | AI 服务提供商名称 |
Expand Down Expand Up @@ -106,9 +105,9 @@ MiniMax所对应的 `type` 为 `minimax`。它特有的配置字段如下:

Anthropic Claude 所对应的 `type` 为 `claude`。它特有的配置字段如下:

| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|-----------|--------|-----|-----|-------------------|
| `version` | string | 必填 | - | Claude 服务的 API 版本 |
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|-----------|--------|------|-----|----------------------------------|
| `claudeVersion` | string | 可选 | - | Claude 服务的 API 版本,默认为 2023-06-01 |

#### Ollama

Expand Down Expand Up @@ -533,6 +532,7 @@ provider:
type: claude
apiTokens:
- "YOUR_CLAUDE_API_TOKEN"
version: "2023-06-01"
```

**请求示例**
Expand All @@ -554,22 +554,25 @@ provider:

```json
{
"id": "msg_01K8iLH18FGN7Xd9deurwtoD",
"type": "message",
"role": "assistant",
"model": "claude-3-opus-20240229",
"stop_sequence": null,
"usage": {
"input_tokens": 16,
"output_tokens": 141
},
"content": [
"id": "msg_01Jt3GzyjuzymnxmZERJguLK",
"choices": [
{
"type": "text",
"text": "你好!我是Claude,一个由Anthropic公司开发的人工智能助手。我的任务是尽我所能帮助人类,比如回答问题,提供建议和意见,协助完成任务等。我掌握了很多知识,也具备一定的分析和推理能力,但我不是人类,也没有实体的身体。很高兴认识你!如果有什么需要帮助的地方,欢迎随时告诉我。"
"index": 0,
"message": {
"role": "assistant",
"content": "您好,我是一个由人工智能公司Anthropic开发的聊天助手。我的名字叫Claude,是一个聪明友善、知识渊博的对话系统。很高兴认识您!我可以就各种话题与您聊天,回答问题,提供建议和帮助。我会尽最大努力给您有帮助的回复。希望我们能有个愉快的交流!"
},
"finish_reason": "stop"
}
],
"stop_reason": "end_turn"
"created": 1717385918,
"model": "claude-3-opus-20240229",
"object": "chat.completion",
"usage": {
"prompt_tokens": 16,
"completion_tokens": 126,
"total_tokens": 142
}
}
```
### 使用 OpenAI 协议代理混元服务
Expand Down Expand Up @@ -757,6 +760,8 @@ provider:

## 完整配置示例

### Kubernetes 示例

以下以使用 OpenAI 协议代理 Groq 服务为例,展示完整的插件配置示例。

```yaml
Expand Down Expand Up @@ -828,3 +833,130 @@ curl "http://<YOUR-DOMAIN>/v1/chat/completions" -H "Content-Type: application/js
]
}'
```

### Docker-Compose 示例

`docker-compose.yml` 配置文件:

```yaml
version: '3.7'
services:
envoy:
image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/envoy:1.20
entrypoint: /usr/local/bin/envoy
# 开启了 debug 级别日志方便调试
command: -c /etc/envoy/envoy.yaml --component-log-level wasm:debug
networks:
- higress-net
ports:
- "10000:10000"
volumes:
- ./envoy.yaml:/etc/envoy/envoy.yaml
- ./plugin.wasm:/etc/envoy/plugin.wasm
networks:
higress-net: {}
```

`envoy.yaml` 配置文件:

```yaml
admin:
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
scheme_header_transformation:
scheme_to_overwrite: https
stat_prefix: ingress_http
# Output envoy logs to stdout
access_log:
- name: envoy.access_loggers.stdout
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
# Modify as required
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: [ "*" ]
routes:
- match:
prefix: "/"
route:
cluster: claude
timeout: 300s
http_filters:
- name: claude
typed_config:
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
name: claude
vm_config:
runtime: envoy.wasm.runtime.v8
code:
local:
filename: /etc/envoy/plugin.wasm
configuration:
"@type": "type.googleapis.com/google.protobuf.StringValue"
value: | # 插件配置
{
"provider": {
"type": "claude",
"apiTokens": [
"YOUR_API_TOKEN"
]
}
}
- name: envoy.filters.http.router
clusters:
- name: claude
connect_timeout: 30s
type: LOGICAL_DNS
dns_lookup_family: V4_ONLY
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: claude
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: api.anthropic.com # API 服务地址
port_value: 443
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
"sni": "api.anthropic.com"
```

访问示例:

```bash
curl "http://localhost:10000/v1/chat/completions" -H "Content-Type: application/json" -d '{
"model": "claude-3-opus-20240229",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": "你好,你是谁?"
}
]
}'
```
Loading
Loading