Skip to content

Commit

Permalink
Merge pull request #23 from tudou2/test_code
Browse files Browse the repository at this point in the history
Test code
  • Loading branch information
congxuma authored Apr 26, 2023
2 parents 089e7fd + d8181b3 commit 67aad3c
Show file tree
Hide file tree
Showing 25 changed files with 472 additions and 84 deletions.
6 changes: 6 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
blank_issues_enabled: false
contact_links:
- name: 知识星球
url: https://public.zsxq.com/groups/88885848842852.html
about: 如果你想了解更多项目细节,并与开发者们交流更多关于AI技术的实践,欢迎加入星球

16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,14 @@
最新版本支持的功能如下:

- [x] **多端部署:** 有多种部署方式可选择且功能完备,目前已支持个人微信,微信公众号和企业微信应用等部署方式
- [x] **基础对话:** 私聊及群聊的消息智能回复,支持多轮会话上下文记忆,支持 GPT-3,GPT-3.5,GPT-4模型
- [x] **语音识别:** 可识别语音消息,通过文字或语音回复,支持 azure, baidu, google, openai等多种语音模型
- [x] **图片生成:** 支持图片生成 和 照片修复,可选择 Dell-E, stable diffusion, replicate 模型
- [x] **图片生成:** 支持图片生成 和 图生图(如照片修复),可选择 Dell-E, stable diffusion, replicate模型
- [x] **丰富插件:** 支持个性化插件扩展,已实现多角色切换、文字冒险、敏感词过滤、聊天记录总结等插件
- [X] **Tool工具:** 与操作系统和互联网交互,支持最新信息搜索、数学计算、天气和资讯查询、网页总结,基于 [chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub) 实现

> 目前已支持 个人微信 和 微信公众号,欢迎接入更多应用,参考 [Terminal代码](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/terminal/terminal_channel.py)实现接收和发送消息逻辑即可接入。 同时欢迎增加新的插件,参考 [插件说明文档](https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins)
> 欢迎接入更多应用,参考 [Terminal代码](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/terminal/terminal_channel.py)实现接收和发送消息逻辑即可接入。 同时欢迎增加新的插件,参考 [插件说明文档](https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins)
**一键部署:**

Expand All @@ -24,7 +25,9 @@ Demo made by [Visionn](https://www.wangpc.cc/)

# 更新日志

>**2023.04.05:** 支持微信公众号部署,兼容角色扮演等预设插件,[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatmp/README.md)。(contributed by [@JS00000](https://github.com/JS00000) in [#686](https://github.com/zhayujie/chatgpt-on-wechat/pull/686))
>**2023.04.26:** 支持企业微信应用号部署,兼容插件,并支持语音图片交互,[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatcom/README.md)。(contributed by [@lanvent](https://github.com/lanvent) in [#944](https://github.com/zhayujie/chatgpt-on-wechat/pull/944))
>**2023.04.05:** 支持微信公众号部署,兼容插件,并支持语音图片交互,[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatmp/README.md)。(contributed by [@JS00000](https://github.com/JS00000) in [#686](https://github.com/zhayujie/chatgpt-on-wechat/pull/686))
>**2023.04.05:** 增加能让ChatGPT使用工具的`tool`插件,[使用文档](https://github.com/goldfishh/chatgpt-on-wechat/blob/master/plugins/tool/README.md)。工具相关issue可反馈至[chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub)。(contributed by [@goldfishh](https://github.com/goldfishh) in [#663](https://github.com/zhayujie/chatgpt-on-wechat/pull/663))
Expand Down Expand Up @@ -97,7 +100,7 @@ pip3 install azure-cognitiveservices-speech
cp config-template.json config.json
```

然后在`config.json`中填入配置,以下是对默认配置的说明,可根据需要进行自定义修改:
然后在`config.json`中填入配置,以下是对默认配置的说明,可根据需要进行自定义修改(请去掉注释)

```bash
# config.json文件内容示例
Expand All @@ -115,7 +118,9 @@ pip3 install azure-cognitiveservices-speech
"speech_recognition": false, # 是否开启语音识别
"group_speech_recognition": false, # 是否开启群组语音识别
"use_azure_chatgpt": false, # 是否使用Azure ChatGPT service代替openai ChatGPT service. 当设置为true时需要设置 open_ai_api_base,如 https://xxx.openai.azure.com/
"character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题,并且可以使用多种语言与人交流。", # 人格描述,
"character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题,并且可以使用多种语言与人交流。", # 人格描述
# 订阅消息,公众号和企业微信channel中请填写,当被订阅时会自动回复,可使用特殊占位符。目前支持的占位符有{trigger_prefix},在程序中它会自动替换成bot的触发词。
"subscribe_msg": "感谢您的关注!\n这里是ChatGPT,可以自由对话。\n支持语音对话。\n支持图片输出,画字开头的消息将按要求创作图片。\n支持角色扮演和文字冒险等丰富插件。\n输入{trigger_prefix}#help 查看详细指令。"
}
```
**配置说明:**
Expand Down Expand Up @@ -150,6 +155,7 @@ pip3 install azure-cognitiveservices-speech
+ `clear_memory_commands`: 对话内指令,主动清空前文记忆,字符串数组可自定义指令别名。
+ `hot_reload`: 程序退出后,暂存微信扫码状态,默认关闭。
+ `character_desc` 配置中保存着你对机器人说的一段话,他会记住这段话并作为他的设定,你可以为他定制任何人格 (关于会话上下文的更多内容参考该 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/43))
+ `subscribe_msg`:订阅消息,公众号和企业微信channel中请填写,当被订阅时会自动回复, 可使用特殊占位符。目前支持的占位符有{trigger_prefix},在程序中它会自动替换成bot的触发词。

**所有可选的配置项均在该[文件](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中列出。**

Expand Down
2 changes: 1 addition & 1 deletion app.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def run():
# os.environ['WECHATY_PUPPET_SERVICE_ENDPOINT'] = '127.0.0.1:9001'

channel = channel_factory.create_channel(channel_name)
if channel_name in ["wx", "wxy", "terminal", "wechatmp", "wechatmp_service"]:
if channel_name in ["wx", "wxy", "terminal", "wechatmp", "wechatmp_service", "wechatcom_app"]:
PluginManager().load_plugins()

# startup channel
Expand Down
4 changes: 4 additions & 0 deletions channel/channel_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,8 @@ def create_channel(channel_type):
from channel.wechatmp.wechatmp_channel import WechatMPChannel

return WechatMPChannel(passive_reply=False)
elif channel_type == "wechatcom_app":
from channel.wechatcom.wechatcomapp_channel import WechatComAppChannel

return WechatComAppChannel()
raise RuntimeError
4 changes: 2 additions & 2 deletions channel/wechat/wechat_channel.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
@itchat.msg_register([TEXT, VOICE, PICTURE, NOTE])
def handler_single_msg(msg):
try:
cmsg = WeChatMessage(msg, False)
cmsg = WechatMessage(msg, False)
except NotImplementedError as e:
logger.debug("[WX]single message {} skipped: {}".format(msg["MsgId"], e))
return None
Expand All @@ -40,7 +40,7 @@ def handler_single_msg(msg):
@itchat.msg_register([TEXT, VOICE, PICTURE, NOTE], isGroupChat=True)
def handler_group_msg(msg):
try:
cmsg = WeChatMessage(msg, True)
cmsg = WechatMessage(msg, True)
except NotImplementedError as e:
logger.debug("[WX]group message {} skipped: {}".format(msg["MsgId"], e))
return None
Expand Down
2 changes: 1 addition & 1 deletion channel/wechat/wechat_message.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from lib.itchat.content import *


class WeChatMessage(ChatMessage):
class WechatMessage(ChatMessage):
def __init__(self, itchat_msg, is_group=False):
super().__init__(itchat_msg)
self.msg_id = itchat_msg["MsgId"]
Expand Down
59 changes: 59 additions & 0 deletions channel/wechatcom/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# 企业微信应用号channel

企业微信官方提供了客服、应用等API,本channel使用的是企业微信的应用API的能力。

因为未来可能还会开发客服能力,所以本channel的类型名叫作`wechatcom_app`

`wechatcom_app` channel支持插件系统和图片声音交互等能力,除了无法加入群聊,作为个人使用的私人助理已绰绰有余。

## 开始之前

- 在企业中确认自己拥有在企业内自建应用的权限。
- 如果没有权限或者是个人用户,也可创建未认证的企业。操作方式:登录手机企业微信,选择`创建/加入企业`来创建企业,类型请选择企业,企业名称可随意填写。
未认证的企业有100人的服务人数上限,其他功能与认证企业没有差异。

本channel需安装的依赖与公众号一致,需要安装`wechatpy``web.py`,它们包含在`requirements-optional.txt`中。

## 使用方法

1.查看企业ID

- 扫码登陆[企业微信后台](https://work.weixin.qq.com)
- 选择`我的企业`,点击`企业信息`,记住该`企业ID`

2.创建自建应用

- 选择应用管理, 在自建区选创建应用来创建企业自建应用
- 上传应用logo,填写应用名称等项
- 创建应用后进入应用详情页面,记住`AgentId``Secert`

3.配置应用

- 在详情页如果点击`企业可信IP`的配置(没看到可以不管),填入你服务器的公网IP
- 点击`接收消息`下的启用API接收消息
- `URL`填写格式为`http://url:port/wxcomapp``port`是程序监听的端口,默认是9898
如果是未认证的企业,url可直接使用服务器的IP。如果是认证企业,需要使用备案的域名,可使用二级域名。
- `Token`可随意填写,停留在这个页面
- 在程序根目录`config.json`中增加配置(**去掉注释**),`wechatcomapp_aes_key`是当前页面的`wechatcomapp_aes_key`

```python
"channel_type": "wechatcom_app",
"wechatcom_corp_id": "", # 企业微信公司的corpID
"wechatcomapp_token": "", # 企业微信app的token
"wechatcomapp_port": 9898, # 企业微信app的服务端口, 不需要端口转发
"wechatcomapp_secret": "", # 企业微信app的secret
"wechatcomapp_agent_id": "", # 企业微信app的agent_id
"wechatcomapp_aes_key": "", # 企业微信app的aes_key
```

- 运行程序,在页面中点击保存,保存成功说明验证成功

4.连接个人微信

选择`我的企业`,点击`微信插件`,下面有个邀请关注的二维码。微信扫码后,即可在微信中看到对应企业,在这里你便可以和机器人沟通。

## 测试体验

AIGC开放社区中已经部署了多个可免费使用的Bot,扫描下方的二维码会自动邀请你来体验。

<img width="200" src="../../docs/images/aigcopen.png">
168 changes: 168 additions & 0 deletions channel/wechatcom/wechatcomapp_channel.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# -*- coding=utf-8 -*-
import io
import os
import time

import requests
import web
from wechatpy.enterprise import create_reply, parse_message
from wechatpy.enterprise.crypto import WeChatCrypto
from wechatpy.enterprise.exceptions import InvalidCorpIdException
from wechatpy.exceptions import InvalidSignatureException, WeChatClientException

from bridge.context import Context
from bridge.reply import Reply, ReplyType
from channel.chat_channel import ChatChannel
from channel.wechatcom.wechatcomapp_client import WechatComAppClient
from channel.wechatcom.wechatcomapp_message import WechatComAppMessage
from common.log import logger
from common.singleton import singleton
from common.utils import compress_imgfile, fsize, split_string_by_utf8_length
from config import conf, subscribe_msg
from voice.audio_convert import any_to_amr

MAX_UTF8_LEN = 2048


@singleton
class WechatComAppChannel(ChatChannel):
NOT_SUPPORT_REPLYTYPE = []

def __init__(self):
super().__init__()
self.corp_id = conf().get("wechatcom_corp_id")
self.secret = conf().get("wechatcomapp_secret")
self.agent_id = conf().get("wechatcomapp_agent_id")
self.token = conf().get("wechatcomapp_token")
self.aes_key = conf().get("wechatcomapp_aes_key")
print(self.corp_id, self.secret, self.agent_id, self.token, self.aes_key)
logger.info(
"[wechatcom] init: corp_id: {}, secret: {}, agent_id: {}, token: {}, aes_key: {}".format(self.corp_id, self.secret, self.agent_id, self.token, self.aes_key)
)
self.crypto = WeChatCrypto(self.token, self.aes_key, self.corp_id)
self.client = WechatComAppClient(self.corp_id, self.secret)

def startup(self):
# start message listener
urls = ("/wxcomapp", "channel.wechatcom.wechatcomapp_channel.Query")
app = web.application(urls, globals(), autoreload=False)
port = conf().get("wechatcomapp_port", 9898)
web.httpserver.runsimple(app.wsgifunc(), ("0.0.0.0", port))

def send(self, reply: Reply, context: Context):
receiver = context["receiver"]
if reply.type in [ReplyType.TEXT, ReplyType.ERROR, ReplyType.INFO]:
reply_text = reply.content
texts = split_string_by_utf8_length(reply_text, MAX_UTF8_LEN)
if len(texts) > 1:
logger.info("[wechatcom] text too long, split into {} parts".format(len(texts)))
for i, text in enumerate(texts):
self.client.message.send_text(self.agent_id, receiver, text)
if i != len(texts) - 1:
time.sleep(0.5) # 休眠0.5秒,防止发送过快乱序
logger.info("[wechatcom] Do send text to {}: {}".format(receiver, reply_text))
elif reply.type == ReplyType.VOICE:
try:
file_path = reply.content
amr_file = os.path.splitext(file_path)[0] + ".amr"
any_to_amr(file_path, amr_file)
response = self.client.media.upload("voice", open(amr_file, "rb"))
logger.debug("[wechatcom] upload voice response: {}".format(response))
except WeChatClientException as e:
logger.error("[wechatcom] upload voice failed: {}".format(e))
return
try:
os.remove(file_path)
if amr_file != file_path:
os.remove(amr_file)
except Exception:
pass
self.client.message.send_voice(self.agent_id, receiver, response["media_id"])
logger.info("[wechatcom] sendVoice={}, receiver={}".format(reply.content, receiver))
elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
img_url = reply.content
pic_res = requests.get(img_url, stream=True)
image_storage = io.BytesIO()
for block in pic_res.iter_content(1024):
image_storage.write(block)
if (sz := fsize(image_storage)) >= 10 * 1024 * 1024:
logger.info("[wechatcom] image too large, ready to compress, sz={}".format(sz))
image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
logger.info("[wechatcom] image compressed, sz={}".format(fsize(image_storage)))
image_storage.seek(0)
try:
response = self.client.media.upload("image", image_storage)
logger.debug("[wechatcom] upload image response: {}".format(response))
except WeChatClientException as e:
logger.error("[wechatcom] upload image failed: {}".format(e))
return

self.client.message.send_image(self.agent_id, receiver, response["media_id"])
logger.info("[wechatcom] sendImage url={}, receiver={}".format(img_url, receiver))
elif reply.type == ReplyType.IMAGE: # 从文件读取图片
image_storage = reply.content
if (sz := fsize(image_storage)) >= 10 * 1024 * 1024:
logger.info("[wechatcom] image too large, ready to compress, sz={}".format(sz))
image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
logger.info("[wechatcom] image compressed, sz={}".format(fsize(image_storage)))
image_storage.seek(0)
try:
response = self.client.media.upload("image", image_storage)
logger.debug("[wechatcom] upload image response: {}".format(response))
except WeChatClientException as e:
logger.error("[wechatcom] upload image failed: {}".format(e))
return
self.client.message.send_image(self.agent_id, receiver, response["media_id"])
logger.info("[wechatcom] sendImage, receiver={}".format(receiver))


class Query:
def GET(self):
channel = WechatComAppChannel()
params = web.input()
logger.info("[wechatcom] receive params: {}".format(params))
try:
signature = params.msg_signature
timestamp = params.timestamp
nonce = params.nonce
echostr = params.echostr
echostr = channel.crypto.check_signature(signature, timestamp, nonce, echostr)
except InvalidSignatureException:
raise web.Forbidden()
return echostr

def POST(self):
channel = WechatComAppChannel()
params = web.input()
logger.info("[wechatcom] receive params: {}".format(params))
try:
signature = params.msg_signature
timestamp = params.timestamp
nonce = params.nonce
message = channel.crypto.decrypt_message(web.data(), signature, timestamp, nonce)
except (InvalidSignatureException, InvalidCorpIdException):
raise web.Forbidden()
msg = parse_message(message)
logger.debug("[wechatcom] receive message: {}, msg= {}".format(message, msg))
if msg.type == "event":
if msg.event == "subscribe":
reply_content = subscribe_msg()
if reply_content:
reply = create_reply(reply_content, msg).render()
res = channel.crypto.encrypt_message(reply, nonce, timestamp)
return res
else:
try:
wechatcom_msg = WechatComAppMessage(msg, client=channel.client)
except NotImplementedError as e:
logger.debug("[wechatcom] " + str(e))
return "success"
context = channel._compose_context(
wechatcom_msg.ctype,
wechatcom_msg.content,
isgroup=False,
msg=wechatcom_msg,
)
if context:
channel.produce(context)
return "success"
21 changes: 21 additions & 0 deletions channel/wechatcom/wechatcomapp_client.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import threading
import time

from wechatpy.enterprise import WeChatClient


class WechatComAppClient(WeChatClient):
def __init__(self, corp_id, secret, access_token=None, session=None, timeout=None, auto_retry=True):
super(WechatComAppClient, self).__init__(corp_id, secret, access_token, session, timeout, auto_retry)
self.fetch_access_token_lock = threading.Lock()

def fetch_access_token(self): # 重载父类方法,加锁避免多线程重复获取access_token
with self.fetch_access_token_lock:
access_token = self.session.get(self.access_token_key)
if access_token:
if not self.expires_at:
return access_token
timestamp = time.time()
if self.expires_at - timestamp > 60:
return access_token
return super().fetch_access_token()
Loading

0 comments on commit 67aad3c

Please sign in to comment.