Merge pull request #25 from tudou2/test_code

Test code

README.md

@@ -25,7 +25,7 @@ Demo made by [Visionn](https://www.wangpc.cc/)
 
 # 更新日志
 
->**2023.04.26：** 支持企业微信应用号部署，兼容插件，并支持语音图片交互，[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatcom/README.md)。(contributed by [@lanvent](https://github.com/lanvent) in [#944](https://github.com/zhayujie/chatgpt-on-wechat/pull/944))
+>**2023.04.26：** 支持企业微信应用号部署，兼容插件，并支持语音图片交互，支持Railway部署，[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatcom/README.md)。(contributed by [@lanvent](https://github.com/lanvent) in [#944](https://github.com/zhayujie/chatgpt-on-wechat/pull/944))
 
 >**2023.04.05：** 支持微信公众号部署，兼容插件，并支持语音图片交互，[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatmp/README.md)。(contributed by [@JS00000](https://github.com/JS00000) in [#686](https://github.com/zhayujie/chatgpt-on-wechat/pull/686))
 
@@ -83,15 +83,13 @@ pip3 install -r requirements-optional.txt
 
 参考[#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)
 
-使用`azure`语音功能需安装依赖（列在`requirements-optional.txt`内，但为便于`railway`部署已注释）:
+使用`azure`语音功能需安装依赖，并参考[文档](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstarts/setup-platform?pivots=programming-language-python&tabs=linux%2Cubuntu%2Cdotnet%2Cjre%2Cmaven%2Cnodejs%2Cmac%2Cpypi)的环境要求。
+:
 
 ```bash
 pip3 install azure-cognitiveservices-speech
 ```
 
-> 目前默认发布的镜像和`railway`部署，都基于`apline`，无法安装`azure`的依赖。若有需求请自行基于[`debian`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/docker/Dockerfile.debian.latest)打包。
-参考[文档](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstarts/setup-platform?pivots=programming-language-python&tabs=linux%2Cubuntu%2Cdotnet%2Cjre%2Cmaven%2Cnodejs%2Cmac%2Cpypi)
-
 ## 配置
 
 配置文件的模板在根目录的`config-template.json`中，需复制该模板创建最终生效的 `config.json` 文件：

bot/chatgpt/chat_gpt_bot.py

@@ -4,6 +4,7 @@
 
 import openai
 import openai.error
+import requests
 
 from bot.bot import Bot
 from bot.chatgpt.chat_gpt_session import ChatGPTSession
@@ -155,3 +156,26 @@ def __init__(self):
         openai.api_type = "azure"
         openai.api_version = "2023-03-15-preview"
         self.args["deployment_id"] = conf().get("azure_deployment_id")
+
+    def create_img(self, query, retry_count=0, api_key=None):
+        api_version = "2022-08-03-preview"
+        url = "{}dalle/text-to-image?api-version={}".format(openai.api_base, api_version)
+        api_key = api_key or openai.api_key
+        headers = {"api-key": api_key, "Content-Type": "application/json"}
+        try:
+            body = {"caption": query, "resolution": conf().get("image_create_size", "256x256")}
+            submission = requests.post(url, headers=headers, json=body)
+            operation_location = submission.headers["Operation-Location"]
+            retry_after = submission.headers["Retry-after"]
+            status = ""
+            image_url = ""
+            while status != "Succeeded":
+                logger.info("waiting for image create..., " + status + ",retry after " + retry_after + " seconds")
+                time.sleep(int(retry_after))
+                response = requests.get(operation_location, headers=headers)
+                status = response.json()["status"]
+            image_url = response.json()["result"]["contentUrl"]
+            return True, image_url
+        except Exception as e:
+            logger.error("create image error: {}".format(e))
+            return False, "图片生成失败"

bot/openai/open_ai_image.py

@@ -15,12 +15,13 @@ def __init__(self):
         if conf().get("rate_limit_dalle"):
             self.tb4dalle = TokenBucket(conf().get("rate_limit_dalle", 50))
 
-    def create_img(self, query, retry_count=0):
+    def create_img(self, query, retry_count=0, api_key=None):
         try:
             if conf().get("rate_limit_dalle") and not self.tb4dalle.get_token():
                 return False, "请求太快了，请休息一下再问我吧"
             logger.info("[OPEN_AI] image_query={}".format(query))
             response = openai.Image.create(
+                api_key=api_key,
                 prompt=query,  # 图片描述
                 n=1,  # 每次生成图片的数量
                 size=conf().get("image_create_size", "256x256"),  # 图片大小,可选有 256x256, 512x512, 1024x1024

channel/chat_channel.py

@@ -48,6 +48,8 @@ def _compose_context(self, ctype: ContextType, content, **kwargs):
         if first_in:  # context首次传入时，receiver是None，根据类型设置receiver
             config = conf()
             cmsg = context["msg"]
+            user_data = conf().get_user_data(cmsg.from_user_id)
+            context["openai_api_key"] = user_data.get("openai_api_key")
             if context.get("isgroup", False):
                 group_name = cmsg.other_user_nickname
                 group_id = cmsg.other_user_id

channel/wechatcom/README.md

@@ -14,6 +14,18 @@
 
 本channel需安装的依赖与公众号一致，需要安装`wechatpy`和`web.py`，它们包含在`requirements-optional.txt`中。
 
+此外，如果你是`Linux`系统，除了`ffmpeg`还需要安装`amr`编码器，否则会出现找不到编码器的错误，无法正常使用语音功能。
+
+- Ubuntu/Debian
+
+```bash
+apt-get install libavcodec-extra
+```
+
+- Alpine
+
+需自行编译`ffmpeg`，在编译参数里加入`amr`编码器的支持
+
 ## 使用方法
 
 1.查看企业ID
@@ -29,7 +41,7 @@
 
 3.配置应用
 
-- 在详情页如果点击`企业可信IP`的配置(没看到可以不管)，填入你服务器的公网IP
+- 在详情页点击`企业可信IP`的配置(没看到可以不管)，填入你服务器的公网IP，如果不知道可以先不填
 - 点击`接收消息`下的启用API接收消息
 - `URL`填写格式为`http://url:port/wxcomapp`，`port`是程序监听的端口，默认是9898
     如果是未认证的企业，url可直接使用服务器的IP。如果是认证企业，需要使用备案的域名，可使用二级域名。
@@ -52,8 +64,24 @@
 
 选择`我的企业`，点击`微信插件`，下面有个邀请关注的二维码。微信扫码后，即可在微信中看到对应企业，在这里你便可以和机器人沟通。
 
+向机器人发送消息，如果日志里出现报错:
+
+```bash
+Error code: 60020, message: "not allow to access from your ip, ...from ip: xx.xx.xx.xx"
+```
+
+意思是IP不可信，需要参考上一步的`企业可信IP`配置，把这里的IP加进去。
+
+### Railway部署方式
+
+公众号不能在`Railway`上部署，但企业微信应用[可以](https://railway.app/template/-FHS--?referralCode=RC3znh)!
+
+[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/-FHS--?referralCode=RC3znh)
+
+填写配置后，将部署完成后的网址```**.railway.app/wxcomapp```，填写在上一步的URL中。发送信息后观察日志，把报错的IP加入到可信IP。（每次重启后都需要加入可信IP）
+
 ## 测试体验
 
 AIGC开放社区中已经部署了多个可免费使用的Bot，扫描下方的二维码会自动邀请你来体验。
 
-<img width="200" src="../../docs/images/aigcopen.png">
+<img width="200" src="../../docs/images/aigcopen.png">

channel/wechatcom/wechatcomapp_channel.py

@@ -19,7 +19,7 @@
 from common.singleton import singleton
 from common.utils import compress_imgfile, fsize, split_string_by_utf8_length
 from config import conf, subscribe_msg
-from voice.audio_convert import any_to_amr
+from voice.audio_convert import any_to_amr, split_audio
 
 MAX_UTF8_LEN = 2048
 
@@ -63,11 +63,17 @@ def send(self, reply: Reply, context: Context):
             logger.info("[wechatcom] Do send text to {}: {}".format(receiver, reply_text))
         elif reply.type == ReplyType.VOICE:
             try:
+                media_ids = []
                 file_path = reply.content
                 amr_file = os.path.splitext(file_path)[0] + ".amr"
                 any_to_amr(file_path, amr_file)
-                response = self.client.media.upload("voice", open(amr_file, "rb"))
-                logger.debug("[wechatcom] upload voice response: {}".format(response))
+                duration, files = split_audio(amr_file, 60 * 1000)
+                if len(files) > 1:
+                    logger.info("[wechatcom] voice too long {}s > 60s , split into {} parts".format(duration / 1000.0, len(files)))
+                for path in files:
+                    response = self.client.media.upload("voice", open(path, "rb"))
+                    logger.debug("[wechatcom] upload voice response: {}".format(response))
+                    media_ids.append(response["media_id"])
             except WeChatClientException as e:
                 logger.error("[wechatcom] upload voice failed: {}".format(e))
                 return
@@ -77,7 +83,9 @@ def send(self, reply: Reply, context: Context):
                     os.remove(amr_file)
             except Exception:
                 pass
-            self.client.message.send_voice(self.agent_id, receiver, response["media_id"])
+            for media_id in media_ids:
+                self.client.message.send_voice(self.agent_id, receiver, media_id)
+                time.sleep(1)
             logger.info("[wechatcom] sendVoice={}, receiver={}".format(reply.content, receiver))
         elif reply.type == ReplyType.IMAGE_URL:  # 从网络下载图片
             img_url = reply.content

channel/wechatmp/README.md

@@ -18,7 +18,7 @@ pip3 install wechatpy
 
 然后在[微信公众平台](https://mp.weixin.qq.com)注册一个自己的公众号，类型选择订阅号，主体为个人即可。
 
-然后根据[接入指南](https://developers.weixin.qq.com/doc/offiaccount/Basic_Information/Access_Overview.html)的说明，在[微信公众平台](https://mp.weixin.qq.com)的“设置与开发”-“基本配置”-“服务器配置”中填写服务器地址`URL`和令牌`Token`。这里的`URL`是`example.com/wx`的形式，不可以使用IP，`Token`是你自己编的一个特定的令牌。消息加解密方式如果选择了需要加密的模式，需要在配置中填写`wechatmp_aes_key`。
+然后根据[接入指南](https://developers.weixin.qq.com/doc/offiaccount/Basic_Information/Access_Overview.html)的说明，在[微信公众平台](https://mp.weixin.qq.com)的“设置与开发”-“基本配置”-“服务器配置”中填写服务器地址`URL`和令牌`Token`。`URL`填写格式为`http://url/wx`，可使用IP（成功几率看脸），`Token`是你自己编的一个特定的令牌。消息加解密方式如果选择了需要加密的模式，需要在配置中填写`wechatmp_aes_key`。
 
 相关的服务器验证代码已经写好，你不需要再添加任何代码。你只需要在本项目根目录的`config.json`中添加
 ```

channel/wechatmp/active_reply.py

@@ -57,10 +57,6 @@ def POST(self):
                 else:
                     context = channel._compose_context(wechatmp_msg.ctype, content, isgroup=False, msg=wechatmp_msg)
                 if context:
-                    # set private openai_api_key
-                    # if from_user is not changed in itchat, this can be placed at chat_channel
-                    user_data = conf().get_user_data(from_user)
-                    context["openai_api_key"] = user_data.get("openai_api_key")  # None or user openai_api_key
                     channel.produce(context)
                 # The reply will be sent by channel.send() in another thread
                 return "success"

channel/wechatmp/passive_reply.py

@@ -63,10 +63,6 @@ def POST(self):
                     logger.debug("[wechatmp] context: {} {} {}".format(context, wechatmp_msg, supported))
 
                     if supported and context:
-                        # set private openai_api_key
-                        # if from_user is not changed in itchat, this can be placed at chat_channel
-                        user_data = conf().get_user_data(from_user)
-                        context["openai_api_key"] = user_data.get("openai_api_key")
                         channel.running.add(from_user)
                         channel.produce(context)
                     else:

common/const.py

@@ -3,3 +3,5 @@
 CHATGPT = "chatGPT"
 BAIDU = "baidu"
 CHATGPTONAZURE = "chatGPTOnAzure"
+
+VERSION = "1.3.0"

docker/Dockerfile.debian.latest → docker/Dockerfile.alpine.latest

@@ -1,4 +1,4 @@
-FROM python:3.10-slim
+FROM python:3.10-alpine
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -9,23 +9,19 @@ ENV BUILD_PREFIX=/app
 
 ADD . ${BUILD_PREFIX}
 
-RUN apt-get update \
-    &&apt-get install -y --no-install-recommends  bash \
-    ffmpeg espeak \
+RUN apk add --no-cache bash ffmpeg espeak \
     && cd ${BUILD_PREFIX} \
     && cp config-template.json config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache -r requirements.txt \
-    && pip install --no-cache -r requirements-optional.txt \
-    && pip install azure-cognitiveservices-speech
+    && pip install --no-cache -r requirements.txt --extra-index-url https://alpine-wheels.github.io/index\
+    && pip install --no-cache -r requirements-optional.txt --extra-index-url https://alpine-wheels.github.io/index
 
 WORKDIR ${BUILD_PREFIX}
 
 ADD docker/entrypoint.sh /entrypoint.sh
 
 RUN chmod +x /entrypoint.sh \
-    && groupadd -r noroot \
-    && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
+    && adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
     && chown -R noroot:noroot ${BUILD_PREFIX}
 
 USER noroot

docker/Dockerfile.latest

@@ -1,27 +1,32 @@
-FROM python:3.10-alpine
+FROM python:3.10-slim
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
 
 ARG CHATGPT_ON_WECHAT_VER
 
+RUN echo /etc/apt/sources.list
+RUN sed -i 's/deb.debian.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list
 ENV BUILD_PREFIX=/app
 
 ADD . ${BUILD_PREFIX}
 
-RUN apk add --no-cache bash ffmpeg espeak \
+RUN apt-get update \
+    &&apt-get install -y --no-install-recommends bash ffmpeg espeak libavcodec-extra\
     && cd ${BUILD_PREFIX} \
     && cp config-template.json config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache -r requirements.txt --extra-index-url https://alpine-wheels.github.io/index\
-    && pip install --no-cache -r requirements-optional.txt --extra-index-url https://alpine-wheels.github.io/index
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt \
+    && pip install azure-cognitiveservices-speech
 
 WORKDIR ${BUILD_PREFIX}
 
 ADD docker/entrypoint.sh /entrypoint.sh
 
 RUN chmod +x /entrypoint.sh \
-    && adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
+    && groupadd -r noroot \
+    && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
     && chown -R noroot:noroot ${BUILD_PREFIX}
 
 USER noroot

docker/entrypoint.sh

@@ -38,9 +38,9 @@ if [ "$CHATGPT_ON_WECHAT_EXEC" == "" ] ; then
 fi
 
 # modify content in config.json
-if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] || [ "$OPEN_AI_API_KEY" == "" ]; then
-    echo -e "\033[31m[Warning] You need to set OPEN_AI_API_KEY before running!\033[0m"
-fi
+# if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] || [ "$OPEN_AI_API_KEY" == "" ]; then
+#     echo -e "\033[31m[Warning] You need to set OPEN_AI_API_KEY before running!\033[0m"
+# fi
 
 
 # go to prefix dir

nixpacks.toml

@@ -2,6 +2,6 @@ providers = ['python']
 
 [phases.setup]
 nixPkgs = ['python310']
-cmds = ['apt-get update','apt-get install -y --no-install-recommends ffmpeg espeak','python -m venv /opt/venv && . /opt/venv/bin/activate && pip install -r requirements-optional.txt']
+cmds = ['apt-get update','apt-get install -y --no-install-recommends ffmpeg espeak libavcodec-extra','python -m venv /opt/venv && . /opt/venv/bin/activate && pip install -r requirements-optional.txt']
 [start]
 cmd = "python ./app.py"

plugins/tool/tool.py

@@ -184,6 +184,7 @@ def _build_tool_kwargs(self, kwargs: dict):
 
             # for visual_dl tool
             "cuda_device": kwargs.get("cuda_device", "cpu"),
+
             # for browser tool
             "phantomjs_exec_path": kwargs.get("phantomjs_exec_path", ""),
 
@@ -192,6 +193,11 @@ def _build_tool_kwargs(self, kwargs: dict):
 
             # for wolfram-alpha tool
             "wolfram_alpha_appid": kwargs.get("wolfram_alpha_appid", wolfram_alpha_appid),
+
+            "think_depth": kwargs.get("think_depth", 3),
+            "arxiv_summary": kwargs.get("arxiv_summary", True),
+            "morning_news_use_llm": kwargs.get("morning_news_use_llm", False),
+
         }
 
     def _filter_tool_list(self, tool_list: list):

requirements-optional.txt

@@ -6,7 +6,7 @@ SpeechRecognition # google speech to text
 gTTS>=2.3.1 # google text to speech
 pyttsx3>=2.90 # pytsx text to speech
 baidu_aip>=4.16.10 # baidu voice
-# azure-cognitiveservices-speech # azure voice
+azure-cognitiveservices-speech # azure voice
 numpy<=1.24.2
 langid # language detect
 
@@ -25,4 +25,4 @@ wechatpy
 # chatgpt-tool-hub plugin
 
 --extra-index-url https://pypi.python.org/simple
-chatgpt_tool_hub>=0.4.1
+chatgpt_tool_hub==0.4.3

voice/audio_convert.py

@@ -92,6 +92,7 @@ def any_to_amr(any_path, amr_path):
     audio = AudioSegment.from_file(any_path)
     audio = audio.set_frame_rate(8000)  # only support 8000
     audio.export(amr_path, format="amr")
+    return audio.duration_seconds * 1000
 
 
 def sil_to_wav(silk_path, wav_path, rate: int = 24000):
@@ -101,3 +102,26 @@ def sil_to_wav(silk_path, wav_path, rate: int = 24000):
     wav_data = pysilk.decode_file(silk_path, to_wav=True, sample_rate=rate)
     with open(wav_path, "wb") as f:
         f.write(wav_data)
+
+
+def split_audio(file_path, max_segment_length_ms=60000):
+    """
+    分割音频文件
+    """
+    audio = AudioSegment.from_file(file_path)
+    audio_length_ms = len(audio)
+    if audio_length_ms <= max_segment_length_ms:
+        return audio_length_ms, [file_path]
+    segments = []
+    for start_ms in range(0, audio_length_ms, max_segment_length_ms):
+        end_ms = min(audio_length_ms, start_ms + max_segment_length_ms)
+        segment = audio[start_ms:end_ms]
+        segments.append(segment)
+    file_prefix = file_path[: file_path.rindex(".")]
+    format = file_path[file_path.rindex(".") + 1 :]
+    files = []
+    for i, segment in enumerate(segments):
+        path = f"{file_prefix}_{i+1}" + f".{format}"
+        segment.export(path, format=format)
+        files.append(path)
+    return audio_length_ms, files