Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] 支持 GoogleCustomSearch 插件 #89

Closed
snakeying opened this issue Dec 11, 2023 · 15 comments · Fixed by #120
Closed

[Feature] 支持 GoogleCustomSearch 插件 #89

snakeying opened this issue Dec 11, 2023 · 15 comments · Fixed by #120
Labels
enhancement New feature or request

Comments

@snakeying
Copy link

在vercel部署

当CHOOSE_SEARCH_ENGINE变量为baidu时候

天气预报 (1)

天气预报

当不使用CHOOSE_SEARCH_ENGINE,默认DuckDuckGo时候则是正常的

天气预报
新的聊天

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Title: [Bug] Search engine is unstable

@Hk-Gosuto
Copy link
Owner

我在本地调试的,并且输出了百度爬到的内容,并没发现问题,可能是网络原因导致的。
image
image

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I debugged it locally and output the content crawled by Baidu. No problem was found. It may be caused by network reasons.
image
image

@snakeying
Copy link
Author

调试反馈,使用docker本地部署的话确实没有问题,看来应该是vercel限制了?

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Debugging feedback, there is indeed no problem if you use docker for local deployment. It seems that it is limited by vercel?

@Hk-Gosuto
Copy link
Owner

可以在日志中把插件中的输出打印一下看看具体原因,也不好说是不是百度限制了vercel的ip访问。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


You can print the output from the plug-in in the log to see the specific reason. It's hard to say whether Baidu has restricted vercel's IP access.

@snakeying
Copy link
Author

snakeying commented Dec 25, 2023

经过多次尝试,总结了一下搜索插件的情况

1.在vercel部署,不建议使用CHOOSE_SEARCH_ENGINE变量,因为baidu自动屏蔽vercel的IP导致返回302错误,google则表现不稳定,经常抽风

2.在私人服务器上部署(配合openwet passwall) 可以使用CHOOSE_SEARCH_ENGINE变量,无论是baidu还是google都可以,但搜索质量不敢恭维,还不如webchatgpt插件。

3.bing跟serp 的api搜索质量明显比直接使用搜索引擎要好,但严重受限,serp每月只有100条,bing每月1000条还有TPS限制。

建议加入Google Search Plugin,需要获取谷歌自定义搜索 API 密钥和谷歌自定义搜索引擎 ID,每天的查询数上限为 10,000 次。

在env中需要加入

GOOGLE_API_KEY="XXXXXXXX"
GOOGLE_CSE_ID="XXXXXXXXX"

相关网址

https://developers.google.com/custom-search/docs/tutorial/creatingcse

https://developers.google.com/custom-search/v1/introduction?hl=zh-cn

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


After many attempts, I summarized the situation of searching for plug-ins.

  1. When deploying in vercel, it is not recommended to use the CHOOSE_SEARCH_ENGINE variable, because Baidu automatically blocks vercel’s IP, causing a 302 error to be returned, and google’s performance is unstable and often convulses.

  2. Deploy on a private server (with openwet passwall) You can use the CHOOSE_SEARCH_ENGINE variable, whether it is Baidu or Google, but the search quality is not flattering and is not as good as the webchatgpt plug-in.

  3. The API search quality of Bing and SERP is obviously better than using the search engine directly, but it is severely limited. SERP only has 100 entries per month, and Bing has 1,000 entries per month and TPS limit.

It is recommended to join the Google Search Plugin and apply for Google's Custom Search JSON API. The maximum number of queries per day is 10,000.

https://developers.google.com/custom-search/v1/introduction?hl=zh-cn

@Hk-Gosuto
Copy link
Owner

经过多次尝试,总结了一下搜索插件的情况

1.在vercel部署,不建议使用CHOOSE_SEARCH_ENGINE变量,因为baidu自动屏蔽vercel的IP导致返回302错误,google则表现不稳定,经常抽风

2.在私人服务器上部署(配合openwet passwall) 可以使用CHOOSE_SEARCH_ENGINE变量,无论是baidu还是google都可以,但搜索质量不敢恭维,还不如webchatgpt插件。

3.bing跟serp 的api搜索质量明显比直接使用搜索引擎要好,但严重受限,serp每月只有100条,bing每月1000条还有TPS限制。

建议加入Google Search Plugin,需要获取谷歌自定义搜索 API 密钥和谷歌自定义搜索引擎 ID,每天的查询数上限为 10,000 次。

在env中需要加入

GOOGLE_API_KEY="XXXXXXXX" GOOGLE_CSE_ID="XXXXXXXXX"

相关网址

https://developers.google.com/custom-search/docs/tutorial/creatingcse

https://developers.google.com/custom-search/v1/introduction?hl=zh-cn

vercel 下的 CHOOSE_SEARCH_ENGINE 配置确实不稳定,该插件是基于爬虫实现的,本身就存在很多不稳定性。
该问题我会在文档中进行标注。

@snakeying
Copy link
Author

有考虑加入Google Search Plugin吗?σ^_^;

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Have you considered joining the Google Search Plugin? σ^_^;

@Hk-Gosuto Hk-Gosuto added the enhancement New feature or request label Dec 25, 2023
@Hk-Gosuto
Copy link
Owner

有考虑加入Google Search Plugin吗?σ^_^;

我看 langchain 有这个插件,我试试引入进来。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Have you considered joining the Google Search Plugin? σ^_^;

I see that langchain has this plug-in, I will try to introduce it.

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Then leave it to the boss ☆*:.。. o(≧▽≦)o .。.:*☆

@Hk-Gosuto Hk-Gosuto changed the title [Bug] 搜索引擎不稳定 [Feature] 支持 GoogleCustomSearch 插件 Dec 26, 2023
@Hk-Gosuto Hk-Gosuto linked a pull request Dec 26, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants