-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
您好,AyugeSpiderTools是否可以增加对elascticsearch的支持 #15
Comments
经考虑,目前
通过 此 |
经再次考虑,还是得需要方式 2 的实现方式,方式 3 的实现还是因为太丑陋而无法接受。 那么, 推荐有 |
非常感谢,这两天测试一下 |
feat: add es support 的 |
支持 |
Is your feature request related to a problem? Please describe.
原来使用的关系型数据库,数据爬到一定量级时,查询索引越来越慢,优化难度很大
Describe the solution you'd like
希望作者考虑一下增加对elascticsearch的入库支持,通过对es的常规配置和item进行封装,从而大幅减少项目的多次重复配置
Describe alternatives you've considered
scrapy入库elascticsearch的场景蛮多的,每天爬取导致日益增加的数据索引降低,需要es这样专业的索引框架解决问题
Additional context
感谢您的开源库,使得我们写爬虫大幅提高生产效率,希望AyugeSpiderTools越来越强大
The text was updated successfully, but these errors were encountered: