The original intention of the project is to crawl and update vulnerabilities (including vulnerability name, vulnerability number, vulnerability POC link, vulnerability update time) in time, and call Vuln interface of shodan's API synchronously to acquire the corresponding assets IP in real time, and send the most regular mail every day. New vulnerabilities, and finally establish a vulnerability Threat Intelligence database, through real-time collection of data, real-time updates to the database, establish a search index, through the query of keywords, get the details of other vulnerabilities and related threats IP.
Current progress: real time crawling of vulnerabilities
Difficulty: Shodan's Vuln interface free channel has been closed.
Tasks: email alert, data access and search index
项目初衷为威胁情报爬虫，通过漏洞的及时爬取与更新（包括漏洞名、漏洞编号、漏洞POC链接、漏洞更新时间），并同步调用shodan的api的vuln接口，实时获取对应漏洞的全国范围内对应的所涉及的资产ip，每天定期的邮件推送最新的漏洞，最后建立一个漏洞威胁情报数据库，通过实时收集的数据，实时的更新到数据库中，建立搜索索引，通过查询关键字，得到其他漏洞详细和相关存在威胁IP。
-----start python3 taskCrawl.py
---设置爬行任务时间周期
---Set up crawling task time cycle
----保存文件说明： 当前路径为程序演示执行路径，实际运行时将pipeline.py文件下的漏洞保存路径改为./vul-file/vul-today.json
---crawl result:
