We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
crawlData 请求数据,使用 CrawlDataDetailTargetConfig 的 params 传递请求参数,发现实际请求并未携带参数。
CrawlDataDetailTargetConfig
params
查看源码发现 此处 将 params 参数处理为 search,然后在 此处 传递给 https.request() 方法进行请求。
search
https.request()
查看 Node.js 文档,http.request() 的解释 HTTP #httprequestoptions-callback | Node.js v18.16.0 Documentation,请求参数应该是拼接在 path 后。
http.request()
path
const pageSize = 30; const sort = 'time'; const rating = 'all'; const filter = 'all'; const mode = 'list'; const url = `https://${type}.${this.BASE_URL}/people/${uid}/${status}`; const request: CrawlDataDetailTargetConfig = { url, method: "GET", params: { start: (page - 1) * pageSize, sort, rating, filter, mode, }, data: {}, timeout: 30000, maxRetry: 0 }; const crawler = Crawl({ enableRandomFingerprint: true }); const res = await crawler.crawlData(request);
无报错
7.0.0
18.15.0
npm
8.3.1
The text was updated successfully, but these errors were encountered:
Message that will be displayed on users' first issue
Sorry, something went wrong.
感谢您的 Bug 反馈,我到时候开机再处理,现在挺晚的。
感谢您的 Bug 反馈,我到时候开机再处理,现在挺晚的。 http 和 https 这两个不能传 RequestOptions 交叉 URL 类型,URL 类型有 search ,但 RequestOptions 没有。
你合适时间处理就好,我的作息有点异常 😄。
已经修复并发布了
Successfully merging a pull request may close this issue.
Bug 预期
crawlData 请求数据,使用
CrawlDataDetailTargetConfig
的params
传递请求参数,发现实际请求并未携带参数。查看源码发现 此处 将
params
参数处理为search
,然后在 此处 传递给https.request()
方法进行请求。查看 Node.js 文档,
http.request()
的解释 HTTP #httprequestoptions-callback | Node.js v18.16.0 Documentation,请求参数应该是拼接在path
后。最小可重复的例子
报错信息
无报错
x-crawl 版本
7.0.0
Node 版本
18.15.0
包管理器
npm
包管理器版本
8.3.1
The text was updated successfully, but these errors were encountered: