New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

downloader 设置 proxy 与 site 设置 proxy 有区别吗？ #1081

Open

nesteiner opened this issue Sep 26, 2022 · 1 comment

nesteiner commented Sep 26, 2022

我发现可以对Site对象设置代理

也可以对downloader设置代理

        HttpClientDownloader downloader = new HttpClientDownloader();
        downloader.setProxyProvider(SimpleProxyProvider.from(
                new Proxy("127.0.0.1", 7890)
        ));

        Spider.create(this)
                .addUrl(starturl)
                .addPipeline(new XiannvPipeline())
                .setDownloader(downloader)
                .runAsync();

我的问题是，这两个方法等效吗，为什么要这样设置?

Collaborator

sutra commented Sep 27, 2022

从0.7.1版本开始，WebMagic开始使用了新的代理APIProxyProvider。因为相对于Site的“配置”，ProxyProvider定位更多是一个“组件”，所以代理不再从Site设置，而是由HttpClientDownloader设置。

http://webmagic.io/docs/zh/posts/ch4-basic-page-processor/proxy.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment