Skip to content

error java.lang.IllegalArgumentException: Illegal character(Fragment_identifier) #629

@alphatok

Description

@alphatok

HttpUriRequestConverter是否应该做个通用处理(像浏览器一样),不用每个用户都要去处理。
https://en.wikipedia.org/wiki/Fragment_identifier

process request Request{url='http://bj.zhongkao.com/beikao/yimo/##', method='null', extras={level=2}, priority=0, headers={HOST=bj.zhongkao.com, Domain=bj.zhongkao.com}, cookies={}} error java.lang.IllegalArgumentException: Illegal character in fragment at index 36: http://bj.zhongkao.com/beikao/yimo/##
at java.net.URI.create(URI.java:859)
at org.apache.http.client.methods.RequestBuilder.setUri(RequestBuilder.java:345)
at us.codecraft.webmagic.downloader.HttpUriRequestConverter.convertHttpUriRequest(HttpUriRequestConverter.java:60)
at us.codecraft.webmagic.downloader.HttpUriRequestConverter.convert(HttpUriRequestConverter.java:35)
at us.codecraft.webmagic.downloader.HttpClientDownloader.download(HttpClientDownloader.java:83)
at us.codecraft.webmagic.Spider.processRequest(Spider.java:404)
at us.codecraft.webmagic.Spider.access$000(Spider.java:61)
at us.codecraft.webmagic.Spider$1.run(Spider.java:320)
at us.codecraft.webmagic.thread.CountableThreadPool$1.run(CountableThreadPool.java:74)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.URISyntaxException: Illegal character in fragment at index 36: http://bj.zhongkao.com/beikao/yimo/##
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.parse(URI.java:3048)
at java.net.URI.(URI.java:595)
at java.net.URI.create(URI.java:857)
... 11 more

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions