help you get a html like json
<dependency>
<groupId>com.github.qlone</groupId>
<artifactId>retrofit-crawler</artifactId>
<version>1.0.0</version>
</dependency>
implementation 'com.github.qlone:retrofit-crawler:1.0.0'
First, you can create a interface like retrofit and add some rules like json.
public interface Google {
@POST("/topstories")
Call<TopStoyResp> topstories();
}
public class TopStoyResp {
@Select("div[jscontroller = d0DtYd]")
private List<News> news;
public static class News{
@Text("article h3 a")
private String titile;
@Attr(value = "div[jsname = gKDw6b] a",attr = "href")
private String url;
}
}
Create a retrofitCrawler in order to init our interface
RetrofitCrawler retrofitCrawler = new RetrofitCrawler.Builder()
.baseUrl("https://news.google.com")
.build();
Google google = retrofitCrawler.create(Google.class);
Now, you can use google.class to get some data. You can execute sync or async.
Call<TopStoyResp> topstories = google.topstories();
Response<TopStoyResp> execute = topstories.execute();
TopStoyResp resp = execute.body();
Report issues to hewinana@gmail.com or github issues