Skip to content

A simple API-like from html website (scrapper) for Android

License

Notifications You must be signed in to change notification settings

MuhammadAmmad/RxRetroJsoup

 
 

Repository files navigation

RxRetroJsoup

Create an interface with @Select annotated method

public interface TutosAndroidFrance {

    @Select("article")
    Observable<Article> articles();

}

Annotate your model with Jsoup queries ( https://jsoup.org/cookbook/extracting-data/selector-syntax )

public class Article {

    @JsoupText(".entry-title a")
    String title;

    @JsoupHref(".read-more a")
    String href;

    @JsoupSrc(".entry-thumb img")
    String image;

    @JsoupText(".entry-content p")
    String description;

    public Article() {
    }
    ...

}

Build a RetroJsoup

final TutosAndroidFrance tutosAndroidFrance = new RetroJsoup.Builder()
                .url("http://tutos-android-france.com/")

                //optionally
                .client(your_okhttp_client)

                .build()
                .create(TutosAndroidFrance.class);

Fetch your objects !

tutosAndroidFrance.articles()
                .toList()

                .subscribeOn(Schedulers.newThread())
                .observeOn(AndroidSchedulers.mainThread())

                .subscribe(
                    new Action1<List<Article>>() {
                            @Override
                            public void call(List<Article> items) {

                            }
                });

#RxJsoup

RxJsoup.with(url)
    .select("article")
    .flatMap(element -> rxJsoup.text(element, ".entry-title a"))
    .subscribe(text -> {});


    .flatMap(element -> rxJsoup.src(element, ".entry img"))
    .flatMap(element -> rxJsoup.href(element, ".entry a"))
    .flatMap(element -> rxJsoup.attr(element, ".entry h1", "id"))
RxJsoup.connect(

           Jsoup.connect("www.thewebsite.com")
               .userAgent(MY_USER_AGENT)
               .data("credential", email)
               .data("pwd", password)
               .cookies(loginForm.cookies())
               .method(Connection.Method.POST)

           )
           .subscibe(response -> {})

#Download

Buy Me a Coffee at ko-fi.com

In your module Download

compile 'com.github.florent37:retrojsoup:1.0.2'
compile 'com.github.florent37:rxjsoup:1.0.2'
annotationProcessor 'com.github.florent37:retrojsoup-compiler:1.0.2'

//don't forget to include jsoup & rxjava
compile 'org.jsoup:jsoup:1.10.2'
compile 'io.reactivex:rxjava:1.2.7'

//optionaly
compile 'com.squareup.okhttp3:okhttp:3.6.0'

Credits

Author: Florent Champigny

Follow me on Google+ Follow me on Twitter Follow me on LinkedIn

#License

Copyright 2017 florent37, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

About

A simple API-like from html website (scrapper) for Android

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 100.0%