Every web site provides APIs.
Switch branches/tags
Clone or download
Latest commit 936ca1a Aug 12, 2018
Permalink
Failed to load latest commit information.
docs Update: docs Jan 9, 2018
examples support ajax Mar 5, 2018
tests 100% test Mar 4, 2018
toapi release 2.1.2 Aug 12, 2018
.gitignore Redesign toapi api Mar 2, 2018
.travis.yml fix coverage==0 Mar 4, 2018
LICENSE Add logo Dec 13, 2017
MANIFEST.in Update: Add MANIFEST.in Dec 19, 2017
README.md update todo Mar 4, 2018
logo.png Update logo Dec 13, 2017
mkdocs.yml Add: articles in docs Dec 26, 2017
requirements.txt code cov Mar 4, 2018
setup.cfg Python3.6+ only Mar 4, 2018
setup.py dependency htmlfetcher Mar 5, 2018

README.md

Toapi

Build Coverage Python Version License

Toapi

Overview

Toapi give you the ability to make every web site provides APIs.

Version v2.0.0, Completely rewrote.

More elegant. More pythonic

Features

  • Automatic converting HTML web site to API service.
  • Automatic caching every page of source site.
  • Automatic caching every request.
  • Support merging multiple web sites into one API service.

Get Started

Installation

$ pip install toapi
$ toapi -v
toapi, version 2.0.0

Usage

create app.py and copy the code:

from flask import request
from htmlparsing import Attr, Text
from toapi import Api, Item

api = Api()


@api.site('https://news.ycombinator.com')
@api.list('.athing')
@api.route('/posts?page={page}', '/news?p={page}')
@api.route('/posts', '/news?p=1')
class Post(Item):
    url = Attr('.storylink', 'href')
    title = Text('.storylink')


@api.site('https://news.ycombinator.com')
@api.route('/posts?page={page}', '/news?p={page}')
@api.route('/posts', '/news?p=1')
class Page(Item):
    next_page = Attr('.morelink', 'href')

    def clean_next_page(self, value):
        return api.convert_string('/' + value, '/news?p={page}', request.host_url.strip('/') + '/posts?page={page}')


api.run(debug=True, host='0.0.0.0', port=5000)

run python app.py

then open your browser and visit http://127.0.0.1:5000/posts?page=1

you will get the result like:

{
  "Page": {
    "next_page": "http://127.0.0.1:5000/posts?page=2"
  }, 
  "Post": [
    {
      "title": "Mathematicians Crack the Cursed Curve", 
      "url": "https://www.quantamagazine.org/mathematicians-crack-the-cursed-curve-20171207/"
    }, 
    {
      "title": "Stuffing a Tesla Drivetrain into a 1981 Honda Accord", 
      "url": "https://jalopnik.com/this-glorious-madman-stuffed-a-p85-tesla-drivetrain-int-1823461909"
    }
  ]
}

Todo

  1. Visualization. Create toapi project in a web page by drag and drop.

Contributing

Write code and test code and pull request.