Skip to content

voidfiles/newspaper-delivery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Newspaper Delivery

newspaper is a python based article extraction framework. It can produce pocket-like results. While it is very easy to use as a python library, I thought it would be interesting to turn this into a hosted app. Its all setup to be heroku app.

Installation

git clone https://github.com/voidfiles/newspaper-delivery.git
cd newspaper-delivery
virtualenv --no-site-packages venv
source venv/bin/activate
pip install -r requirements.txt
python app.py

You should now have a local version of the app running at http://127.0.0.1:5000/

User

You can now try it out there are two main endpoints. First is the /article endpoint. This works with newspapers article extraction.

curl http://localhost:5000/article?url=http://www.macdrifter.com/2014/01/list-of-natural-language-apis-link.html

Will produce something like this

{
    "additional_data": {},
    "authors": [],
    "canonical_link": "http://www.macdrifter.com/2014/01/list-of-natural-language-apis-link.html",
    "images": [
        "http://www.macdrifter.com/theme/images/logo.svg",
        ...
    ],
    "keywords": [
        "rundown",
		...
    ],
    "meta_description": "",
    "meta_favicon": "/theme/images/touch-icon-iphone.png",
    "meta_keywords": [
        ""
    ],
    "meta_lang": "en",
    "movies": [],
    "summary": [],
    "tags": [
        "Programming",
        "Link"
    ],
    "text": "This is a nice rundown of some natural language APIs.\n\nThe source is Mashape which looks like an API middleman. It's an interesting idea and their index looks very good and it's searchable.\n\nDon't miss the link at the bottom for PublicAPI.com, which I've seen before but always forget about.",
    "title": "List of Natural Language APIs [Link]",
    "top_image": "/theme/images/favicon.png",
    "url": "http://www.macdrifter.com/2014/01/list-of-natural-language-apis-link.html"
}

Lastly is the /source endpoint. Which is synonmous with the main interface of newspaper.

curl http://localhost:5000/source?url=http://cnn.com

Will produce something like this

{
    "articles": [
        "http://www.cnn.com/2012/04/04/opinion/colb-strip-search/index.html",
        "http://ac360.blogs.cnn.com/2013/12/03/pick-your-favorite-ridiculist-of-2013/",
        "http://piersmorgan.blogs.cnn.com/2013/11/08/william-shatner-on-obamacare-ponder-the-mystery/",
        ...
    ],
    "brand": "cnn",
    "categories": [
        "http://cnn.com",
        "http://cnn.com/HLN",
        "https://portfolio.money.cnn.com",
        "http://ireport.cnn.com",
        ...
    ],
    "description": "CNN.com delivers the latest breaking news and information on the latest top stories, weather, business, entertainment, politics, and more. For in-depth coverage, CNN.com provides special reports, video, audio, photo galleries, and interactive guides.",
    "domain": "cnn.com",
    "favicon": "",
    "feeds": [
        "http://rss.cnn.com/rss/cnn_tech.rss",
        ...
    ],
    "logo_url": ""
}

About

A heroku app that exposes features from https://github.com/codelucas/newspaper/ as json API

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published