eye on the news
Python
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
data
reality
feedfinder.py
feeds.txt
listeners.txt
main.py
readme.md
requirements.txt
setup.py

readme.md

  • add feeds to feeds.txt
  • run main.py
    • this will update the feeds every hour

to retrieve articles for a feed:

from reality import get_articles
feed = 'http://feeds.theguardian.com/theguardian/us/rss'
articles = get_articles(feed)

listeners

you can add "listeners" (which are really just FIFO queues) that have article data appended to them when new articles are retrieved. see listeners.txt for an example.

an example listener script:

import hashlib
import fasteners

fifo = '/tmp/fifo'
while True:
    hash = hashlib.md5(fifo.encode('utf8')).hexdigest()
    with fasteners.InterProcessLock('/tmp/{}.lock'.format(hash)):
        try:
            with open(fifo, 'r+') as f:
                for l in f:
                    article = json.loads(l.strip())
                    on_article(article)
            open(fifo, 'w').close()
        except FileNotFoundError:
            print('no fifo')
        sleep(10)
    except (KeyboardInterrupt, SystemExit):
        break

feed discovery

to find rss feeds for a given url, you can use the included feedfinder.py script:

python feedfinder.py foo.com