Opinoid is a simple web application which syndicates news from a set of known RSS/Atom sources and provides visitors with a sorted list of articles (like Reddit), based on user’s votes and comments. It uses TornadoWeb, MongoDB, jQuery and Bootstrap.
The web app file are in the
webapp folder, some additional utilities are in the
utils folder. The utils initialize the db and and update the list of articles periodically.
The python modules required to get the web app running:
$ pip freeze pep8==0.6.1 pycurl==7.19.0 pygeoip==0.2.2 pymongo==2.0.1 six==1.0.0 textile==2.1.5 tornado==2.1.1 wsgiref==0.1.2
If you’re using python <= 2.6 you will also need to install the
argparse module. You’ll also need a working install of mongodb.
$ ./opinoid.py --help Usage: ./opinoid.py [OPTIONS] Options: --help show this help information --log_file_max_size max size of log files before rollover --log_file_num_backups number of log files to keep --log_file_prefix=PATH Path prefix for log files. Note that if you are running multiple tornado processes, log_file_prefix must be different for each of them (e.g. \ include the port number) --log_to_stderr Send log output to stderr (colorized if possible). By default use stderr if --log_file_prefix is not set and no other logging is configured. --logging=info|warning|error|none Set the Python log level. If 'none', tornado won't touch the logging configuration. --db_host DB host --db_pass DB pass --db_port DB port --db_user DB user
You will need to run such a populate.py script at least once, before starting the web app, to populate the db. We suggest you to set a cronjob to spawn it hourly.
15 * * * * /home/release/virtualenv/bin/python /home/release/opinoid/utils/populate.py >> /tmp/populate.log 2>&1
A python script which parses the feeds and populates the db accordingly.
$ ./populate.py 2012-04-06 14:15:02,164 INFO STARTED populate 2012-04-06 14:15:02,165 INFO working on http://www.ilfattoquotidiano.it/feed/ 2012-04-06 14:15:03,396 INFO entries received 20, skipped 6, added 14 [...] 2012-04-06 14:15:16,024 INFO STARTED cleanup 2012-04-06 14:15:16,025 INFO deleting articles older than 1331036102 2012-04-06 14:15:16,029 INFO articles deleted 1, comments 0 2012-04-06 14:15:16,029 INFO STARTED logrun 2012-04-06 14:15:16,029 INFO last full run at 1333714502
It will work with any python2 version providing the following modules:
$ pip freeze feedparser==5.0.1 pep8==0.6.1 pymongo==2.0.1 tldextract==0.3.1 wsgiref==0.1.2
populate.py also provides a —h flag to specify some additional options.
This is the mongodb initialization file. It simply adds a short list of urls (feeds) to be parsed, including their country and category for better categorization. Assuming your mongodb is running on localhost, you can do the initial population with:
$ mongo 127.0.0.1 < init.mongo
Opinoid is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
Opinoid is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License along with Opinoid. If not, see <http://www.gnu.org/licenses/>.
Copyright 2012 Giulio Fidente