Caching hashtags in Wikipedia edit summaries
Python
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
README.md
TODO.md
crontab
dal.py
log.py
schema.sql
tags.py
update.py
utils.py

README.md

Wikipedia Hashtags

A set of scripts to cache Wikipedia Hashtag.

This script connects to the Wikipedia database replica on Wikimedia Tool Labs, searches for hashtags, and then stores it in another database with nice indexes for searching. See hashtag search. On Tool Labs, the database is kept up-to-date on 10-20 minute interval scheduled via crontab.

Usage

$ python update.py --lang <two letter language code>

Options:

$ python update.py --help

  -h, --help     show this help message and exit
  --lang LANG
  --hours HOURS
  --debug

TODOs

  • Add support for other Wikimedia projects
  • Add additional lanugages

Notes

Connect to the hashtag database on Tool Labs:

mysql --defaults-file=${HOME}/replica.my.cnf -h s1.labsdb s52467__hashtags