Skip to content
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

hatena-haiku-spam-filter (はてなハイクスパムフィルタ)

Hatena Haiku has a lot of spam, but I believe that, with some work and some custom rules, SpamAssassin can be retooled to detect the spam with a high level of accuracy.

Requires spamassassin (yum install spamassassin) and PHP (for now)

Along with SpamAssassin classification, I also believe that individual users and their history can also be further factored into the classification (outside of SpamAssassin).

This may take some data gathering over time, but will be well-worth the effort.

Interactive post classifier

Bayesian filter

Train it with ./

user_lists folder

Inside this folder we will have two text files in which we can place a list of Hatena IDs to manually whitelist or blacklist them.


This will be integrated into both the Interactive Classifier, and soon, into a utility that builds the user_prefs file from a template. This will allow us to generate the file without manually going in and editing it. In this case, we can use whitelist_from and blacklist_from.

Local spamd

NOTE: Currently being run under a non-privileged user on a CentOS 7 box.

Start in a screen session with ./

We have a custom formatted which I will add to the repository soon


  • Redesign and possibly merge Unclassified (misleading name, as it is the stuff that has been automatically classified) with the manually classified items database
    • We want to be able to supply immediate data no matter whether I have gone through the recent entries or not to classify them)
  • Build a database of users by Hatena ID and store a history of their processed entries and associated scores to help avoid one-off filter misses
  • Possibly move away from SQLite or make it work with both MySQL and SQLite.


Filtering spam from Hatena Haiku with SpamAssassin




No releases published


No packages published
You can’t perform that action at this time.