Fu-fu: The Profanity Filter for Rails. This fork is about: - adding feature to ban all words starting with a prefix (like: abc*), - adding list of Polish profanity words, - adding configuration option to choose one or many dictionaries of profanities.
Ruby
Pull request Compare This branch is 11 commits ahead, 12 commits behind adambair:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
config
lib
test
.gitignore
MIT-LICENSE
README.rdoc
Rakefile
init.rb

README.rdoc

Fu-fu: The Profanity Filter for Rails

This plugin will allow you to filter profanity using basic replacement or a dictionary term.

Reasons for this fork

This fork is supposed to add a few features I needed:

  • ban all words starting with a prefix (like: abc*), [ready]

  • list of Polish profanity words, [ready]

  • configuration option to choose one or many dictionaries of profanities, [planned]

  • ban all words containing some root (like: *abc*) [maybe someday]

Disclaimer

This plugin is provided as is - therefore, the creators and contributors of this plugin are not responsible for any damages that may result from it's usage. Use at your own risk; backup your data.

Install

./script/plugin install git://github.com/szeryf/fu-fu.git

Example

You can use it in your models:

Notice – there are two profanity filters, one is destructive. Beware the exclamation point (profanity_filter!).

Non-Destructive (filters content when called, original text remains in the database)

profanity_filter :foo, :bar
  # banned words will be replaced with @#$%
profanity_filter :foo, :bar, :method => 'dictionary'
  # banned words will be replaced by value in config/dictionary.yml
profanity_filter :foo, :bar, :method => 'vowels'
  # banned words will have their vowels replaced
profanity_filter :foo, :bar, :method => 'hollow'
  # all letters except the first and last will be replaced

The non-destructive profanity_filter provides different versions of the filtered attribute:
  some_model.foo => 'filtered version'
  some_model.foo_original => 'non-filtered version'

Destructive (saves the filtered content to the database)

profanity_filter! :foo, :bar
  # banned words will be replaced with @#$%
profanity_filter! :foo, :bar, :method => 'dictionary'
  # banned words will be replaced by value in config/dictionary.yml
profanity_filter! :foo, :bar, :method => 'vowels'
  # banned words will have their vowels replaced
profanity_filter! :foo, :bar, :method => 'hollow'
  # all letters except the first and last will be replaced

You can also use the filter directly:

ProfanityFilter::Base.clean(text)
ProfanityFilter::Base.clean(text, 'dictionary')
ProfanityFilter::Base.clean(text, 'vowels')
ProfanityFilter::Base.clean(text, 'hollow')

Prefix matching

If the dictionary contains words ending with asterisk *, all words starting with this prefix will be replaced. So, if you add something like this:

abc*: a...

all words starting with abc (like abc, abcdef, abcxyz) will be considered banned and replaced by the filter.

Benchmarks

Inquiring minds can checkout the simple benchmarks I've included so you can have an idea of what kind of performance to expect. I've included some quick scenarios including strings of (100, 1000, 5000, 1000) words and dictionaries of (100, 1000, 5000, 25000, 50000, 100000) words.

You can run the benchmarks via:

ruby test/benchmark/fu-fu_benchmark.rb

Please note – the prefix matching feature introduced by me makes the benchmarks run about 2.7 times slower (on my computer it's original: ~49s vs mine: ~2m15s), so if really care about performance you might want to stick with the original. The individual operation on a 50-words string is still just a couple of milliseconds, of course (on my computer it's original: ~1ms vs mine: ~2-3ms). Prefix matching could (and probably will) be optimized a little bit.

TODO

  • Turn this into a gem (and move over to rspec)

  • May break ProfanityFilter out on it's own

  • Clean up dictionary implementation and subsitution (suboptimal and messy)

  • Move benchmarks into a rake task

License

Fu-fu: The Profanity Filter for Rails uses the MIT License. Please see the MIT-LICENSE file.

Resources

Created by Adam Bair (adam@intridea.com) of Intridea (www.intridea.com) in the open source room at RailsConf 2008.