Skip to content

AllProfanity v2.1.0 Trie Based Matching

Choose a tag to compare

View all tags
@ayush-jadaun ayush-jadaun released this 16 Jun 10:53
· 16 commits to main since this release
v2.1.0
34ff9a3

AllProfanity v2.1.0

🚀 Major Release: O(n) TRIE-Powered, Multi-Language Profanity Filtering

AllProfanity v2.1.0 is a complete overhaul that brings blazing-fast detection, multi-language support, advanced leet-speak normalization, and a robust, extensible API. This release is built for modern moderation needs across the globe.

✨ What's New

🏁 TRIE-Based O(n) Profanity Detection

  • Replaces O(n²) set/regex approach with a TRIE (prefix tree)
  • Massive speedup for large texts and dictionaries (up to 50x faster)
  • 75% lower memory consumption

🌐 Multi-Language & Multi-Script Support

  • Built-in dictionaries: English, Hindi, French, German, Spanish, Bengali, Tamil, Telugu
  • Detects profanity in Latin/Roman, Devanagari, and other scripts
  • Easy bulk language loading (e.g., loadIndianLanguages())

🧠 Advanced Leet-Speak & Obfuscation

  • Context-aware mapping for leet-speak and special unicode
  • Detects obfuscated profanities like f#ck, a55hole, etc.

🔒 Security & Reliability

  • Never exposes the full list of loaded profanities (.list() is removed)
  • Proper input validation, error handling, and logging
  • No dependencies except for the wordlists

🎛️ Modern, Flexible API

  • Severity scoring (MILD, MODERATE, SEVERE, EXTREME)
  • Clean by character (clean()) or by word (cleanWithPlaceholder())
  • Detailed detection (detect()) with positions for UI highlighting
  • Add/remove custom words or dictionaries at runtime
  • Whitelist/blacklist support to eliminate false positives
  • Extensive configuration and TypeScript support

📈 Benchmarks

  • Detection time: 10,000-word dictionary, 10,000-word text: 2.3s → 0.047s
  • Almost zero false positives in professional/academic contexts thanks to precise word boundaries and whitelisting

🛠️ Upgrade Guide

  • Remove any use of .list() (now deprecated for security)
  • Use new API methods for language/dictionary management
  • See new detect(), clean(), cleanWithPlaceholder(), and severity scoring in examples below

📝 Example Usage

import profanity from 'allprofanity';

// Simple check
profanity.check('What the f#ck is this?'); // true

// Cleaning
profanity.clean('This is bullshit.'); // "This is ********."
profanity.cleanWithPlaceholder('This is bullshit.'); // "This is ***."

// Multi-language
profanity.loadLanguage('french');
profanity.check('Ce mot est merde.'); // true

// Severity scoring
const result = profanity.detect('This is fucking bullshit and chutiya.');
console.log(result.severity); // 3 (SEVERE)

📚 Changelog

  • Fully TRIE-based engine for O(n) detection
  • Multi-language & script support (8 built-in languages)
  • Robust leet-speak normalization
  • Severity scoring system
  • New cleaning modes and configuration options
  • Secure, extensible, and fully documented API

💬 Discussion & Feedback

I welcome your feedback and contributions!
Open issues for bugs, suggestions, or language packs.
See the process.md for a deep-dive into the engineering decisions and architecture.

🏁 Get Started

See the README for complete documentation, examples, and API details.

Assets 2
Loading