Stopwords

[emilorol]

Thank you for sharing your work.

How can I implement stopwords?

[noogen]

You can pass in your own tokenizer function in the constructor. Something like.

// array containing stopwords
$stopwords = array("der", "die", "das", "the");

// escape the stopword array and implode with pipe
$s = '~^\W*('.implode("|", array_map("preg_quote", $stopwords)).')\W+\b|\b\W+(?1)\W*$~i';

$options['tokenizer'] = function($text)  use ($s) {
            // convert everything to lowercase
            $text = mb_strtolower($text);

            // remove stop words
            $text = preg_replace($s, '', $text);

            // split the words
            preg_match_all('/[[:alpha:]]+/u', $text, $matches);
            // first match list of words
            return $matches[0];
        };

$classifier = new \niiknow\Bayes($options);

[emilorol]

Perfect. Thank you