v1.7.0 - Document Count Normalization

https://luarocks.org/modules/leafo/lapis-bayes

Added

• doc_count column on categories, the number of documents trained into the category. Requires a migration, see below.
• normalize_doc_count option for BayesClassifier: compares per-document word rates with a uniform prior, so an unbalanced training set doesn't bias every shared word toward the larger category. Requires doc_count on both categories.

Changed

• increment_words and decrement_words take an optional opts.doc_count (default 1) and update the category's doc_count.
• Categories:increment takes an optional second argument for incrementing doc_count.

Upgrading

Add a migration that calls run_migrations again, then run lapis migrate:

[1792300000] = require("lapis.bayes.schema").run_migrations,

Existing categories will have a doc_count of 0. Retrain or backfill them before using normalize_doc_count.

Full Changelog: v1.6.0...v1.7.0