A few structures for doing NLP analysis / experiments.
A map-like data structure for representing discrete probability distributions. Contains an underlying map of event -> probability along with a probability for all other events. Supports some element-wise mathematical operations with other counter.Counter objects.
// Create a counter with 0 probability for unknown events (and with "" // corresponding to the unknown event) balls := counter.New(0.0) // Add some observations balls.Incr("blue") balls.Incr("blue") balls.Incr("red") // Normalize into a discrete distribution balls.Normalize() // blue => 0.666666 balls.Get("blue") // purple => 0.0 balls.Get("purple") preference = counter.New(0.0) preference.Set("red", 2.0) preference.Set("blue", 1.0) preference.Normalize() expected_with_preference = counter.Multiply(balls, preference) expected_with_preference.Normalize() // blue => 0.5 expected_with_preference.Get("blue") // red => 0.5 expected_with_preference.Get("red") // You can also use log probabilities balls.LogNormalize() preferences.LogNormalize() // And do in-place operations balls.Add(preferences) // Log-normalize expects counters with positive counts, so // exponentiate-then-normalize balls.Exp() balls.LogNormalize() // blue => -1 (== lg(0.5)) balls.Get("blue")
Similar to counter.Counters, but with a fixed set of keys and no default value. Represented under the hood as an array of doubles (with order fixed according to the set of keys). Supports element-wise math operations with other frozencounter.Counters that share the same set of keys. Some mathematical operations are accelerated by the BLAS library.
fBalls := frozencounter.Freeze(balls) fPrefs := frozencounter.Freeze(preference) fExpectedWithPreference := frozencounter.Multiply(fBalls, fPrefs)