Make rating normalization a strategy

[ghost]

(originally reported in Trac by @elehack on 2011-03-24 17:56:17)

Right now, the only normalization we support in item-item and user-user CF is subtracting baselines. It would be useful to abstract this into a strategy so we can do things like z-score normalization.

Note: This issue has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:24.439719+00:00, last updated: 2013-02-01T22:35:43.098551+00:00

[ghost]

Michael Ekstrand michael@elehack.net on 2011-04-09 19:37:07 said:

In [eb04faf0ece310f9871a7c93b57ab0b1b05a88ec]:

Actually use the normalizations for item-item CF (refs #31)

* Make the item-item CF builder use a normalizer
* Use a baseline to supplement missing values at predict time
* Configure the .js file to use this

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:27.657945+00:00, last updated: None

[ghost]

Michael Ekstrand ekstrand@cs.umn.edu on 2011-04-09 19:37:07 said:

In [c36372d7f34df8e61e03164afe53397ae6ca8850]:

Add a normalized build context to share normalizing (refs #31)

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:27.282269+00:00, last updated: None

[ghost]

Michael Ekstrand ekstrand@cs.umn.edu on 2011-04-09 19:37:06 said:

In [4c41c84d5c81231ecce1bff0d58ccec30e12d2a1]:

Implement baseline-subtracting normalization (refs #31).

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:26.946299+00:00, last updated: None

[ghost]

Michael Ekstrand ekstrand@cs.umn.edu on 2011-04-09 19:37:06 said:

In [6f533b076f0a349de364795e0b8fdf0767f0c0bb]:

Create rating normalization strategy interface (refs #31)

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:26.595471+00:00, last updated: None

[ghost]

ekstrand on 2011-04-08 14:26:39 said:

I've collected a variety of NormalizationMethods to see what we actually need from this feature. From that, it looks like a good idea to separate normalizations and baseline predictors, keeping normalizers opaque. Rating predictors which can use baseline predictors will fill in unpredictable values with the baseline values ''alongside the de-normalized predictions''. The predict phase for e.g. item-item CF would then be as follows:

1. Normalize user rating vector with normalizer
2. Compute normalized predictions
3. De-normalize prediction vector
4. Fill in missing values in prediction vector from the baseline predictor

The normalizer can, of course, operate by adding or subtracting the baseline. This may result in some duplicate computations, as the baseline may need to compute the user average twice, but that can be mitigated by having baselines use WeakHashMap to cache predictions.

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:26.225469+00:00, last updated: None

[ghost]

ekstrand on 2011-04-06 22:09:26 said:

I've started working on this, but moving from baseline prediction to general normalization messes with our ability to identify and deal with coverage issues - the item-item CF engine can know that it doesn't have a baseline and return with less-than-perfect coverage, but how do we do this with general normalization?

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:25.878243+00:00, last updated: None

[ghost]

ekstrand on 2011-05-25 15:52:06 said:

Updating already-fixed tickets to 0.0.3 milestone.

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:25.527846+00:00, last updated: None

[ghost]

ekstrand on 2011-04-26 17:03:59 said:

This is done now. We have normalizers and normalizer builders.

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:25.169217+00:00, last updated: None

[ghost]

Michael Ekstrand michael@elehack.net on 2011-04-09 19:37:08 said:

In [81ce61597be4c214ad07e12f3679777c5d134130]:

Use the normalization framework in SVD (refs #31)

Note: This comment has been automatically migrated from Bitbucket
Created by grouplens on 2013-02-01T21:55:24.778750+00:00, last updated: None