rmse gives different result when shapes mismatch

[arvoelke]

Code:

x = np.linspace(0, 1, 100)

print(nengo.utils.numpy.rmse(x, x))
print(nengo.utils.numpy.rmse(x[:, None], x))
print(x[:, None].shape, x.shape)

Output:

0.0
0.412351391454
(100, 1) (100,)

Explanation:
When the shapes match, we get the expected result of 0. When there is an extra axis (making the shape (100, 1) instead of (100,)) we get the result 0.412.... This happens because of numpy's broadcasting rules, which interprets (x[:, None] - x) as a (100, 100) matrix. The RMS of this square matrix then happens to be this weird result.

Why this is a problem:
In many common situations, vectors end up having an additional axis. For instance, if p probes a scalar Nengo ensemble, then sim.data[p].shape == (len(sim.trange()), 1). And usually this is okay, because it is handled naturally/consistently by most operations in numpy/nengo. However, if we then compare this probe to some expected vector, the reported RMSE will be completely incorrect. This has happened to me twice now, and is difficult to diagnose or even recognize. I wouldn't be surprised if this has even gone completely unnoticed before in some of our experiments / tests.

Possible solution:
Raise an error if the shape of the arguments mismatch. The drawback of this approach is that it may break reverse-compatibility, for instance if it is being used like rmse(sim.data[p], 0.5) when we expect p to represent some constant. But since this is not a public method (?), it may be worth it to be more explicit in these special cases, in order to prevent this problem from reoccurring.

Temporary solution:
If noticed, use .squeeze() to remove the additional axis before passing to rmse.