SVD did not converge

[gkb999]

Hi Niclas,

The XMCA worked fine when I used it directly on my raw data.
As it produced results of what I was expecting.
However, I tried to use processed data (like anomalies and detrend)
it gives the following error - SVG didn't converge

Does XMCA only accept raw data or is something wrong with my i/p?
This is how my data looks:

Even the previously worked data was in a similar format.
What could be the issue?

[nicrie]

I cannot say that I exactly know what's happening, but very often the warning you get RuntimeWarning: invalid value encountered in divide self._fields[k] = fields[k] / std[k] is an indicator for a grid point in your data, which has a standard deviation of zero. I encountered this myself when working with sea ice data where you sometimes have a grid point which is 0 or 1 for all time steps. Standard deviation then will be zero, so if you normalize the data, you end up with NaNs after normalizing which is why SVD fails as it cannot handle NaNs.

So one way of dealing with this would be to remove these grid points prior to your analysis, e.g. by the following command:

epsilon = 1e-5
is_valid_gridpoint = sic.stack(x=('lat', 'lon')).std('time') > epsilon
sic = sic.stack(x=('lat', 'lon')).sel(x=is_valid_gridpoint).unstack()

# now use sic for your analysis...

This finds all grid points with a standard deviation below a certain threshold epsilon (because direct comparison with zero is not a good idea for floats) and removes them.

---

That being said in the case of sic it may be better to work with non-normalized data. Since sic has many grid points which barely deviate from 0 or 1 (think about the center of the Arctic, or some very low latitude sea surface), dividing by the standard deviation may artificially blow up these time series.

[gkb999]

Hello Niclas, Thanks a ton for the detailed explanation of the above.
I understand from the package tutors that we can use xMCA with both np.ndarrays and xarrays.

I tried with np.ndarray, and I run into the following error. My i/p is np.ndarray for sure

Why is it?

[nicrie]

That's an easy one ;-) Although you loaded the module for numpy you finally used to module for xarray.
So instead of

pca = xMCA(sic_detrended)

use

pca = MCA(sic_detrended)

The x stands for xarray