matrixfit with 'lm' and 'optim' have different chi^2

[kostrzewa]

I've noticed that for some reason matrixfit, when called with 'lm' seems to produce a tiny chi^2 compared to the same run with 'optim'. In the case that I tested, I do not believe the chi^2 given by the 'lm' run. Has anybody observed this before?

[kostrzewa]

Just to quantify things: optim gives a reduced chi^2 of around 1 for the case at hand while lm gives the typical kind of chi^2 that one would expected from a fit without a covariance matrix. useCov = TRUE is passed...

[kostrzewa]

Note that in the optim case, the resulting chi^2 compares very well to the chi^2 of the corresponding effective mass fit with covariance.

[urbach]

Note that in the ```optim``` case, the resulting chi^2 compares very

well to the chi^2 of the corresponding effective mass fit with covariance. I have observed such an effect from time to time, but usually the other way around. But, I think, this is worth exploring since a lot in hadron changed.

[j-ostmeyer]

I tried this out for several examples without covariance and there optim and lm agree exactly. So I guess this is some stability issue with covariance.

[kostrzewa]

Would it be too much effort (or impossible) to try this with covariance on the data that you tested with?

[j-ostmeyer]

I tested with real data, for which I don't have the covariance. Of course I could simply use a random covarinace matrix. Do you think this is worth trying?

[kostrzewa]

If you have real data you can estimate a covariance matrix, what am I missing?

[kostrzewa]

Or do you just have central values and errors?

[j-ostmeyer]

Or do you just have central values and errors?

Yes, I assume fully uncorrelated data.

[j-ostmeyer]

@pittlerf mentioned a case where gnuplot gives a better chi^2 than our fit routine. Maybe this example would help?

[pittlerf]

I put an example R script with the necessary data to /qbigwork2/pittler/fit_check.

With gnuplot I obtain:
final sum of squares of residuals : 0.00424951
And for the parameters:
Final set of parameters Asymptotic Standard Error
======================= ==========================
a = 1.70752 +/- 0.002055 (0.1204%)
b = 0.173368 +/- 6.264e-05 (0.03613%)

With matrixfit using either optim or lm I got:
mass 0.17336742
amplitude 1.84797631
chisq2 0.00557006

The mass is perfectly fine, but the amplitude is a bit different and the chi2 is larger.

[pittlerf]

I was using a PS correlator from the A60 ensemble.

[urbach]

Are you using the exact same fit range? What does the hadron chi^2 function give with gnuplot parameters?

…

On 1 October 2019 17:37:09 CEST, pittlerf ***@***.***> wrote: I was using a PS correlator from the A60 ensemble.

-- Carsten Urbach mobile

[j-ostmeyer]

I cannot load the file "check_fitting.RData".

file ‘check_fitting.RData’ has magic number 'RDX3'
Use of save versions prior to 2 is deprecated

[pittlerf]

Do you try this on qbig ?

[urbach]

Rdata files are architecture independent...

…

On 4 October 2019 12:29:30 CEST, pittlerf ***@***.***> wrote: Do you try this on qbig ?

-- Carsten Urbach mobile

[pittlerf]

I am just asking, because I can read it on qbig. It contains a 'data' dataframe, from which you can extract the PS correlator with

ppcorrelator <- extract.obs(data, vec.obs=c(1))

[pittlerf]

I checked this, and found out that difference in the amplitude was due to the different normalization:

matrixChisqr(par=c(0.1733674,1.84797631),t=seq(16,20),y=bspp$cf0[17:21],M=M,T=48,matrix(c(2,2),nrow=1,ncol=2),sign.vec=1,ov.sign.vec=1)=0.005570092

Using the results from gnuplot I indeed observe larger chi^2:

matrixChisqr(par=c(0.1733674,sqrt(2*1.70752)),t=seq(16,20),y=bspp$cf0[17:21],M=M,T=48,matrix(c(2,2),nrow=1,ncol=2),sign.vec=1,ov.sign.vec=1)=0.005579476

I think my issue is solved then, the difference that I saw, came only from the different parametrization.

[urbach]

But why is the gnuplot chi^2 smaller?

…

On 4 October 2019 15:16:55 CEST, pittlerf ***@***.***> wrote: I checked this, and found out that difference in the amplitude was due to the different normalization: ``` matrixChisqr(par=c(0.1733674,1.84797631),t=seq(16,20),y=bspp$cf0[17:21],M=M,T=48,matrix(c(2,2),nrow=1,ncol=2),sign.vec=1,ov.sign.vec=1)=0.005570092 ``` Using the results from gnuplot I indeed observe larger chi^2: ``` matrixChisqr(par=c(0.1733674,sqrt(2*1.70752)),t=seq(16,20),y=bspp$cf0[17:21],M=M,T=48,matrix(c(2,2),nrow=1,ncol=2),sign.vec=1,ov.sign.vec=1)=0.005579476 ``` I think my issue is solved then, the difference that I saw, came only from the different parametrization.

-- Carsten Urbach mobile

[pittlerf]

Basically because I just fitted the output of print to the correlation function, when I extend the number of digits in the print, and do the fit once more, I got the same chi^2 (0.00557007) with gnuplot as well.