mu_shoes_error_diag in Pantheon+SH0ES data, help explan the warning?

[mikehelland]

An important part of advancing an alternative cosmology is how it fits the data. And the Pantheon+SH0ES data is the current best set of measurements of redshift and distance modulus, or μ (mu).

https://github.com/PantheonPlusSH0ES/DataRelease/tree/main/Pantheon%2B_Data/4_DISTANCES_AND_COVAR

Finding a best fit to this data can be done using the sum of squared errors (SSE):

In that formula is the error, σ, and I would like to use them in the calculations. The dataset has:

MU_SH0ES_ERR_DIAG - Uncertainty on MU_SH0ES as determined from the diagonal of the covariance matrix. WARNING, DO NOT FIT COSMOLOGICAL PARAMETERS WITH THESE UNCERTAINTIES. YOU MUST USE THE FULL COVARIANCE. THIS IS ONLY FOR PLOTTING/VISUAL PURPOSES

I've been making little progress on understanding exactly what that warning is telling me. Can anyone shed some light on it?

When I use that value as σ in my SSE calculations, the best fit for LCDM 0.7,0.3 pulls ahead. I suspect that this value in the dataset is fine tuned for that cosmological model. And that's what the warning says.

Is that right? If so, how to get around that? I'm not entirely sure what the "full covariance" is or how to use it.

Working this out might be helpful in investigating alternatives to LCDM for others like me, with a keen interest, but not fully qualified to know exactly what all the data means or how to use it.

[RedshiftDrift]

MU_SH0ES_ERR_DIAG - Uncertainty on MU_SH0ES as determined from the diagonal of the covariance matrix

This variable refers to the diagonal elements of the covariance which are uncertainties resulting from the sum of all uncertainties used in the calculation. But that doesn't include correlations between uncertainties from different measurements, you have to use the covariance matrix if you want to include the possible correlations between measurements.

Riess et al. have kept track of all the data processing done in every single database they use, have read the papers and figured out the common sources of errors in different measurements. Over the years they've built up a correlation matrix of the variance ("the full covariance") that you must use to fit data. They often show the matrix in a graphical format that looks like this $\downarrow$

There is a massive amount of work in this... a lot for one person.

In any case $\Omega_\Lambda = 0.7$ and $\Omega_m = 0.3$ is good enough for a model that's wrong anyway, why do you care?

[mikehelland]

In any case and is good enough for a model that's wrong anyway, why do you care?

Well... diligence I suppose. If we're going to be looking at models, comparing them to the data seems important. And the Pantheon data has this "WARNING, DO NOT FIT COSMOLOGICAL PARAMETERS WITH THESE UNCERTAINTIES."

So... what if I want to test the best fit to a different FLRW, or a non-FLRW model in my case.

I want to compare LCDM and my luminosity distance equation side by side against the Pantheon+SH0ES data.

I thought that the SSE described above would be a good way to do that.

So here's some Javascript that downloads the data and parses it:

var snData = []
var url = "https://raw.githubusercontent.com/PantheonPlusSH0ES/DataRelease/main/Pantheon%2B_Data/4_DISTANCES_AND_COVAR/Pantheon%2BSH0ES.dat"
fetch(url).then(res => res.text()).then(text => {
    var lines = text.split("\n")
    for (var i = 1; i < lines.length - 1; i++) {
        var line = lines[i].split(" ")
        snData.push({
            z: parseFloat(line[2]),
            mu: parseFloat(line[10]),
            muerr: parseFloat(line[11]),
        })
    }
})

Now that you have the SN data loaded, pick a range for Hubble's constant (say 60 to 80) and calculate the sum of squared errors:

var lowest
for (var H = 70; H < 75; H+=0.1) {
    var sse = calculateSSE(H)

    if (!lowest || lowest.sse > sse) {
        lowest = {H, sse}
    }
}
console.log(lowest)


function calculateSSE(H0) {
    var d, mu
    var err = 0

    snData.forEach(sn => {
        d = getModelLuminosityDistance(sn.z)

        mu = 5 * Math.log10(d) + 25

        err += Math.pow(sn.mu - mu, 2)
    })
    return err
} 

All you have to do is supply getModelLuminosityDistance(z).

So I can use the equation for my hypothesis, and I can use LCDM, and compare the results.

So my hypothesis is a better fit, but a nice amount. But.... this isn't using the error margin from the SSE equation. It's just:

err += Math.pow(sn.mu - mu, 2)

It should be:

err += Math.pow(sn.mu - mu, 2) / Math.pow(sn.muerr, 2)

When I do that, I get this.

And LCDM just inches out the hypothesis. That's why I was wondering, the data has that warning to not use it to fit cosmological parameters. I'm wondering if that's because MU_SHOES_ERR_DIAG is fine tuned for that model. In which case it might not make much sense to include it in my calculations. In which case, I'd like to use the right value for the error margins in the calculations.

[RedshiftDrift]

If we're going to be looking at models, comparing them to the data seems important.

I do support this statement. This was the consensus reached at the Second Crisis in Cosmology Conference. However data is always pre-processed by astronomers before publications in tables, and LCDM assumptions are used to derive the numbers.

So, while it is perfectly fine to use databases to test the LCDM model, LCDM assumptions might have to be removed from the database before attempting a fit to any other cosmological model.

Specifically for the fit you describe above, redshift data is obtained directly from spectra and not corrected, so it can be used safely. However, SNe luminosity $\mu$ is corrected for all sorts of effects (dust absorption, evolution of the host galaxy, calculated absolute luminosity, etc.) based on LCDM models.

To get an accurate fit to your model you would need to redo the data analysis from scratch... Professional astronomer Martín López-Corredoira has done such analyses (see for example this paper and this one were the distance $(1+z)ln(1+z)$ is examined) but he also uses data processed with LDCM assumptions. The logarithmic model $(1+z)ln(1+z)$ doesn't do too badly, but López-Corredoira's conclusions are still biased by the LCDM assumptions that remain in the source data.

A correct re-analysis would require the development of new analysis tools that would replace decades of work in astronomy that produced data tainted with LCDM assumptions - a task that is too big for a single person.

[mikehelland]

Maybe one or two motivated people and a sufficiently trained AI/ChatGPT....

[bgreenman]

You have to look for someone who dosen't use measurements based on LCDM