Add an example of a gold_standard samples object

[MansMeg]

I would be nice to have a model with actual posterior samples already present so that we can judge the structure of this part as well

[eerolinna]

My original idea was that for the posterior 8_schools|centered we would store a bayesbench result object in gold_standards/8_schools|centered.zip (or json instead of zip)

That makes the posterior database coupled to bayesbench though, so if bayesbench changes the output format then every gold standard would need to be updated. Maybe there is some better way that doesn't couple the posterior database to bayesbench.

In any case, the gold standard needs to include

• Posterior samples
• Which posterior produced the samples (it doesn't need to be the same one, in this example it could also be 8_schools|noncentered)
• Diagnostic values that were used to determine that the posterior is good enough
• Which inference engine was used (I'm not sure yet if this is 100% mandatory)

[MansMeg]

Yes. We want to have a bayesbenchr object as gold standard. But we should not call it bayesbenchr object and it should not be coupled. But the structure should be the same. I think our structure was good.

[eerolinna]

What would be the workflow for adding gold standards to posterior database, something like this?

Lets say we want to add gold standard for 8_schools|noncentered

1. Run bayesbench on 8_schools|noncentered using some inference method we believe to produce correct results
2. Check that the result is good (using something like checking diagnostic values)
3. Copy the output file to posterior_database/gold_standards and rename it to be called 8_schools|noncentered.zip
4. Add gold_standards/8_schools|noncentered.zip to the posterior info file under the key gold_standard

This workflow means that posterior database is coupled to bayesbench (as we rely bayesbench to return the exact format that posterior database uses). If we want to avoid it here's a few possible solutions:

1. Add step 5: run ./bayesbench_to_posterior_db_gold_standard.R 8_schools|noncentered.zip that modifies the file to a format that is independent of bayesbench. Then if bayesbench changes output format we would have another script like /bayesbench_v2_to_posterior_db_gold_standard.R
2. Add version field to bayesbench output. The gold standards directory is allowed to have gold standards in different versions and when the user requests a gold standard we have code to change every version to a common format
3. Just store the bayesbench output as is. When bayesbench changes the output format write a script to convert each stored gold standard to the new format. If this conversion is impossible (the new format includes something that is not present in the old format) then just run inference again to obtain a result in the new format (and check that it satisfies the diagnostics)
4. Never change the output format of bayesbench (probably impossible)

[MansMeg]

Now added and fixed