Skip to content

Conversation

@paul-buerkner
Copy link
Contributor

This PR further improves the diagnostics module:

  • Introduces a basic VariableArray class to carry around variable_keys and variable_names with an array containing posterior draws. This class is only meant for internal use for now and we can see if extending it in the future could be sensible.

  • Further improves posterior pairs plots to enable plotting the prior again.

For example, using the linear regression notebook, we can now do something like:

bf.diagnostics.plots.pairs_posterior(
    estimates=post_draws, 
    targets=val_sims,
    priors=sim_draws,
    dataset_id=0,
    variable_names=par_names,
)
plt.show()

to show both posterior, prior, and the true parameter values. The previous implementation didn't quite work in terms of correctly showing the prior.

There is one issue which I couldn't quite resolve and I would like to get your input @stefanradev93 and @jerrymhuang: Currently, the posterior is always plotted behind the prior on the diagonal. I tried to change that via hue_order for example but it either breaks the off diagonal plots or mislabels prior and posterior. It kind of looks like a bug in seaborn to me as I cannot imagine this to be intended. Any ideas would be highly appreciated.

@codecov-commenter
Copy link

codecov-commenter commented Feb 18, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 6.66667% with 70 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
bayesflow/diagnostics/plots/pairs_samples.py 0.00% 30 Missing ⚠️
bayesflow/utils/dict_utils.py 16.00% 21 Missing ⚠️
bayesflow/diagnostics/plots/pairs_posterior.py 11.11% 8 Missing ⚠️
bayesflow/utils/plot_utils.py 0.00% 5 Missing ⚠️
...sflow/diagnostics/metrics/posterior_contraction.py 0.00% 2 Missing ⚠️
...low/diagnostics/metrics/root_mean_squared_error.py 0.00% 2 Missing ⚠️
bayesflow/diagnostics/metrics/calibration_error.py 0.00% 1 Missing ⚠️
bayesflow/diagnostics/plots/calibration_ecdf.py 0.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Files with missing lines Coverage Δ
bayesflow/diagnostics/metrics/calibration_error.py 21.05% <0.00%> (-1.17%) ⬇️
bayesflow/diagnostics/plots/calibration_ecdf.py 14.89% <0.00%> (ø)
...sflow/diagnostics/metrics/posterior_contraction.py 36.36% <0.00%> (-3.64%) ⬇️
...low/diagnostics/metrics/root_mean_squared_error.py 30.76% <0.00%> (-2.57%) ⬇️
bayesflow/utils/plot_utils.py 19.75% <0.00%> (-1.03%) ⬇️
bayesflow/diagnostics/plots/pairs_posterior.py 37.50% <11.11%> (+5.50%) ⬆️
bayesflow/utils/dict_utils.py 21.25% <16.00%> (-1.39%) ⬇️
bayesflow/diagnostics/plots/pairs_samples.py 16.32% <0.00%> (-5.90%) ⬇️

@paul-buerkner
Copy link
Contributor Author

I think I now fixed also the histogram issue to the degree that seaborn allows. I think this PR is ready to merge once @stefanradev93 approves.

@stefanradev93 stefanradev93 merged commit e46b962 into dev Feb 19, 2025
13 checks passed
@stefanradev93 stefanradev93 deleted the improve-diag branch February 19, 2025 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants