Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support PyMC 5.13 and fix bayeux related issues #803

Merged
merged 4 commits into from
Apr 11, 2024

Conversation

tomicapretto
Copy link
Collaborator

@tomicapretto tomicapretto commented Apr 9, 2024

  • Modify PyMCModel._clean_results() to
    1. Compute deterministics using the new pm.compute_deterministics() function when a bayeux based sampler is used.
    2. Be more robust at renaming dimension names when a bayeux based sampler is used.
    3. Drop unused dimensions before transposing.
  • Use pt.specify_broadcastable with group-specific effects when the expression side is a factor variable of two levels. It's a very specific case, but we support it and it was failing with PyMC 5.13 as all coords started to be mutable with this PyMC version.

This PR closes #800


Why I modified how we rename dims with bayeux. See the following example:

from io import StringIO

import requests

import arviz as az
import bambi as bmb
import pandas as pd

url = "https://raw.githubusercontent.com/crnolan/pyrba/main/data.txt"  # replace with your url
response = requests.get(url)
df = pd.read_table(StringIO(response.text), delimiter=r"\s+")

results = model.fit(
    num_warmup=250, num_samples=200, num_chains=2, inference_method="numpyro_nuts"
)

With the old implementation, it would have failed. Look at

bambi/bambi/backend/pymc.py

Lines 275 to 281 in 5c00663

if idata_from == "bayeux":
pymc_model_dims = [dim for dim in dims_original if "_obs" not in dim]
bayeux_dims = [
dim for dim in idata.posterior.dims if not dim.startswith(("chain", "draw"))
]
cleaned_dims = dict(zip(bayeux_dims, pymc_model_dims))
idata = idata.rename(cleaned_dims)

what happened in that case was that pymc_model_dims and bayeux_dims where of different lengths and thus the dictionary created was mixing things.

This new approach

        dims_original = list(self.model.coords)

        # Identify bayeux idata and rename dims and coordinates to match PyMC model
        if idata_from == "bayeux":
            cleaned_dims = {
                f"{dim}_0": dim
                for dim in dims_original
                if not dim.endswith("_obs") and f"{dim}_0" in idata.posterior.dims
            }

exploits the fact that we know we can only rename dims passed to the model, they can't finish with _obs and they have to be in the posterior. And most importantly, the "bad" bayeux names are like the original names with an extra _0 (e.g. party_id_dim original and party_id_dim_0 as result in bayeux).

@tomicapretto tomicapretto changed the title [WIP] Support PyMC 5.13 and fix bayeux related issues Support PyMC 5.13 and fix bayeux related issues Apr 9, 2024
@tomicapretto tomicapretto marked this pull request as ready for review April 9, 2024 19:56
Copy link
Collaborator

@GStechschulte GStechschulte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this up. Everything looks good to me.

@tomicapretto tomicapretto merged commit 793be6a into bambinos:main Apr 11, 2024
@tomicapretto tomicapretto deleted the support_pymc_5_13 branch April 11, 2024 16:54
GStechschulte pushed a commit to GStechschulte/bambi that referenced this pull request Apr 14, 2024
GStechschulte added a commit that referenced this pull request Apr 15, 2024
* add inference_methods class to obtain names of methods and kwargs

* re-run notebook

* update notebook to include new methods

* convienent methods for getting inference names and kwargs

* Fix `get_model_covariates()` utility function (#801)

* Support PyMC 5.13 and fix bayeux related issues (#803)

* run black to fix formatting

* add test to check for inference method names

* test get_kwargs method of InferenceMethods class

---------

Co-authored-by: Tomás Capretto <tomicapretto@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bambi dev version with bayeux give wrong posterior dims for hierarchical model (mixed and dropped dimensions)
2 participants