Using classes and inheritance to define models

[kyleabeauchamp]

So the current context manager scheme for defining models seems to be focused on hacking together a quick model and sampling it.

However, with PYMC2, I was able to define models using python classes, leveraging inheritance to allow multiple variants and extensions of the same model. We should probably do two things:

1. Verify that the model definition is still amenable to inheritance / classes
2. Include a class-based model definition in the tutorials somewhere

[jsalvatier]

Do you have an example of what you would like to be able to do Kyle?

[kyleabeauchamp]

I will sit down and think about this at some point. In the meantime, here are three examples of my and my colleagues work that will eventually use inheritance to help define classes of related models. I'm afraid the first of these three was written long before I knew what I was doing, while the latter two are still works in progress.

https://github.com/kyleabeauchamp/FitEnsemble/blob/master/fitensemble/belt.py
https://github.com/choderalab/bayesian-itc/blob/master/bitc/models.py
https://github.com/choderalab/gbff/blob/master/model.py

[twiecki]

I just came up with this. The main motivation is actually speed. Compiling a model often takes way longer than actually running. Here the model is created once and cached. The inputs are turned into shared variables that can be updated in-place:

class BayesianModel(object):
    def __init__(self, cache_model=True):
        self.cached_model = None
        self.shared_vars = {}

    def cache_model(self, **inputs):
        self.shared_vars = self._create_shared_vars(**inputs)
        self.cached_model = self.create_model(**self.shared_vars)

    def create_model(self, **inputs):
        raise NotImplementedError('This method has to be overwritten.')

    def _create_shared_vars(self, **inputs):
        shared_vars = {}
        for name, data in inputs.items():
            shared_vars[name] = shared(data, name=name)
        return shared_vars

    def run(self, **inputs):
        if self.cached_model is None:
            self.cache_model(**inputs)

        for name, data in inputs.items():
            self.shared_vars[name].set_value(data)

        trace = self._inference()
        return trace

    def _inference(self, samples=5000):
        with self.cached_model:
            start = pm.find_MAP(fmin=sp.optimize.fmin_powell)
            step = pm.NUTS(scaling=start)
            trace = pm.sample(samples, step, start=start)

        return trace


class ReturnsNormal(BayesianModel):
    def create_model(self, data=None):
        with pm.Model() as model:
            mu = pm.Normal('mean returns', mu=0, sd=.01, testval=data.mean())
            sigma = pm.HalfCauchy('volatility', beta=1, testval=data.std())
            returns = pm.Normal('returns', mu=mu, sd=sigma, observed=data)
            pm.Deterministic(
                'annual volatility',
                returns.distribution.variance**.5 *
                np.sqrt(252))
            pm.Deterministic(
                'sharpe',
                returns.distribution.mean /
                returns.distribution.variance**.5 *
                np.sqrt(252))

        return model

returns_normal = ReturnsNormal()

[fonnesbeck]

I agree that having a more structured way to build models would be helpful, particularly where a large number of similar models are required. This used to be the default method for building models in the PyMC 1 days, in fact. For most people in most situations, an OO approach would be over-engineering, though.

I like what @twiecki has done above, and wonder if you could even have the model created automatically when the class is instantiated, and have all the methods decorated with the model context, say, using a metaclass?

[twiecki]

The reason I didn't create the model on initialization is that this is for a library and would severely prolong import time.

[twiecki]

Adressed by #1525.