Add example how to construct network structure

[jachymb]

Hi! I would like to use bayespy in my diploma thesis. It seems more advanced than other libraries available. On the other hand, I spent most of this day reading the documentation trying to figure out how to make a basic network structure according to a given DAG. The closest thing I found was the _add_child() method on Nodes, which is not a public method. Is it okay to use it, to add arbitrary children? As I understand it now, I can't add arbitrary number of parents/children to a node in it's constructor, the number of parents it limited by the number of distribution parameters. I also found the pomegranate library which also implements Bayesian networks and has nice examples for this: https://github.com/jmschrei/pomegranate/blob/master/tutorials/Tutorial_4_Bayesian_Networks.ipynb https://github.com/jmschrei/pomegranate/blob/master/examples/bayesnet_asia.py but pomegranate currently does not implement gaussian nodes, which I'd like to use. Would it be possible to add similar demos to bayespy? Or am I just missing some theory?

Also, I think I'm not the only person who'd appreciate this: http://stackoverflow.com/questions/28431350/create-bayesian-network-and-learn-parameters-with-python3-x

[jluttine]

Hi, and thanks for the message!

First, I need to emphasize that BayesPy isn't really planned to be used for categorical graphs, because one can use the junction tree algorithm in those cases to perform exact inference. BayesPy focuses on variational Bayes approximation. However, I'm planning to implement support for using the junction tree algorithm for subgraphs (or the whole graph if it is entirely categorical) if I find the time for it.

That said, it is possible to use BayesPy for categorical graphs, but one should understand that the current implementation factorizes the nodes in the posterior approximation. Thus, in general, one will not get the true posterior exactly.

I quickly rewrote this example for BayesPy https://github.com/jmschrei/pomegranate/blob/master/examples/bayesnet_asia.py

# This example could be simplified a little bit by using Bernoulli instead of
# Categorical, but Categorical makes it possible to use more categories than
# just TRUE and FALSE.

import numpy as np

from bayespy.nodes import Categorical, Mixture
from bayespy.inference import VB

# NOTE: Python's built-in booleans don't work nicely for indexing, thus define
# own variables:
FALSE = 0
TRUE = 1

def _or(p_false, p_true):
    """
    Build probability table for OR-operation of two parents

    p_false: Probability table to use if both are FALSE

    p_true: Probability table to use if one or both is TRUE
    """
    return np.take([p_false, p_true], [[FALSE, TRUE], [TRUE, TRUE]], axis=0)

asia = Categorical([0.5, 0.5])

tuberculosis = Mixture(asia, Categorical, [[0.99, 0.01], [0.8, 0.2]])

smoking = Categorical([0.5, 0.5])

lung = Mixture(smoking, Categorical, [[0.98, 0.02], [0.25, 0.75]])

bronchitis = Mixture(smoking, Categorical, [[0.97, 0.03], [0.08, 0.92]])

xray = Mixture(tuberculosis, Mixture, lung, Categorical,
               _or([0.96, 0.04], [0.115, 0.885]))

dyspnea = Mixture(bronchitis, Mixture, tuberculosis, Mixture, lung, Categorical,
                  [_or([0.6, 0.4], [0.18, 0.82]),
                   _or([0.11, 0.89], [0.04, 0.96])])

# Mark observations
tuberculosis.observe(TRUE)
smoking.observe(FALSE)
bronchitis.observe(TRUE) # not a "chance" observation as in the original example

# Run inference
Q = VB(dyspnea, xray, bronchitis, lung, smoking, tuberculosis, asia)
Q.update(repeat=100)

# Show results
print("P(asia):", asia.get_moments()[0][TRUE])
print("P(tuberculosis):", tuberculosis.get_moments()[0][TRUE])
print("P(smoking):", smoking.get_moments()[0][TRUE])
print("P(lung):", lung.get_moments()[0][TRUE])
print("P(bronchitis):", bronchitis.get_moments()[0][TRUE])
print("P(xray):", xray.get_moments()[0][TRUE])
print("P(dyspnea):", dyspnea.get_moments()[0][TRUE])

The result

P(asia): 0.952380952381
P(tuberculosis): 1.0
P(smoking): 0.0
P(lung): 0.02
P(bronchitis): 1.0
P(xray): 0.885
P(dyspnea): 0.96

As I wrote this quickly, there might be errors, but I hope you get the idea. As you probably notice, the syntax for building categorical graphs isn't as elegant as in pomegranate. If I ever implement the junction tree algorithm to BayesPy, I'll probably implement some slightly more convenient syntax for building categorical graphs.

And no, don't use _add_child method. :)

I'll look into that stackoverflow question someday. If you have further questions, please don't hesitate to ask. :)

[jachymb]

Thank you very much, this example is really enlightening. Basically, I was missing the idea of using Mixture distribution to represent conditional probability.

[jluttine]

Note to self: add an example to the documentation.

[jluttine]

Not sure if you're interested anymore, but just wanted to let you know that I have started sketching CategoricalGraph node which would allow exact inference for discrete graphs. I give no promises about when the feature might be ready (if ever) but I really would like to implement it in the near future. If you want to follow the progress of the feature, subscribe to issue #37, and free feel to comment the plan if you have ideas.

[dlevitryon]

Hi jullttine,

Your link: https://github.com/jmschrei/pomegranate/blob/master/examples/bayesnet_asia.py
Does not seem to work anymore, I am following the Asia example, but I seem to use another node in my text book with tuberculosis Or cancer, as the parent stage of xray.

I was wondering if there was a image of the Asia module you used. Main reason why is I am trying to see an example with nodes with two parents.

For example: Parent(Tuberculosis) Parent(Lung Cancer) connected to Child(Tuberculosis or cancer)
For example: Parent(Tuberculosis Or Cancer) Parent(Bronchitis) connected to Child (Dyspnea)

What I been trying is.
TubercolosisORCancer = MultiMixture((Tubercolosis, LungCancer), Categorical, ....)
Dsypnea = MultiMixture((TubercolosisORCancer, Bronchitis), Categorical, ....)