Bayesian networks

Bayesian network is a probabilistic graphical model (a type of statistical model) that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG) (Wiki

• Overview
  • Introducing Bayesian Networks (2004) - free chapter from the Bayesian Artificial Intelligence book Kevin B. Korb, Ann E. Nicholson
  • A Tutorial on Learning With Bayesian Networks (2020) David Heckerman

Types

• Discrete Bayesian Networks
  • Distributions are assumed to be multinomial, represented by tables
• Gaussian Bayesian Networks
  • Distributions are normal
  • Learning Gaussian Networks (1994) Dan Geiger, David Heckerman
• Mixed
  • Bayesian networks with variables that have different distributions
  • Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative (1989) S. L. Lauritzen, N. Wermuth
  • Copula Bayesian Networks (2010) Gal Elidan
• Dynamic Bayesian Networks
  • Relates variables to each other over some time steps
  • Dynamic Network Models for Forecasting (1992) Paul Dagum, Adam Galper, Eric Horvitz
  • Dynamic Bayesian Networks: Representation, Inference and Learning (2002) Kevin Patrick Murphy

Structure learning

Learning the graph structure that represents the conditional independencies between variables. Main approaches are constraint-based (conditional independence tests) and score-based (goodness-of-fit scores)

• Inductive Causation
  • Equivalence and Synthesis of Causal Models (1990) TS Verma, Judea Pearl
• Sparse Candidate
  • Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm (1999) Nir Friedman, Iftach Nachman, Dana Peer
• Greedy Search
  • Optimal Structure Identification With Greedy Search (2002) David M. Chickering
  • Learning Bayesian Networks with Thousands of Variables (2015) Mauro Scanagatta, Cassio P. de Campos, Giorgio Corani, Marco Zaffalon
  • Learning Bayesian networks from big data with greedy search:computational complexity and efficient implementation (2019) Marco Scutari, Claudia Vitolo, Allan Tucker
• Grow-Shrink
  • Learning Bayesian Network Model Structure from Data (Ph.D. thesis, 2003) Dimitris Margaritis
• Incremental Association
  • Algorithms for Large Scale Markov Blanket Discovery (2003) Ioannis Tsamardinos, Constantin F. Aliferis, Alexander Statnikov
  • Interleaved Incremental Association
  • Fast Incremental Association
    Speculative Markov BlanketDiscoveryforOptimalFeature Selection (2005) Sandeep Yaramakala, Dimitris Margaritis
• Optimal Reinsertion
  • Optimal Reinsertion: A new search operator for accelerated and more accurate Bayesian network structure learning (2003) Andrew Moore, Weng-Keen Wong
• Max-Min Parents and Children
  • The max-min hill-climbing Bayesian network structure learning algorithm (2006) Ioannis Tsamardinos, Laura E. Brown, Constantin F. Aliferis
• Other
  • Learning Bayesian Networks: The Combination of Knowledge and Statistical Data (1995) D. Heckerman, D.Geiger, D. M. Chickering

Parameter learning

Estimation of the parameters of the global distribution with known graph structure.

• Learning Bayesian network parameters under incomplete data with domain knowledge (2009) Wenhui Liaoa, Qiang Ji
• MLE Maximum Likelihood Estimate
• Bayesian method
• EM Expectation-maximization
• RBE Robust Bayesian Estimate
• Monte-Carlo Method
• Gaussian approximation

Packages

• R
  • Package: abn (CRAN, Paper)
  • Package: bnlearn (Homepage, Paper)
  • Package: deal (CRAN, Paper) - supports learning from mixed data
  • Package: pcalg (CRAN)
• Python
  • Package: BNfinder (PyPI)
• Julia
  • Package: BayesNets.jl (Code)