LINFA

LINFA is a library for variational inference with normalizing flow and adaptive annealing. It is designed to accommodate computationally expensive models and difficult-to-sample posterior distributions with dependent parameters.

The code for the masked autoencoders for density estimation (MADE), masked autoregressive flow (MAF) and real non volume-preserving transformation (RealNVP) is based on the implementation provided by Kamen Bliznashki.

Installation

To install LINFA type

pip install linfa-vi

Documentation

The documentation can be found on readthedocs

References

Background theory and examples for LINFA are discussed in the two papers:

• Y. Wang, F. Liu and D.E. Schiavazzi, Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models
• E.R. Cobian, J.D. Hauenstein, F. Liu and D.E. Schiavazzi, AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation

Requirements

• PyTorch 1.13.1
• Numpy 1.22
• Matplotlib 3.6 (only plot functionalities linfa.plot_res)

Numerical Benchmarks

LINFA includes five numerical benchmarks:

• Trivial example.
• High dimensional example (Sobol' function).
• Two-element Windkessel model (a.k.a. RC model).
• Three-element Windkessel model (a.k.a. RCR model).
• Friedman 1 dataset example.

The implementation of the lumped parameter network models (RC and RCR models) follows closely from the code developed by the Schiavazzi Lab at the University of Notre Dame.

To run the tests type

python -m unittest linfa.linfa_test_suite.NAME_example

To run a limited number of iterations (say 100), you can instead type

it=100 python3 -m unittest linfa.linfa_test_suite.NAME_example

where NAME need to be replaced by

• trivial for the trivial example (Ex 1).
• highdim for the high-dimensional example (Ex 2).
• rc for the RC model (Ex 3).
• rcr for the RCR model (Ex 4).
• adaann for the Friedman model example (Ex 5).
• rcr_nofas_adaann for the RCR model, combining NoFAS with adaptive annealing (AdaAnn)

If used with adaptive annealing (AdaAnn) the it=100 option runs 100 iterations only at T=1 (i.e., to approximate the untempered posterior distribution). Therefore the total number of iterations is generally higher than specified through the it option.

At regular intervals, set by the parameter experiment.save_interval, LINFA writes a few results files. The sub-string NAME refers to the experiment name specified in the experiment.name variable, and IT indicates the iteration at which the file is written. The results files are

• log.txt contains the log profile information, i.e.
  • Iteration number.
  • Annealing temperature at each iteration.
  • Loss function at each iteration.
• NAME_grid_IT contains the inputs where the true model was evaluated.
• NAME_params_IT contains the batch of input parameters $\boldsymbol{z}_{K}$ in the physical space generated at iteration IT.
• NAME_samples_IT contains the batch of normalized parameters (parameter values before the coordinate transformation) generated at iteration IT.
• NAME_logdensity_IT contains the value of the log posterior density corresponding to each parameter realization.
• NAME_outputs_IT contains the true model (or surrogate model) outputs for each batch sample at iteration IT.
• NAME_IT.nf contains a backup of the normalizing flow parameters at iteration IT.

A post processing script is also available to plot all results. To run it type

python -m linfa.plot_res -n NAME -i IT -f FOLDER

where NAME and IT are again the experiment name and iteration number corresponding to the result file of interest, while FOLDER is the name of the folder with the results of the inference task are kept. Also the file format can be specified throught the -p option (options: pdf, png, jpg) and images with dark background can be generated using the -d flag.

The coverage resulting from these tests can be found at this link

Usage

To use LINFA with your model you need to specify the following components:

• A computational model.
• A surrogate model.
• A log-likelihood model.
• An optional transformation.

In addition you need to specify a list of options as discussed in the documentation.

Tutorial

Two step-by-step tutorials (tutorial 1 and tutorial 2) are also available which will guide you through an inference problem for a ballistic simulation.

Contributing

If you are interested in contributing to this project, plase take a look at the contributed guide provided with LINFA.

Citation

Did you use LINFA? Please cite our paper using:

@article{linfa-vi-paper,
  title={LINFA: a Python library for variational inference with normalizing flow and annealing},
  author={Wang, Yu and Cobian, Emma R and Lee, Jubilee and Liu, Fang and Hauenstein, Jonathan D and Schiavazzi, Daniele E},
  journal={arXiv preprint arXiv:2307.04675},
  year={2023}
}