Fork Info

This is a fork of the DAGMM repo : https://github.com/tnakae/DAGMM

I have patched the code to work with Tensorflow V2. I have tested it with Python 3.11 and Tensorflow 2.20.0. In additional, I have also implemented STD-DAGMM (https://arxiv.org/abs/1911.12560) which is an extension of DAGMM. Unfortunately, this is not a fully migrated code from TF1 to TF2, but patched with compatibility layers provided by tf.compat.v1. Please create an issue incase there is some incorrectness with the implementaion.

DAGMM Tensorflow implementation

Deep Autoencoding Gaussian Mixture Model.

This implementation is based on the paper Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection [Bo Zong et al (2018)]

STD-DAGMM Tensorflow implementation

Implemented based on Free-riders in Federated Learning: Attacks and Defenses [Lin et al (2019)]

Caution

All implementations are unofficial. The patching was done with the help of LLMs. While I have verified the code to the best of my abilities and the results with the test notebooks are similar to that of the forked repo, I still advise double-checking your results.

Requirements

• python 3.11 (should probably work with 3.12)
• tensorflow==2.20.0
• numpy==2.2.6
• scikit-learn==1.6.1

Usage instructions

To use DAGMM model, you need to create "DAGMM" object. At initialize, you have to specify next 4 variables at least. "STD-DAGMM" object works the same way.

• comp_hiddens : list of int
  • sizes of hidden layers of compression network
  • For example, if the sizes are [n1, n2], structure of compression network is: input_size -> n1 -> n2 -> n1 -> input_sizes
• comp_activation : function
  • activation function of compression network
• est_hiddens : list of int
  • sizes of hidden layers of estimation network.
  • The last element of this list is assigned as n_comp.
  • For example, if the sizes are [n1, n2], structure of estimation network is: input_size -> n1 -> n2 (= n_comp)
• est_activation : function
  • activation function of estimation network

Then you fit the training data, and predict to get energies (anomaly score). It looks like the model interface of scikit-learn.

For more details, please check out dagmm/dagmm.py docstrings.

Example

Small Example

import tensorflow as tf
from dagmm import DAGMM,STDDAGMM

# Initialize
model = DAGMM(
  comp_hiddens=[32,16,2], comp_activation=tf.nn.tanh,
  est_hiddens=[16.8], est_activation=tf.nn.tanh,
  est_dropout_ratio=0.25
)

# for STD-DAGMM
# model = STDDAGMM(
#   comp_hiddens=[32,16,2], comp_activation=tf.nn.tanh,
#   est_hiddens=[16.8], est_activation=tf.nn.tanh,
#   est_dropout_ratio=0.25
# )

# Fit the training data to model
model.fit(x_train)

# Evaluate energies
# (the more the energy is, the more it is anomary)
energy = model.predict(x_test)

# Save fitted model to the directory
model.save("./fitted_model")

# Restore saved model from dicrectory
model.restore("./fitted_model")

Jupyter Notebook Example

You can use next jupyter notebook examples using DAGMM model.

• Simple DAGMM Example notebook : This example uses random samples of mixture of gaussian. If you want to know simple usage, this notebook is recommended.
• KDDCup99 10% Data Evaluation : Performance evaluation of anomaly detection for KDDCup99 10% Data with the same condition of original paper (need pandas)

Notes

GMM Implementation

The equation to calculate "energy" for each sample in the original paper uses direct expression of multivariate gaussian distribution which has covariance matrix inversion, but it is impossible sometimes because of singularity.

Instead, this implementation uses cholesky decomposition of covariance matrix. (this is based on GMM code in Tensorflow code)

In DAGMM.fit(), it generates and stores triangular matrix of cholesky decomposition of covariance matrix, and it is used in DAGMM.predict(),

In addition to it, small perturbation (1e-6) is added to diagonal elements of covariance matrix for more numerical stability (it is same as Tensorflow GMM implementation, and another author of DAGMM also points it out)

Parameter of GMM Covariance (lambda_2)

Default value of lambda_2 is set to 0.0001 (0.005 in original paper). When lambda_2 is 0.005, covariances of GMM becomes too large to detect anomaly points. But perhaps it depends on distribution of data and method of preprocessing (for example a method of normalization). Recommend to control lambda_2 when performance metrics is not good.