Skip to content
This repository has been archived by the owner on Feb 15, 2024. It is now read-only.

logicalclocks/aml_end_to_end

Repository files navigation

AML End to End Example

Project description

This project demonstrates end to end pipeline how to train binary anti money laundering (AML) classifier based on Generative Adversarial Networks (GANs) and Graph embeddings. Proposed solution includes following sub sections:

  • Data ingestion - We will use sample of transactions data generated by AMLSim
  • Feature store – We use Hopsworks Feature Store to compute features, organize them as feature groups and store for downstream analysis, such as creating training datasets for model training, as well as retrieving them
  • Graph Embeddings - We will use StellarGraph library to compute graph embeddings.
  • Anomaly detection model - We will use keras implementation of adversarial anomaly detection that was adapted to tabular data.
  • Hyper parameter tuning - We will use Maggy to conduct experiments for hyperparameter tuning.
  • Model serving - We will use Hopsworks model server to predict anomalous transactions.

Demo dataset

A sample of transaction data is provided in the folder ./demodata, including upload alert_transactions.csv, party.csv and transactions.csv.

Anomaly detection model

Keras implementation of adversarial anomaly detection is provided in the folder ./adversarialaml. To use this library install as python library from https://github.com/logicalclocks/AMLend2end.git. intall_python_lib_from_git.gif

End to End pipeline

To successfully complete this tutorial clone this repository to your Hopsworks project git_clone.gif

Jupyter notebooks step by step

Run jupyter notebooks in the following order:

  1. 1_transaction_feature_engineering_ingestion.ipynb
  2. 2_prep_training_dataset_for_embeddings.ipynb
  3. 3_maggy_node_embeddings.ipynb
  4. 4_compute_node_embeddings.ipynb
  5. 5_predict_and_create_node_embeddings_fg.ipynb
  6. 6_create_anomaly_detection_td.ipynb
  7. 7_maggy_adversarial_aml.ipynb
  8. 8_train_adversarial_aml.ipynb
  9. 9_aml_model_server.ipynb