CAOF-Blend

Augmentation Blending with Clustering-Aware Outlier Factor: An Outlier-Driven Perspective for Enhanced Contrastive Learning [Paper] [Code] Authors: Qianwen Meng, Hangwei Qian, Yonghui Xu, Lizhen Cui This work is submitted to Knowledge-Based Systems.

CAOF-Blend Overview:

Abstract

Data augmentation is used to generate a wider range of variations by making subtle modifications to the data without explicit costly and labor-intensive collection. It is particularly useful for boosting contrastive learning by introducing diversity and variability into the original dataset. However, selecting optimal augmentations that preserve the semantic consistency with the original samples poses challenges owing to variations in the quality of different augmentation transformations. To address this challenge, we formulate the data augmentation optimization problem as a novel outlier-driven augmentation blending problem. Specifically, we design an innovative metric, the Clustering-Aware Outlier Factor (CAOF), to quantify the semantic inconsistency between the augmented and original samples. The CAOF can identify semantically inconsistent outliers in augmented samples that are generated by various transformations, thereby offering more nuanced and context-aware density assessments by considering the intra-cluster neighbors for individual samples. Each augmentation transformation is evaluated using the proposed CAOF, which reflects the likelihood of producing semantically inconsistent augmented samples. The likelihood determines the importance of the transformation in generating a refined blended augmentation, which is subsequently fed into the contrastive learning framework. We conducted empirical evaluations using six time-series datasets. We compared our method with three state-of-the-art augmentation selection methods using five popular contrastive learning models. The results demonstrate the superiority of the proposed approach in discovering outliers and refining augmentations in contrastive learning for time-series classification tasks.

Requirements for this project:

• Python ≥ 3.6
• PyTorch ≥ 1.4

Required Packages:

• torch
• os
• argparse
• numpy
• sklearn
• collections
• sklearn

To perform downstream classification task based on the learned representations, please refer to the options below:

python main.py \
--dataset HAR \
--p 0.5 \
--contamination 0.1 \
--dimension 2 \
--use_tsne \
--original \
--k_dblof 10 \
--k_iclof 10 \
--cont_dblof 0.1 \
--cont_iclof 0.1

Results:

• The experimental results will be saved in "experiment_{args.dataset_name}" directory by default
• If you choose different partitions or masking strategies, the suffix will be added automatically such as "experiment_{args.dataset_name}_{args.mask_mode}layer0{args.dist_threshold}"

Citation:

If you find any of the codes helpful, kindly cite our paper.

@article{meng4770304augmentation,
  title={Augmentation Blending with Clustering-Aware Outlier Factor: An Outlier-Driven Perspective for Enhanced Contrastive Learning},
  author={Meng, Qianwen and Qian, Hangwei and Xu, Yonghui and Cui, Lizhen},
  journal={Available at SSRN 4770304}
}