feat: Implement Learnable Gated Pooling model

[devin-ai-integration]

Learnable Gated Pooling Implementation

This PR implements the Learnable Gated Pooling approach, which combines learnable weights with a gating mechanism for dynamic feature importance weighting.

Implementation Details

• Implemented LearnableGatedPooling model with learnable weights and gating mechanism
• Added training loop with validation phase
• Implemented evaluation metrics (accuracy, precision, recall, F1)
• Created data preprocessing utilities with PyTorch Dataset/DataLoader

Directory Structure

.
├── config/          # For experiment parameters
├── data/           # For training and evaluation data
├── models/         # For storing trained models
├── output/         # (unchanged)
└── src/
    ├── models.py      # LearnableGatedPooling implementation
    ├── train.py       # Training loop implementation
    ├── evaluate.py    # Model evaluation
    ├── preprocess.py  # Data preprocessing
    └── main.py        # Main experiment script

Mathematical Representation

The implementation follows the mathematical formulation:

1. weighted_x = x * w (element-wise multiplication)
2. gated_x = weighted_x * sigmoid(g)
3. pooled_vector = mean(gated_x)

Dependencies

• PyTorch (>=2.0.0)
• NumPy (>=1.21.0)
• scikit-learn (>=1.0.0)

Testing

The implementation includes dummy data generation for initial testing. Users can replace this with actual data by modifying the data loading logic in preprocess.py.

Link to Devin run: https://app.devin.ai/sessions/694c2bca7e9f45e38d7abbf99ed21867