feat: implement Learnable Gated Pooling methodology

[devin-ai-integration]

Implement Learnable Gated Pooling Methodology

This PR implements the Learnable Gated Pooling approach for sequence pooling, which combines learnable weights with a gating mechanism for dynamic feature importance weighting.

Implementation Details

• Created LearnableGatedPooling model class with learnable weights and gating mechanism
• Implemented training, evaluation, and data preprocessing scripts
• Added configuration file for model hyperparameters
• Updated requirements.txt with necessary dependencies

Mathematical Implementation

The implementation follows the mathematical representation:

1. Learnable weights (w) for each dimension
2. Gating mechanism (g) applied element-wise
3. Average pooling operation on the gated and weighted input

Testing Status

The implementation includes the core functionality but has not been tested with real data. The preprocess.py currently uses placeholder data for demonstration. Real data integration and testing will be needed before production use.

Files Changed

• src/models.py: Core LearnableGatedPooling implementation
• src/train.py: Training logic with validation
• src/evaluate.py: Model evaluation
• src/preprocess.py: Data preparation (with placeholder data)
• src/main.py: Main experiment workflow
• config/config.yaml: Configuration parameters
• requirements.txt: Updated dependencies

Link to Devin run: https://app.devin.ai/sessions/96286c2468ab45119dd9eb00053b85f1

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add "(aside)" to your comment to have me ignore it.
• Look at CI failures and help fix them

⚙️ Control Options:

• Disable automatic comment and CI monitoring