feat: implement learnable gated pooling model

[devin-ai-integration]

Learnable Gated Pooling Implementation

This PR implements the Learnable Gated Pooling model as described in the research proposal. The implementation combines learnable weights with a gating mechanism to dynamically control the contribution of each element in the input sequence.

Implementation Details

• Added LearnableGatedPooling model with learnable weights and gating mechanism
• Implemented training, evaluation, and preprocessing utilities
• Added configuration management
• Updated dependencies in requirements.txt

Mathematical Representation

The Learnable Gated Pooling can be represented as:

weighted_x = x * w  (element-wise multiplication, broadcasting w across sequence length)
gated_x = weighted_x * sigmoid(g) (element-wise multiplication)
pooled_vector = pooling_operation(gated_x)

Testing

The implementation includes a complete training pipeline with:

• Data preprocessing with padding
• Training loop with loss tracking
• Evaluation functionality
• Example usage in main.py

Link to Devin run: https://app.devin.ai/sessions/2c875739803f45ee89a2fa141ea1bf90

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add "(aside)" to your comment to have me ignore it.
• Look at CI failures and help fix them

⚙️ Control Options:

• Disable automatic comment and CI monitoring