(Work in Progress) Joint Intent Detection and Slot Labeling via Self-Routing Capsule Networks

PyTorch implementation of a theoretized custom deep learning architecture that jointly solves intent detection and slot labeling by adapting the self-routing capsule model from computer vision.

References

1. Capsule Theory

• Transforming Auto-Encoders
• Dynamic Routing Between Capsules
• Self-Routing Capsule Networks

2. CapsNets in ID-SF

• Joint Slot Filling and Intent Detection via Capsule Neural Networks
• Zero-shot User Intent Detection via Capsule Neural Networks
• Intent Detection and Slot Filling with Capsule Net Architectures for a Romanian Home Assistant

3. Transformers / Self-Attention

• Attention is all you need
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• DIET: Lightweight Language Understanding for Dialogue Systems

4. Transformer-Capsule Architectures

• Capsule-Transformer for Neural Machine Translation
• Transformer-Capsule Model for Intent Detection
• Trans-Caps: Transformer Capslue Networks with Self-Attention Routing

5. Word Embeddings

6. Others

• Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradiewnt Method for Stochastic Optimization

7. Data

• NLU-Benchmark Dataset