Sentence Classification using Transformers

This project trains a transformer-based model to classify English sentences as questions or non-questions with up to 99% accuracy.

Files

• questions_solution.py: Custom implementation of transformer training and evaluation.
• questions_base.py: Base code (unchanged).
• questions_common.py: Utility functions.
• questions_dataset/: Contains training, validation, and test data.

Note

The files questions_base.py and questions_common.py were provided by Dr. Vassilis Athitsos as part of the coursework at the University of Texas at Arlington. All model implementations in questions_solution.py are original work.

Models

• train_transformer: Trains model
• evaluate_transformer: Evaluates model accuracy

Model Architecture

• Transformer encoder with:
  • 3 Attention Heads
  • 32 Dense Dimensions
  • Positional Embeddings (100 Dims)
  • 10 Epochs

How to Run

python3 reverse_base.py

Result

Classification accuracy: 98.67%