Implementing GPT-2 architecture
- Built GPT-2 architecture by implementing unidirectional attention with multiple heads and batching using Python, PyTorch, and Einops.
- Implemented greedy search, temperature, and top-k algorithms to sample text from autoregressive models.
This project was guided by Redwood Research's Machine Learning for Alignment course.