Project Description

• This project involves writing the transformer code without using the PyTorch library. The main purpose is to understand the code and the underlying concept.

Project Specification

• Study linear algebra (Khan Academy)
• Understand the structure of the transformer (+ attention)
• Rewrite the code
  • First, without using the PyTorch Library
  • Second, using the PyTorch Library

Project Setting

• Create a new conda environment
  • conda create -n [your env] python=3.9
  • conda activate [your env]