selfAttention selfAttention

Constructors

selfAttention(size_t n_embd)

Initializes a self attention block with three square weight matrices following the passed n_embd dimensions.

---

Basic Info

Applies self attention.

Instance Variables

Variable	Type
wq	utility::matrix
wk	utility::matrix
wv	utility::matrix
q	utility::matrix
k	utility::matrix
v	utility::matrix
p	utility::matrix
x	utility::matrix
n_tok	size_t
n_embd	size_t
generator	std::default_random_engine
initDist	std::normal_distribution<float>

---

Functions

• attention()
• backward()
• changeOne()
• getNEmbed()
• readFromFile()
• saveToFile()

---

Example

// Initializing a self attention block
selfAttention a(91);