Skip to content

Bond1995/Markov

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Code organization

A few pointers

  • Markov-LLM contains the full transformer model for first-order Markov sources.
  • Markov-LLM-k contains the full transformer model for k-order Markov sources of the form described in Sec. 4.
  • Markov-LLM-m contains the full transformer model for k-order Markov sources with arbitrary vocabulary size.
  • Markov-Simple contains the simplified transformer model for first-order Markov sources used for the experiments in Sec. 3.

The script to run the experiments is in src/main.py.

About

Code for experiments on transformers using Markovian data.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published