Pinned Loading

1. Universal_Decoder Universal_Decoder Public

   This is the Transformers Decoder version, based on the "Universal Transformers (Encoder-Decoder)" paper, and trained on natural data rather than algorithmic tasks.

   Python 2

2. FP_vs_1Bit_InductionHeadCircuit FP_vs_1Bit_InductionHeadCircuit Public

   Comparing the full-precision Query, Keys, and Values matrices with their 1-bit counterparts in a two-layer, attention-only transformer trained on a synthetic copying task.

   Python 1

3. baby_mamba baby_mamba Public

   An implementation of Mamba to develop an understanding of its functioning.

   Python 2

4. BERT_with_Residual_vs_Highway BERT_with_Residual_vs_Highway Public

   Comparing between residual stream and highway stream in transformers(BERT) .

   Python 3

5. ResLM ResLM Public

   Querying a Language model in a residual path for drug-target interaction task

   Python 2

6. test_liger_kernels test_liger_kernels Public

   Benchmarking the performance of Liger Kernels Library Using instruction following and reasoning tasks

   Python