Official release of InternLM2.5 base and chat models. 1M context support
-
Updated
Aug 5, 2024 - Python
Official release of InternLM2.5 base and chat models. 1M context support
Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
Transformers with Arbitrarily Large Context
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)
The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"
Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718
Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch
LongQLoRA: Extent Context Length of LLMs Efficiently
[COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality
Implementation of paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
LongAlign: A Recipe for Long Context Alignment Encompassing Data, Training, and Evaluation
The official repo for "LLoCo: Learning Long Contexts Offline"
My own attempt at a long context genomics model, leveraging recent advances in long context attention modeling (Flash Attention + other hierarchical methods)
ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models
This is the official implementation of the paper "Needle In A Multimodal Haystack"
Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch
Add a description, image, and links to the long-context topic page so that developers can more easily learn about it.
To associate your repository with the long-context topic, visit your repo's landing page and select "manage topics."