#

attention-mechanisms

Here are 83 public repositories matching this topic...

lucidrains / mmdit

Implementation of a single layer of the MMDiT, proposed in Stable Diffusion 3, in Pytorch and Jax

deep-learning artificial-intelligence attention-mechanisms multi-modal-attention

Updated May 6, 2024
Python

cmhungsteve / Awesome-Transformer-Attention

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

computer-vision deep-learning transformers transformer awesome-list vit papers attention-mechanism attention-mechanisms self-attention transformer-architecture transformer-models detr vision-transformer transformer-cv transformer-with-cv transformer-awesome visual-transformer

Updated May 6, 2024

lucidrains / make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

deep-learning artificial-intelligence attention-mechanisms text-to-video axial-convolutions

Updated May 3, 2024
Python

lucidrains / magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

deep-learning transformers artificial-intelligence attention-mechanisms video-generation finite-scalar-quantization

Updated May 3, 2024
Python

lucidrains / MEGABYTE-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

deep-learning transformers artificial-intelligence attention-mechanisms long-context learned-tokenization

Updated May 3, 2024
Python

jshuadvd / LongRoPE

Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper

nlp machine-learning natural-language-processing ai deep-learning transformers artificial-intelligence gpt language-model natural-language-inference natural tokenization natural-language-understanding attention-is-all-you-need attention-mechanisms transformer-architecture natural-language-procressing tokenizers llm

Updated Apr 28, 2024
Python

kyegomez / MambaFormer

Implementation of MambaFormer in Pytorch ++ Zeta from the paper: "Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks"

ai ml transformer attention mamba ssms attention-is-all-you-need attention-mechanisms

Updated May 6, 2024
Python

lucidrains / BS-RoFormer

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

deep-learning transformers artificial-intelligence music-source-separation attention-mechanisms

Updated Apr 21, 2024
Python

lucidrains / local-attention

An implementation of local windowed attention for language modeling

deep-learning artificial-intelligence attention-mechanisms

Updated Apr 20, 2024
Python

lucidrains / meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

deep-learning transformers artificial-intelligence mesh-generation attention-mechanisms

Updated Apr 20, 2024
Python

kyegomez / CELESTIAL-1

Omni-Modality Processing, Understanding, and Generation

openai attention multi-modal multimodality attention-is-all-you-need attention-mechanisms multimodal multimodal-deep-learning gpt-4 gpt4 omnimodal

Updated May 3, 2024
Python

pouyasattari / Automatic-Generative-Code-with-Neural-Machine-Translation-for-data-security-purpose

Transformers, including the T5 and MarianMT, enabled effective understanding and generating complex programming codes. Consequently, they can help us in Data Security field. Let's see how!

transformers levenshtein-distance neural-machine-translation transfer-learning nmt tokenization npl fine-tuning attention-mechanisms zero-shot-classification

Updated Apr 6, 2024
Jupyter Notebook

kyegomez / Jamba

PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"

ai ml transformers artificial-neural-networks gpt attention-mechanism ssm attention-is-all-you-need attention-mechanisms

Updated Apr 1, 2024
Python

kyegomez / MambaTransformer

Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling

language machine-learning ai neural-network tensorflow pytorch recurrent-neural-networks artificial-intelligence neural-networks zeta ssm rnns attention-is-all-you-need attention-mechanisms multimodal gpt4

Updated Mar 15, 2024
Python

kyegomez / SparseAttention

Pytorch Implementation of the sparse attention from the paper: "Generating Long Sequences with Sparse Transformers"

machine-learning artificial-intelligence sparse-matrix attention-mechanism attention-is-all-you-need attention-mechanisms sparse-attn

Updated Mar 14, 2024
Python

kyegomez / PaLM2-VAdapter

Implementation of "PaLM2-VAdapter:" from the multi-modal model paper: "PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter"

ai models ml transformers attention deeplearning multi-modal neural-nets attention-is-all-you-need attention-mechanisms

Updated Mar 11, 2024
Python

kyegomez / KosmosG

My implementation of the model KosmosG from "KOSMOS-G: Generating Images in Context with Multimodal Large Language Models"

computer-vision attention-mechanism attention-is-all-you-need multimodal-learning attention-mechanisms multimodal

Updated Mar 11, 2024
Python

lucidrains / phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

deep-learning transformers artificial-intelligence attention-mechanisms text-to-video imagination-machine

Updated Mar 8, 2024
Python

kyegomez / Hedgehog

Implementation of the model "Hedgehog" from the paper: "The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry"

open-source ai ml attention multi-modal neural-nets feedforward softmax attention-is-all-you-need attention-mechanisms mlps ffns opensource-ai

Updated Mar 11, 2024
Python

lucidrains / flash-attention-jax

Implementation of Flash Attention in Jax

deep-learning artificial-intelligence attention-mechanisms jax long-context-attention

Updated Mar 1, 2024
Python

Improve this page

Add a description, image, and links to the attention-mechanisms topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the attention-mechanisms topic, visit your repo's landing page and select "manage topics."