Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JetMoE model #30005

Merged
merged 122 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
122 commits
Select commit Hold shift + click to select a range
b63bcaf
init jetmoe code
yikangshen Apr 2, 2024
03f646e
Merge branch 'huggingface:main' into main
yikangshen Apr 2, 2024
ed52b57
update archive maps
yikangshen Apr 2, 2024
150cd93
remove flax import
yikangshen Apr 2, 2024
436a44c
fix import error
yikangshen Apr 2, 2024
bcf597f
update README
yikangshen Apr 2, 2024
5c0400e
ruff fix
yikangshen Apr 2, 2024
e61d131
update readme
yikangshen Apr 2, 2024
57b13eb
fix
yikangshen Apr 2, 2024
1f27ad4
update config
yikangshen Apr 2, 2024
2ea5542
fix issue
yikangshen Apr 2, 2024
109a8c2
merge files
yikangshen Apr 2, 2024
21a4c2d
fix model bug
yikangshen Apr 2, 2024
9d542ac
fix test
yikangshen Apr 2, 2024
c5092b4
auto fix
yikangshen Apr 3, 2024
41f2436
model size
yikangshen Apr 3, 2024
3052ce8
add comments
yikangshen Apr 3, 2024
539cfb9
fix form
yikangshen Apr 3, 2024
0f6af1d
add flash attention support
yikangshen Apr 3, 2024
165e20d
fix attention head number
yikangshen Apr 3, 2024
68633f9
fix init
yikangshen Apr 3, 2024
d39a0e9
fix support list
yikangshen Apr 3, 2024
ef62bf3
sort auto mapping
yikangshen Apr 3, 2024
c0a3076
fix test
yikangshen Apr 3, 2024
4d79ce6
fix docs
yikangshen Apr 3, 2024
e5336b5
update test
yikangshen Apr 3, 2024
67aedd1
fix test
yikangshen Apr 3, 2024
c87de94
fix test
yikangshen Apr 3, 2024
2f02e7e
change variable name
yikangshen Apr 3, 2024
fc39dcc
fix config
yikangshen Apr 3, 2024
b4b5738
fix init
yikangshen Apr 3, 2024
c370377
update format
yikangshen Apr 3, 2024
f443c29
clean code
yikangshen Apr 3, 2024
852ef61
fix config
yikangshen Apr 3, 2024
9517a2b
fix config
yikangshen Apr 3, 2024
30b826d
change default config
yikangshen Apr 3, 2024
a18a67a
update config
yikangshen Apr 3, 2024
88991b5
fix issues
yikangshen Apr 15, 2024
7d23d95
Merge branch 'main' into main
yikangshen Apr 15, 2024
28ed7c4
update formate
yikangshen Apr 15, 2024
913dc9e
update config argument
yikangshen Apr 15, 2024
5611f09
update format
yikangshen Apr 15, 2024
430ea6c
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 18, 2024
3aff069
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 18, 2024
5528995
Merge branch 'main' into main
yikangshen Apr 18, 2024
c51e987
change to mixtral aux loss
yikangshen Apr 20, 2024
4927e60
change to cache_position
yikangshen Apr 22, 2024
cc89ea3
debug
yikangshen Apr 22, 2024
4dcbd26
fix bugs
yikangshen Apr 22, 2024
7a37970
Merge branch 'main' into main
yikangshen Apr 22, 2024
5cdc9c7
debug
yikangshen Apr 22, 2024
535c24c
fix format
yikangshen Apr 22, 2024
192202e
fix format
yikangshen Apr 23, 2024
ddb11d0
fix copy
yikangshen Apr 23, 2024
f9877f2
fix format
yikangshen Apr 23, 2024
7ec96dc
fix format
yikangshen Apr 23, 2024
2a0e123
fix sort
yikangshen Apr 23, 2024
cecb26f
fix sort
yikangshen Apr 23, 2024
5f2cffb
fix sort
yikangshen Apr 23, 2024
797a89b
add copy comment
yikangshen Apr 23, 2024
a7a6e2d
add copy from
yikangshen Apr 23, 2024
b37bb87
remove debug code
yikangshen Apr 23, 2024
675867d
Merge branch 'main' into main
yikangshen Apr 24, 2024
ae19e2c
revert readme update
yikangshen Apr 24, 2024
d478796
add copy
yikangshen Apr 24, 2024
b8cdc4a
debug
yikangshen Apr 24, 2024
15170ee
remove debug code
yikangshen Apr 24, 2024
6e46417
fix flash attention
yikangshen Apr 24, 2024
be88983
add comments
yikangshen Apr 25, 2024
06a5d62
Merge branch 'huggingface:main' into main
yikangshen Apr 25, 2024
8633fc5
clean code
yikangshen Apr 26, 2024
97125e3
Merge branch 'main' of https://github.com/yikangshen/transformers
yikangshen Apr 26, 2024
d3002f3
clean format
yikangshen Apr 26, 2024
bcad4fa
fix format
yikangshen Apr 26, 2024
16f6fd8
fix format
yikangshen Apr 26, 2024
71f6431
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 30, 2024
fb26a0e
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 30, 2024
6cb4df0
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 30, 2024
330a89b
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 30, 2024
0ff62a9
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 30, 2024
f6ffe33
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen Apr 30, 2024
e40e170
change variable name
yikangshen Apr 30, 2024
7db942f
add copied from
yikangshen Apr 30, 2024
8a2b593
Merge branch 'huggingface:main' into main
yikangshen Apr 30, 2024
05635bf
fix variable name
yikangshen Apr 30, 2024
8914263
Merge branch 'main' of https://github.com/yikangshen/transformers
yikangshen Apr 30, 2024
ea7daa1
remove deprecated functinos
yikangshen Apr 30, 2024
22a03f0
sync to llama implementation
yikangshen Apr 30, 2024
303942a
fix format
yikangshen Apr 30, 2024
3003124
fix copy
yikangshen Apr 30, 2024
5b7101a
fix format
yikangshen May 1, 2024
5f335c3
update format
yikangshen May 1, 2024
d5a6604
remove repr
yikangshen May 2, 2024
82069a1
add comment for moe weight
yikangshen May 2, 2024
01733bd
Merge branch 'huggingface:main' into main
yikangshen May 3, 2024
c6e5e8b
fix copy
yikangshen May 3, 2024
5cfc652
Update src/transformers/models/jetmoe/configuration_jetmoe.py
yikangshen May 10, 2024
42c02b0
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
976b4cf
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
6588db5
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
dea51cb
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
410882a
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
077e46a
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
193a9ef
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
14512fc
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
0bbfc87
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
ecb0337
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
7f6d529
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 10, 2024
4b327ba
add comments and reformat config
yikangshen May 10, 2024
7f44751
fix format
yikangshen May 10, 2024
6c0ea95
fix format
yikangshen May 10, 2024
a9e2c22
Merge branch 'main' into main
yikangshen May 10, 2024
9c8081d
fix format
yikangshen May 10, 2024
cf17204
update test
yikangshen May 10, 2024
41d1a70
update doc string in config
yikangshen May 10, 2024
58e5627
Update src/transformers/models/jetmoe/modeling_jetmoe.py
yikangshen May 13, 2024
8341eea
update config doc
yikangshen May 13, 2024
71a2939
Merge branch 'main' of https://github.com/yikangshen/transformers
yikangshen May 13, 2024
9e8b759
update attention cache
yikangshen May 13, 2024
5c21dfe
Merge branch 'huggingface:main' into main
yikangshen May 13, 2024
1b8ed08
fix format
yikangshen May 13, 2024
060af34
fix copy
yikangshen May 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -397,6 +397,7 @@ Current number of checkpoints: ![](https://img.shields.io/endpoint?url=https://h
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (from Salesforce) released with the paper [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500) by Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi.
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (from MyShell AI) released in the repository [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE) by Yikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (from Microsoft Research Asia) released with the paper [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) by Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_de.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,7 @@ Aktuelle Anzahl der Checkpoints: ![](https://img.shields.io/endpoint?url=https:/
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (from Salesforce) released with the paper [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500) by Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi.
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (from MyShell AI) released with the paper [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE) by Yikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (from Microsoft Research Asia) released with the paper [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) by Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_es.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,7 @@ Número actual de puntos de control: ![](https://img.shields.io/endpoint?url=htt
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (from Salesforce) released with the paper [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500) by Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi.
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (from MyShell AI) released with the paper [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE) by Yikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (from Microsoft Research Asia) released with the paper [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) by Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_fr.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,7 @@ Nombre actuel de points de contrôle : ![](https://img.shields.io/endpoint?url=h
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (d'OpenAI) a été publié dans l'article [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) par Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (de l'Université de Beihang, UC Berkeley, Rutgers University, SEDD Company) a été publié dans l'article [Informer : Au-delà du Transformer efficace pour la prévision de séries temporel
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (de Salesforce) a été publié dans l'article [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500) de Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi.
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (de MyShell AI) publié dans l'article [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE) parYikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (d'OpenAI) a été publié dans l'article [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) de Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (de Microsoft Research Asia) a été publié dans l'article [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) de Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (de Microsoft Research Asia) a été publié dans l'article [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) de Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_hd.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,7 @@ conda install conda-forge::transformers
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (Salesforce से) Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi. द्वाराअनुसंधान पत्र [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500) के साथ जारी किया गया
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (MyShell AI से) Yikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin. द्वाराअनुसंधान पत्र [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE) के साथ जारी किया गया
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (from Microsoft Research Asia) released with the paper [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) by Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,7 @@ Flax、PyTorch、TensorFlowをcondaでインストールする方法は、それ
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (OpenAI から) Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever から公開された研究論文: [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/)
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (Salesforce から) Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi. から公開された研究論文 [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500)
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (MyShell AI から) Yikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin. から公開された研究論文 [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE)
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (OpenAI から) Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever から公開された研究論文: [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf)
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (from Microsoft Research Asia) released with the paper [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) by Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (Microsoft Research Asia から) Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou から公開された研究論文: [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318)
Expand Down
1 change: 1 addition & 0 deletions README_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,7 @@ Flax, PyTorch, TensorFlow 설치 페이지에서 이들을 conda로 설치하는
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (OpenAI 에서) Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever 의 [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) 논문과 함께 발표했습니다.
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (Salesforce 에서 제공)은 Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi.의 [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500)논문과 함께 발표했습니다.
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (MyShell AI 에서 제공)은 Yikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin.의 [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE)논문과 함께 발표했습니다.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (OpenAI 에서) Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever 의 [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) 논문과 함께 발표했습니다.
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (from Microsoft Research Asia) released with the paper [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) by Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (Microsoft Research Asia 에서) Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou 의 [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) 논문과 함께 발표했습니다.
Expand Down
1 change: 1 addition & 0 deletions README_pt-br.md
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,7 @@ Número atual de pontos de verificação: ![](https://img.shields.io/endpoint?ur
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[InstructBLIP](https://huggingface.co/docs/transformers/model_doc/instructblip)** (from Salesforce) released with the paper [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://arxiv.org/abs/2305.06500) by Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, Steven Hoi.
1. **[JetMoE](https://huggingface.co/docs/transformers/model_doc/jetmoe)** (from MyShell AI) released with the paper [myshell-ai/JetMoE](https://github.com/myshell-ai/JetMoE) by Yikang Shen, Zhen Guo, Tianle Cai, and Zengyi Qin.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[KOSMOS-2](https://huggingface.co/docs/transformers/model_doc/kosmos-2)** (from Microsoft Research Asia) released with the paper [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://arxiv.org/abs/2306.14824) by Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
Expand Down