Feature/vitt #4096

klhhhhh · 2023-06-27T09:38:19Z

📌 Checklist before creating the PR

I have created an issue for this PR for traceability
The title follows the standard format: [doc/gemini/tensor/...]: A concise description
I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

I have linked my PR to an issue (instruction)
My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
I have performed a self-review of my code
I have added thorough tests.
I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

🌝 Yes, I do.
🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

colossalai/shardformer/layer/embedding1d.py

colossalai/shardformer/policies/vit.py

tests/test_device/test_device_mesh.py

tests/test_shardformer/test_model/test_shard_t5.py

tests/test_shardformer/test_model/test_shard_bert.py

* first v of vit shardformer * keep vit * update * vit shard add vitattention vitlayer * update num head shard para * finish test for vit * add new_model_class & postprocess * add vit readme * delete old files & fix the conflict * fix sth

klhhhhh added 8 commits June 27, 2023 11:15

first v of vit shardformer

fb7ea37

keep vit

162dd1e

update

4047e4a

vit shard add vitattention vitlayer

326b3cd

update num head shard para

113eb55

finish test for vit

4c47c5b

add new_model_class & postprocess

6db5acc

add vit readme

7e9883a