Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update 2.0.0 #60

Merged
merged 54 commits into from
Mar 6, 2024
Merged

Update 2.0.0 #60

merged 54 commits into from
Mar 6, 2024

Conversation

mgqa34
Copy link
Contributor

@mgqa34 mgqa34 commented Feb 29, 2024 •
edited

Major Features and Improvements

  • Adapt to fate-v2.0 framework:
    • Migrate parameter-efficient fine-tuning training methods and models.
    • Migrate Standard Offsite-Tuning and Extended Offsite-Tuning（Federated Offsite-Tuning+)
    • Newly trainer，dataset, data_processing function design
  • New FedKSeed Federated Tuning Algorithm: train large language models in a federated learning setting with extremely low communication cost

mgqa34 and others added 30 commits February 19, 2024 21:51
@mgqa34
update code for llm-2.0
e3e6df5 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
update dataset for llm-2.0
bc1a310 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
fix chatglm2 detection
8d25ba8 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@talkingwallace
Merge pull request #44 from FederatedAI/feature-2.0.0-pellm
948efd2 
Feature 2.0.0 pellm
@talkingwallace
Add offsite-tuning contents Signed-off-by: weijingchen <talkingwallac…
6231f45 
…e@sohu.com>

Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
delete unused code
dad2cde 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@talkingwallace
Add offsite-tuning support Signed-off-by: weijingchen <talkingwallace…
127c427 
…@sohu.com>

Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
Merge pull request #45 from FederatedAI/dev-2.0.0-ot
9238182 
Dev 2.0.0 ot
@talkingwallace
Update modelzoo Signed-off-by: weijingchen <talkingwallace@sohu.com>
8733232 
Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
Merge pull request #46 from FederatedAI/dev-2.0.0-ot
29af45a 
Update modelzoo Signed-off-by: weijingchen <talkingwallace@sohu.com>
@sagewe
feat: implement fedkseed (#47)
32ae458 
Signed-off-by: sagewe <wbwmat@gmail.com>
@sagewe
feat: add fedkseed runner (#47)
3b77e4d 
Signed-off-by: sagewe <wbwmat@gmail.com>
@sagewe
fix: refactor args (#47)
40ecdeb 
Signed-off-by: sagewe <wbwmat@gmail.com>
@talkingwallace
Fix ot bug Signed-off-by: weijingchen <talkingwallace@sohu.com>
321f426 
Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
support qwen
69bc8a6 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
Merge pull request #49 from FederatedAI/feature-2.0.0-pellm
76e2d5f 
Feature 2.0.0 pellm
@mgqa34
Merge pull request #50 from FederatedAI/dev-2.0.0-ot-fix
8c26e81 
Fix ot bug Signed-off-by: weijingchen <talkingwallace@sohu.com>
@mgqa34
update data_collator and tokenizer
18f82a6 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
Merge pull request #51 from FederatedAI/feature-2.0.0-pellm
d25c4a1 
update data_collator and tokenizer
@talkingwallace
Fix codes Signed-off-by: weijingchen <talkingwallace@sohu.com>
b7c02c7 
Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
Merge pull request #53 from FederatedAI/dev-2.0.0-fix-ot-files
63ace1d 
Fix codes Signed-off-by: weijingchen <talkingwallace@sohu.com>
@talkingwallace
Remove model_zoo codes & fix runner Signed-off-by: weijingchen <talki…
86fa886 
…ngwallace@sohu.com>

Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
update code to support seq_cls in pellm
a8add72 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
fix trainer save trainable bug
2a20d86 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
Merge pull request #54 from FederatedAI/feature-2.0.0-pellm
1f4b6e5 
Feature 2.0.0 pellm
@talkingwallace
Fix model forward Signed-off-by: weijingchen <talkingwallace@sohu.com>
17e2d49 
Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
Merge pull request #55 from FederatedAI/dev-2.0.0-fix-ot-files
2e31a24 
Dev 2.0.0 fix ot files
@mgqa34
update requirements
7654d93 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@talkingwallace
Merge pull request #56 from FederatedAI/feature-2.0.0-pellm
fa157a5 
update requirements
@sagewe
fix: add hf dataset and hf model (#47)
e0e7cac 
Signed-off-by: sagewe <wbwmat@gmail.com>
sagewe and others added 5 commits February 29, 2024 19:28
@sagewe
docs: add fedkseed (#47)
f6924c4 
Signed-off-by: sagewe <wbwmat@gmail.com>
@mgqa34
update doc of pellm
b84c88f 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
Merge pull request #59 from FederatedAI/dev-2.0.0-fedkseed
7c2e019 
docs: add fedkseed (#47)
@mgqa34
update release note of fate-llm 2.0
955e511 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
Merge pull request #58 from FederatedAI/feature-2.0.0-pellm
8fcc0a6 
fix pellm
@mgqa34 mgqa34 marked this pull request as draft February 29, 2024 13:20
@mgqa34 mgqa34 mentioned this pull request Feb 29, 2024
Merged
@sagewe
chore: add licence (#47)
5397005 
Signed-off-by: sagewe <wbwmat@gmail.com>
@sagewe
Merge pull request #62 from FederatedAI/dev-2.0.0-fedkseed
6133366 
chore: add licence (#47)
mgqa34 and others added 7 commits March 1, 2024 16:05
@mgqa34
update doc of ChatGLM3
646c6ac 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
Merge pull request #63 from FederatedAI/feature-2.0.0-update_pellm_doc
0261e17 
update doc of ChatGLM3
@talkingwallace
Update ot doc Signed-off-by: weijingchen <talkingwallace@sohu.com>
d01a77a 
Signed-off-by: cwj <talkingwallace@sohu.com>
@talkingwallace
Fix doc path Signed-off-by: weijingchen <talkingwallace@sohu.com>
efc1074 
Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
Merge pull request #64 from FederatedAI/dev-2.0.0-llm-doc
2cf6dbc 
Update ot doc Signed-off-by: weijingchen <talkingwallace@sohu.com>
@talkingwallace
Update Readme Signed-off-by: weijingchen <talkingwallace@sohu.com>gi
c7cdd02 
Signed-off-by: cwj <talkingwallace@sohu.com>
@mgqa34
Merge pull request #65 from FederatedAI/dev-2.0.0-llm-doc
c7a3822 
Update Readme Signed-off-by: weijingchen <talkingwallace@sohu.com>
@mgqa34 mgqa34 added documentation Improvements or additions to documentation enhancement New feature or request labels Mar 6, 2024
@mgqa34 mgqa34 marked this pull request as ready for review March 6, 2024 06:11
@mgqa34
update readme
7a9a02a 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
update builtin_pellm_models doc
10643aa 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
fix builtin_pellm_models doc
9de48e8 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
update deployment desc of llm-2.0
c53749b 
Signed-off-by: mgqa34 <mgq3374541@163.com>
@mgqa34
Merge pull request #69 from FederatedAI/feature-2.0.0-update_doc
fea0580 
update readme
@mgqa34 mgqa34 merged commit abee189 into main Mar 6, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dylan-fan dylan-fan dylan-fan approved these changes

Assignees

@talkingwallace talkingwallace

@mgqa34 mgqa34

@sagewe sagewe

Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@mgqa34 @dylan-fan @talkingwallace @sagewe