Skip to content

Commit

Permalink
v0.5.0
Browse files Browse the repository at this point in the history
  • Loading branch information
Tongjilibo committed Apr 18, 2024
1 parent dbe0b95 commit a181533
Show file tree
Hide file tree
Showing 6 changed files with 8 additions and 7 deletions.
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,9 @@ pip install git+https://github.com/Tongjilibo/bert4torch
### 4.1 版本历史
|更新日期| bert4torch | torch4keras | 版本说明 |
|------| ---------------- | ----------------- |----------- |
|20240418| 0.5.0 | 0.2.2 | 修复chatglm3的bug, 修复save_pretrained时多文件的bug,增加CausalLMLoss, 修改deepspeed的传参逻辑,修改Text2Vec的bug, 完善openai client, 增加get_weight_decay_optim_groups|
|20240317| 0.4.9.post2 | 0.2.1.post2 |增加get_weight_decay_optim_groups函数, attention中允许is_causal,修改repetition_penalty的bug,把baichuan从llama中剥离,修复config_path的bug,允许num_key_value_heads参数,[torch4keras-v0.2.1.post2](https://github.com/Tongjilibo/torch4keras/releases/tag/v0.2.1.post2)更新特性|
|20240221| 0.4.8 | 0.2.0|fastapi发布服务允许闲时offload到cpu, `build_transformer_model`允许从hf下载, 添加`FillMask`的pipeline, 添加`SequenceClassificationTrainer`|
|20240204| 0.4.7 | 0.1.9|修改`save_pretrained`用于保存文件夹, 增加GenerateSpeed用于统计token生成速度,修复t5在use_states=True时候的错误, 修改层次编码的bug, 增加deepseek_moe模型,修复generation并发错误,优化大模型耗时|
|20240116| 0.4.6 | 0.1.8|bug修复,增加`save_pretrained`用于保存`transformer`格式的权重, 增加部分`embedding`模型|

[更多版本](https://github.com/Tongjilibo/bert4torch/blob/master/docs/Update.md)

Expand Down
2 changes: 1 addition & 1 deletion docs/History.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## 更新历史

- **20240403**:修改Text2Vec的bug
- **20240418**:修改Text2Vec的bug, 完善openai client, 增加get_weight_decay_optim_groups
- **20240331**: 修复chatglm3的bug, 修复save_pretrained时多文件的bug,增加CausalLMLoss, 修改deepspeed的传参逻辑
- **20240317**: 修复config_path的bug,允许num_key_value_heads参数
- **20240316**: 增加get_weight_decay_optim_groups函数, attention中允许is_causal,修改repetition_penalty的bug,把baichuan从llama中剥离,[torch4keras-v0.2.1](https://github.com/Tongjilibo/torch4keras/releases/tag/v0.2.1)更新特性
Expand Down
2 changes: 2 additions & 0 deletions docs/Update.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

|更新日期| bert4torch版本 | torch4keras版本 | 版本说明 |
|------| ---------------- | ----------------- |----------- |
|20240418| 0.5.0 | 0.2.2 | 修复chatglm3的bug, 修复save_pretrained时多文件的bug,增加CausalLMLoss, 修改deepspeed的传参逻辑,修改Text2Vec的bug, 完善openai client, 增加get_weight_decay_optim_groups|
|20240317| 0.4.9.post2 | 0.2.1.post2 |增加get_weight_decay_optim_groups函数, attention中允许is_causal,修改repetition_penalty的bug,把baichuan从llama中剥离,修复config_path的bug,允许num_key_value_heads参数,[torch4keras-v0.2.1.post2](https://github.com/Tongjilibo/torch4keras/releases/tag/v0.2.1.post2)更新特性|
|20240221| 0.4.8 | 0.2.0|fastapi发布服务允许闲时offload到cpu, `build_transformer_model`允许从hf下载, 添加`FillMask`的pipeline, 添加`SequenceClassificationTrainer`|
|20240204| 0.4.7 | 0.1.9|修改`save_pretrained`用于保存文件夹, 增加GenerateSpeed用于统计token生成速度,修复t5在use_states=True时候的错误, 修改层次编码的bug, 增加deepseek_moe模型,修复generation并发错误,优化大模型耗时|
|20240116| 0.4.6 | 0.1.8|bug修复,增加`save_pretrained`用于保存`transformer`格式的权重, 增加部分`embedding`模型|
Expand Down
2 changes: 1 addition & 1 deletion examples/llm/task_chatglm_lora.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
max_target_length = 64
max_seq_length = max_source_length + max_target_length
lr = 5e-4
batch_size = 4 # 根据显存大小调整
batch_size = 16 # 根据显存大小调整
eval_batch_size = 4
grad_accumulation_steps = 1 # 根据显存大小调整
epochs = 1
Expand Down
2 changes: 1 addition & 1 deletion examples/sequence_labeling/uie/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# 说明
- convert.py: 下载uie模型并且转成pytorch格式
- convert.py: 下载uie模型并且转成pytorch格式,作者已转换的[权重](https://huggingface.co/Tongjilibo/uie-base)
- finetune.sh:数据处理和训练全流程
- finetune_step1_dataprocess.py:数据预处理1
- finetune_step2_doccano.py:数据预处理2
Expand Down
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@

setup(
name='bert4torch',
version='v0.4.9.post2',
version='v0.5.0',
description='an elegant bert4torch',
long_description=long_description,
long_description_content_type="text/markdown",
license='MIT Licence',
url='https://github.com/Tongjilibo/bert4torch',
author='Tongjilibo',
install_requires=['numpy', 'tqdm', 'torch>1.6', 'torch4keras==0.2.1.post2', 'six'],
install_requires=['numpy', 'tqdm', 'torch>1.6', 'torch4keras==0.2.2', 'six'],
packages=find_packages()
)

0 comments on commit a181533

Please sign in to comment.