Add llama training support

[andreaskoepf]

• further work based on new-format-experiment branch

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[theblackcat102]

@andreaskoepf @sanagno After merging conflict from main, the pre-commit behaving very weird, my local precommit check was fine, but github action was not happy

index f125a3d..434066b 100755
--- a/model/model_training/tools/model_chat.py
+++ b/model/model_training/tools/model_chat.py
@@ -19,9 +19,8 @@ if __name__ == "__main__":
 
     sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
 from tokenizers import pre_tokenizers
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
 
 class ChatRole(str, Enum):
diff --git a/model/model_training/tools/model_cli.py b/model/model_training/tools/model_cli.py
index a3fac77..b89e4c0 100755
--- a/model/model_training/tools/model_cli.py
+++ b/model/model_training/tools/model_cli.py
@@ -6,9 +6,8 @@ import torch
 import transformers
 from custom_datasets.formatting import QA_SPECIAL_TOKENS, format_pairs, format_system_prefix
 from models import get_specific_model
-from utils import _strtobool
-
 from tokenizers import pre_tokenizers
+from utils import _strtobool
 
 if __name__ == "__main__":
     import warnings
diff --git a/model/model_training/utils.py b/model/model_training/utils.py
index 7a710ee..92eccc4 100644
--- a/model/model_training/utils.py
+++ b/model/model_training/utils.py
@@ -15,11 +15,10 @@ from models import freeze_top_n_layers, get_specific_model
 from models.reward_model import RewardModel, RewardModelConfig
 from models.tokenization_llama import LLaMATokenizer
 from sklearn.model_selection import train_test_split
+from tokenizers import pre_tokenizers
 from torch.utils.data import ConcatDataset, Subset
 from torch.utils.data.distributed import DistributedSampler
 
-from tokenizers import pre_tokenizers
-
 
 def _strtobool(x):
     return bool(strtobool(x))
diff --git a/model/reward/instructor/utils.py b/model/reward/instructor/utils.py
index d1fcc2a..1e97df9 100644
--- a/model/reward/instructor/utils.py
+++ b/model/reward/instructor/utils.py
@@ -3,11 +3,10 @@ from typing import AnyStr, List
 
 import yaml
 from sklearn.model_selection import train_test_split
+from tokenizers import pre_tokenizers
 from torch.utils.data import Subset
 from transformers import AutoTokenizer, T5Tokenizer
 
-from tokenizers import pre_tokenizers
-

these difference are simply just change of import line?

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md