Skip to content

Fix samples, LoRA training. Add system prompt, use_flash_attn #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sdbds merged 6 commits into from
Feb 24, 2025
Merged

Fix samples, LoRA training. Add system prompt, use_flash_attn #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
025cca6
Fix samples, LoRA training. Add system prompt, use_flash_attn
rockerBOO Feb 23, 2025
6d7bec8
Remove non-used code
rockerBOO Feb 23, 2025
42a8015
Fix system prompt in datasets
rockerBOO Feb 23, 2025
ba725a8
Set default discrete_flow_shift to 6.0. Remove default system prompt.
rockerBOO Feb 23, 2025
48e7da2
Add sample batch size for Lumina
rockerBOO Feb 24, 2025
2c94d17
Fix typo
rockerBOO Feb 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions library/config_util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ class BaseSubsetParams:
custom_attributes: Optional[Dict[str, Any]] = None
validation_seed: int = 0
validation_split: float = 0.0
system_prompt: Optional[str] = None


@dataclass
Expand Down Expand Up @@ -106,6 +107,7 @@ class BaseDatasetParams:
debug_dataset: bool = False
validation_seed: Optional[int] = None
validation_split: float = 0.0
system_prompt: Optional[str] = None


@dataclass
Expand Down Expand Up @@ -196,6 +198,7 @@ def __validate_and_convert_scalar_or_twodim(klass, value: Union[float, Sequence]
"caption_prefix": str,
"caption_suffix": str,
"custom_attributes": dict,
"system_prompt": str,
}
# DO means DropOut
DO_SUBSET_ASCENDABLE_SCHEMA = {
Expand Down Expand Up @@ -241,6 +244,7 @@ def __validate_and_convert_scalar_or_twodim(klass, value: Union[float, Sequence]
"validation_split": float,
"resolution": functools.partial(__validate_and_convert_scalar_or_twodim.__func__, int),
"network_multiplier": float,
"system_prompt": str,
}

# options handled by argparse but not handled by user config
Expand Down Expand Up @@ -526,6 +530,7 @@ def print_info(_datasets, dataset_type: str):
batch_size: {dataset.batch_size}
resolution: {(dataset.width, dataset.height)}
enable_bucket: {dataset.enable_bucket}
system_prompt: {dataset.system_prompt}
""")

if dataset.enable_bucket:
Expand Down Expand Up @@ -559,6 +564,7 @@ def print_info(_datasets, dataset_type: str):
token_warmup_step: {subset.token_warmup_step},
alpha_mask: {subset.alpha_mask}
custom_attributes: {subset.custom_attributes}
system_prompt: {subset.system_prompt}
"""), " ")

if is_dreambooth:
Expand Down
Loading