gemma3-270m: reduce batch size for sample packing

[djsaunde]

Quoting myself:

[...] with sample packing, we always fill out the full batch_size * seq_length (or nearly so). Without it, we might not hit nearly this, since trl truncates batches to the longest sequence in the batch, so if the batch doesn't have a max length sequence in it, we can train for a while without OOMing. Hopefully that makes sense

This applies to the Gemma3-270M notebook in particular; hence the change from batch size 8 -> 4 here.

To be merged with unslothai/unsloth#3566.

[gemini-code-assist]

Summary of Changes

Hello @djsaunde, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses potential Out-Of-Memory issues during the training of the Gemma3-270M model, particularly when using sample packing. By halving the per_device_train_batch_size in both the Python script and Jupyter notebook configurations, the change aims to ensure more stable and reliable training runs without encountering memory constraints.

Highlights

• Batch Size Reduction: The per_device_train_batch_size has been reduced from 8 to 4 in the Gemma3-270M training configurations.
• OOM Prevention: This batch size reduction specifically targets the Gemma3-270M model to prevent Out-Of-Memory (OOM) errors, especially when sample packing is utilized, as sample packing ensures full batch utilization.
• Notebook Execution Count: The execution_count for a code cell in the Gemma3_(270M).ipynb notebook has been reset to null.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Summary of Changes

Hello @djsaunde, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses potential Out-of-Memory (OOM) issues encountered during the training of the Gemma3-270M model, particularly when sample packing is enabled. By halving the training batch size, the changes aim to reduce the memory footprint during model training, thereby preventing OOM errors and ensuring more stable and successful training runs for this specific model configuration.

Highlights

• Batch Size Reduction: The per_device_train_batch_size has been reduced from 8 to 4 for the Gemma3-270M model's training configuration. This change is applied in both the Jupyter notebook and its corresponding Python script.
• Notebook Metadata Update: The execution_count metadata field in the Gemma3_(270M).ipynb notebook has been reset to null.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request reduces the per_device_train_batch_size from 8 to 4 for the Gemma3-270M model to prevent out-of-memory errors. This is a sensible change. My review includes a suggestion to also increase gradient_accumulation_steps from 1 to 2 in both the notebook and the Python script. This would maintain the original effective batch size of 8, which is often beneficial for training stability, while still achieving the goal of lower memory usage per step.

[gemini-code-assist]

Code Review

This pull request reduces the per_device_train_batch_size from 8 to 4 for the Gemma3-270M model in both the Jupyter notebook and the corresponding Python script. This change is intended to prevent out-of-memory errors when using sample packing. My review suggests an improvement: to maintain the original effective batch size of 8, you could also update gradient_accumulation_steps to 2. This would preserve the training dynamics while still achieving the memory reduction benefits.