OneTrainerWindowsRocmProfile.cs currently installs bitsandbytes from
0xDELUXA/bitsandbytes_win_rocm release 0.50.0.dev0-py3.12-rocm7.15-win_amd64_all.
OneTrainer pins bitsandbytes==0.49.1 and doesn't support 0.50 yet: 0.50 removed the block_wise= and percentile_clipping= arguments from all optimizer constructors, which OneTrainer still passes. Any training run using a bitsandbytes optimizer (AdamW8bit etc.) fails at startup:
File "...\modules\util\create.py", line 251, in create_optimizer
optimizer = bnb.optim.AdamW8bit(
TypeError: AdamW8bit.__init__() got an unexpected keyword argument 'percentile_clipping'
Details on the API break and the planned OneTrainer-side upgrade are here:
Nerogar/OneTrainer#1519 (comment)
This looks like it started with the July 15 wheel bump ("Update bitsandbytes wheel for ROCm 7.13/7.14/7.15 compatibility") — the previously used 0.49-based wheel wouldn't have hit this.
Until OneTrainer supports 0.50, could the profile pin a 0.49-based wheel instead? The same repo has v0.49.2.dev0-py312-rocm7.12 — though it's built against ROCm 7.12 rather than 7.15, so it may not fit the torch build the profile installs. If no compatible 0.49 wheel exists for ROCm-on-Windows, it might be worth noting in the profile that bnb optimizers are currently broken there; OneTrainer will catch up once the 0.50 migration lands.
Drafted by Claude
OneTrainerWindowsRocmProfile.cscurrently installs bitsandbytes from0xDELUXA/bitsandbytes_win_rocmrelease0.50.0.dev0-py3.12-rocm7.15-win_amd64_all.OneTrainer pins
bitsandbytes==0.49.1and doesn't support 0.50 yet: 0.50 removed theblock_wise=andpercentile_clipping=arguments from all optimizer constructors, which OneTrainer still passes. Any training run using a bitsandbytes optimizer (AdamW8bit etc.) fails at startup:Details on the API break and the planned OneTrainer-side upgrade are here:
Nerogar/OneTrainer#1519 (comment)
This looks like it started with the July 15 wheel bump ("Update bitsandbytes wheel for ROCm 7.13/7.14/7.15 compatibility") — the previously used 0.49-based wheel wouldn't have hit this.
Until OneTrainer supports 0.50, could the profile pin a 0.49-based wheel instead? The same repo has
v0.49.2.dev0-py312-rocm7.12— though it's built against ROCm 7.12 rather than 7.15, so it may not fit the torch build the profile installs. If no compatible 0.49 wheel exists for ROCm-on-Windows, it might be worth noting in the profile that bnb optimizers are currently broken there; OneTrainer will catch up once the 0.50 migration lands.Drafted by Claude