Skip to content

Fix training issues and compatability with transformers v4.x and v5#326

Merged
Ingvarstep merged 8 commits intomainfrom
fix/training
Jan 29, 2026
Merged

Fix training issues and compatability with transformers v4.x and v5#326
Ingvarstep merged 8 commits intomainfrom
fix/training

Conversation

@Ingvarstep
Copy link
Copy Markdown
Collaborator

@Ingvarstep Ingvarstep commented Jan 29, 2026

The following issues were fixed:

  1. Incorrect loading of spm.model tokenizers in the case of transformers v5;
  2. Data processing issues for token-level labels preparation;
  3. Trainer compatibility between v4.x and v5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant