Skip to content

feat: keep tokenizer assets v4-compatible#1465

Merged
akoumpa merged 5 commits intomainfrom
akoumparouli/feat_v4_compatible_tokenizer
Mar 6, 2026
Merged

feat: keep tokenizer assets v4-compatible#1465
akoumpa merged 5 commits intomainfrom
akoumparouli/feat_v4_compatible_tokenizer

Conversation

@akoumpa
Copy link
Copy Markdown
Contributor

@akoumpa akoumpa commented Mar 5, 2026

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Changelog

  • Add specific line by line info of high level changes in this PR.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

  • Related to # (issue)

@akoumpa
keep tokenizer assets v4-compatible
89076c3 
Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Mar 5, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@akoumpa akoumpa added the r0.3.0 Add for cherry-pick into release branch r0.3.0 label Mar 5, 2026
@akoumpa
fix tokenizer compatibility with v4
9ae4f61 
Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
@akoumpa
Copy link
Copy Markdown
Contributor Author

akoumpa commented Mar 6, 2026

/ok to test 9ae4f61

@akoumpa
update test
5d492e1 
Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
@akoumpa
Copy link
Copy Markdown
Contributor Author

akoumpa commented Mar 6, 2026

/ok to test 5d492e1

@akoumpa
skip saving config.json
5b26d8c 
Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
@akoumpa
Copy link
Copy Markdown
Contributor Author

akoumpa commented Mar 6, 2026

/ok to test 5b26d8c

@akoumpa akoumpa marked this pull request as ready for review March 6, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HuiyingLi HuiyingLi HuiyingLi approved these changes

@adil-a adil-a Awaiting requested review from adil-a adil-a is a code owner

@hemildesai hemildesai Awaiting requested review from hemildesai hemildesai is a code owner

@ZhiyuLi-Nvidia ZhiyuLi-Nvidia Awaiting requested review from ZhiyuLi-Nvidia ZhiyuLi-Nvidia is a code owner

Assignees

No one assigned

Labels

r0.3.0 Add for cherry-pick into release branch r0.3.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@akoumpa @HuiyingLi