Skip to content

[WIP][Ernie 4.5] Add ernie models #39228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 36 commits into
base: main
Choose a base branch
from
Draft

[WIP][Ernie 4.5] Add ernie models #39228

wants to merge 36 commits into from
+3,378 −1

Conversation

vasqu
Copy link
Contributor

@vasqu vasqu commented Jul 4, 2025 •
edited
Loading

Adding the Ernie 4.5 suite of models.

Progress:

  • Ernie 4.5 pure text model (0.3B)
  • Integration test ^ (needs slow runs to cross check)
  • MoE Ernie
  • MoE Ernie VL
  • Check whether the MoE also need a rotation conversion (0.3B modeling files differ to the other ones regarding RoPE)
    • Yes, they do - turns out they use a similar trick as I did in 393c2c7
  • Fixup tokenization
  • Docs

@huggingface huggingface deleted a comment from github-actions bot Jul 4, 2025
@vasqu
Copy link
Contributor Author

vasqu commented Jul 7, 2025

run-slow: ernie4_5

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 7, 2025

This comment contains run-slow, running the specified jobs:

models: ['models/ernie4_5']
quantizations: [] ...

@ArthurZucker
Copy link
Collaborator

Let's go! 🚀

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 8, 2025

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, ernie4_5, ernie4_5_moe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
New model
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vasqu @ArthurZucker