Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Why .mlpackage format no need the vocab.json & merges.txt file? #274

Open
TimYao18 opened this issue Oct 3, 2023 · 3 comments
Open

Why .mlpackage format no need the vocab.json & merges.txt file? #274

TimYao18 opened this issue Oct 3, 2023 · 3 comments

Comments

@TimYao18
Copy link

TimYao18 commented Oct 3, 2023

Hi all,

I want to know why the vocab.json and merges.txt are the same ones from the converting process?
If we use a different CLIP model or Text Encoder model, shouldn't it be different?
@TimYao18
Copy link
Author

TimYao18 commented Oct 5, 2023 •
edited

Its about the transformer.
huggingface/transformers#1083

@TimYao18 TimYao18 closed this as completed Oct 5, 2023
@TimYao18
Copy link
Author

TimYao18 commented Nov 6, 2023

Sorry but I use this issue for questioning:

Why .mlpackage format no need the vocab.json & merges.txt file?

I found .mlmodelc and diffusers format both need these files but .mlpackage didn't.

@TimYao18 TimYao18 reopened this Nov 6, 2023
@TimYao18 TimYao18 changed the title Vocab.json and merges.txt for .mlmodelc Why .mlpackage format no need the vocab.json & merges.txt file? Nov 6, 2023
@jrittvo
Copy link

jrittvo commented Nov 9, 2023

It is possible that those files are stored inside the text encoder piece of the .mlpackage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@TimYao18 @jrittvo