Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically upcasting GGUF values #2243

Closed
EricLBuehler opened this issue Jun 3, 2024 · 2 comments
Closed

Automatically upcasting GGUF values #2243

EricLBuehler opened this issue Jun 3, 2024 · 2 comments

Comments

@EricLBuehler
Copy link
Member

This references EricLBuehler/mistral.rs#375, where it was discovered that models may have u32 or u64 as GGUF metadata values. A fix for this is to manually check and then convert the types. However, I thought it may be better to, in the GGUF Value::to_*() code, detect when a subtyping relationship can be used. For example, we can automatically upcast a u32 to a u64.

This should be relatively easy to implement. However, my only concern is that this sort of hidden behavior may be confusing, and we actually want to detect an explicit mismatch?

@LaurentMazare
Copy link
Collaborator

I think it's fine to have the different Value::to_*() be flexible, e.g. to_u64 be ok with u32, to_i64 be ok with i32 and uu32 etc as long as the conversion cannot lose data. If we start feeling that it could be an issue, we could introduce some to_u64_strict or whatever variant but I doubt that this will be required. Feel free to make a PR for this (ideally with a change as minimal as possible so that it will be easy to review).

@EricLBuehler
Copy link
Member Author

Closed by #2244.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants