Is there any prospect of adding the lance model from ByteDance as one of the supported models in stable-diffusion.cpp? #1763
felipedilho
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I know that this must be quite an herculean task, since it would mean having to synergize the image generation, image understanding, video generation, video understanding (?) and even text output, from llama.cpp, frameworks to a single model. I don't know if this is why they require an 40+Gb VRAM system to run the model, even if it only has 3 billion parameters. Or even if this requirement can be lowered. But being able to have a model as flexible as that would be quite interesting and useful, even if it only has a proof of concept, experimental quality. Like having a (possibly) local nano banana, and the possibility of training Lora's for it could be even more interesting, since I think that people doing fine-tunes and loras are really where useful things are made. It is way more difficult to try to steer a private model like nano-banana to producing what you want only using prompts than it is to train a lora for a "worse" model like SDXL to at least give you a good approximation of what you would have liked to obtain. I know that this is somewhat a legal grey area, but I would like to test if I could train a lora for Lance with pixel art spread sheets to try to see if I could get it to generate other postures for a given pixel art sprite in the same style as a given base image of the sprite, since getting other postures for pixel art is rather difficult by hand.
Beta Was this translation helpful? Give feedback.
All reactions