Support for Chroma - Flux1 Schnell distilled with CFG #7355
Conversation
silveroxides
requested review from
Kosinkadink,
huchenlei,
ltdrdata,
pythongosssss,
robinjhuang,
webfiltered and
yoland68
as code owners
|
+1, native support for chroma would be great. |
|
It would be wonderful if official support made it easier for more people to use it. |
|
At the moment I only use the Chroma model and it would be quite nice with official support. |
|
Would like to see this model with official support. One of my go to models and very versatile. |
|
+1 |
|
+1 |
|
+1 agree, it would be awesome to have native Chroma support! |
|
Chroma is very promising, and very good so far. Please integrate this. |
city96
left a comment
city96
left a comment
There was a problem hiding this comment.
I was just messing with this on my local fork, decided to leave some comments on parts that could (probably) be simplified.
I'm not a part of the core comfy team so feel free to ignore lol.
|
|
||
| NOISE_LEVELS = {"FLUX": [0.9968, 0.9886, 0.9819, 0.975, 0.966, 0.9471, 0.9158, 0.8287, 0.5512, 0.2808, 0.001], | ||
| "Wan":[1.0, 0.997, 0.995, 0.993, 0.991, 0.989, 0.987, 0.985, 0.98, 0.975, 0.973, 0.968, 0.96, 0.946, 0.927, 0.902, 0.864, 0.776, 0.539, 0.208, 0.001], | ||
| "Chroma": [0.9919999837875366, 0.9900000095367432, 0.9879999756813049, 0.9850000143051147, 0.9819999933242798, 0.9779999852180481, 0.9729999899864197, 0.9679999947547913, 0.9610000252723694, 0.953000009059906, 0.9430000185966492, 0.9309999942779541, 0.9169999957084656, 0.8999999761581421, 0.8809999823570251, 0.8579999804496765, 0.8320000171661377, 0.8019999861717224, 0.7689999938011169, 0.7310000061988831, 0.6899999976158142, 0.6460000276565552, 0.5989999771118164, 0.550000011920929, 0.5009999871253967, 0.45100000500679016, 0.4020000100135803, 0.35499998927116394, 0.3109999895095825, 0.27000001072883606, 0.23199999332427979, 0.19900000095367432, 0.16899999976158142, 0.14300000667572021, 0.11999999731779099, 0.10100000351667404, 0.08399999886751175, 0.07000000029802322, 0.057999998331069946, 0.04800000041723251, 0.0], |
There was a problem hiding this comment.
This is probably overkill, you can round it down to make it more readable to match the rest. i.e
[round(x, 5) for x in steps]
[0.992, 0.99, 0.988, 0.985, 0.982, 0.978, 0.973, 0.968, 0.961, 0.953, 0.943, 0.931, 0.917, 0.9, 0.881, 0.858, 0.832, 0.802, 0.769, 0.731, 0.69, 0.646, 0.599, 0.55, 0.501, 0.451, 0.402, 0.355, 0.311, 0.27, 0.232, 0.199, 0.169, 0.143, 0.12, 0.101, 0.084, 0.07, 0.058, 0.048, 0.0]| out['image_cond'] = comfy.conds.CONDNoiseShape(self.process_latent_in(image_cond)) | ||
| return out | ||
|
|
||
| class Chroma(BaseModel): |
There was a problem hiding this comment.
I think you can just inherit from Flux instead of BaseModel here.
The only difference seems to be "guidance" being set to 0.0 instead of 3.5 in extra conds. For that you can do sth like:
def extra_conds(self, **kwargs):
out = super().extra_conds(**kwargs)
guidance = 0.0
out['guidance'] = comfy.conds.CONDRegular(torch.FloatTensor((guidance,)))
return out| q = q.float().reshape(*q.shape[:-1], -1, 1, 2) | ||
| k = k.float().reshape(*k.shape[:-1], -1, 1, 2) |
There was a problem hiding this comment.
Any reason to cast these + the one in apply_rope below to float? This part could be reused from comfy/ldm/flux/math.py if not, since it seems to be the same otherwise.
| key_map["base_model.model.{}".format(key_lora)] = k #official hunyuan lora format | ||
|
|
||
| if isinstance(model, comfy.model_base.Flux): #Diffusers lora Flux | ||
| if isinstance(model, comfy.model_base.Flux) or isinstance(model, comfy.model_base.Chroma): #Diffusers lora Flux or a diffusers lora Chroma |
There was a problem hiding this comment.
This is probably cleaner, I think it works even on older versions, dunno which one is preferred...
isinstance(model, (comfy.model_base.Flux, comfy.model_base.Chroma))| class T5XXLTokenizer(sd1_clip.SDTokenizer): | ||
| def __init__(self, embedding_directory=None, tokenizer_data={}): | ||
| tokenizer_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "t5_tokenizer") | ||
| super().__init__(tokenizer_path, embedding_directory=embedding_directory, pad_with_end=False, embedding_size=4096, embedding_key='t5xxl', tokenizer_class=T5TokenizerFast, has_start_token=False, pad_to_max_length=False, max_length=99999999, min_length=1, tokenizer_data=tokenizer_data) |
There was a problem hiding this comment.
Any particular reason for min_length=1? The only other model that needed that was pixart iirc
12 participants
Chroma is a 8.9B parameter model based on FLUX.1-schnell.
It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping.
The model is still training right now, and I’d love to hear your thoughts! Your input and feedback are really appreciated.
What Chroma Aims to Do
Training on a 5M dataset, curated from 20M samples including anime, furry, artistic stuff, and photos.
Fully uncensored, reintroducing missing anatomical concepts.
Built as a reliable open-source option for those who need it.
See the Progress
Hugging Face Repo: https://huggingface.co/lodestones/Chroma
Hugging Face Debug Repo: https://huggingface.co/lodestones/chroma-debug-development-only
Live WandB Training Logs: https://wandb.ai/lodestone-rock/optimal%20transport%20unlocked
ComfyUI Inference node [WIP]: https://github.com/lodestone-rock/flux-mod
Training code!: https://github.com/lodestone-rock/flow
CivitAi gallery: https://civitai.com/posts/13766416
CivitAi model: [soon]
Support Open-Source AI
The current pretraining run has already used 5000+ H100 hours, and keeping this going long-term is expensive.
If you believe in accessible, community-driven AI, any support would be greatly appreciated.
👉 ko-fi link — Every bit helps!
Reddit thread at r/StableDiffusion
