Question about: `ConfigMixin` vs `XYZConfig class`

[mishig25]

I see that ConfigMixin & @register_to_config are used to handle model configs:

 class UNet2DConditionModel(ModelMixin, ConfigMixin): 
     @register_to_config

# there is no `UNet2DConfig` class (i.e. XYZConfig class)

comparing it to transformers (for example here), there are explicit config classes such as BertConfig that inherit from PretrainedConfig

class BertPreTrainedModel(PreTrainedModel):
    config_class = BertConfig

Why was ConfigMixin approach preferred over the XYZConfig class?

Thanks a lot! 🤗

[patrickvonplaten]

Good question! Think you can find most of the answers here:
#11

For me the big thing is the following:

By forcing the init to be == the configuration of the model class, we have a safety mechanism to not add parameters to model configs that shouldn't be there because the don't contribute to "instantiating" the model. E.g. in transformers we erroneously added some things like max_length for text-generation or use_gradient_checkpointing which are not related to training the model or configuring a model, but just for inference => the current design prevents us from adding such things to the config because they make no sense in a model config really

[mishig25]

oh I see 😊
(closing the issue as it was previously discussed #11)