Skip to content

first FC layer is missing for time_emb generation #11

@ishan-modi

Description

@ishan-modi

self.t_emb_layers = nn.ModuleList([ nn.Sequential( nn.SiLU(), nn.Linear(t_emb_dim, out_channels) ) for _ in range(num_layers) ])

shouldn't it be following instead

self.t_emb_layers = nn.ModuleList([ nn.Sequential( nn.Linear(t_emb_dim, out_channels), nn.SiLU(), nn.Linear(out_channels, out_channels) ) for _ in range(num_layers) ])

I checked unet_2d.py from huggingface and it also has 2 linear FC layers, let me know if I am missing something.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions