Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add options to not use bias in Linear (and Conv in future) #100

Closed
Tracked by #278
coreylowman opened this issue Jul 20, 2022 · 3 comments · Fixed by #494
Closed
Tracked by #278

Add options to not use bias in Linear (and Conv in future) #100

coreylowman opened this issue Jul 20, 2022 · 3 comments · Fixed by #494

Comments

@coreylowman
Copy link
Owner

I think this code be done by implementing Linear as a tuple instead: (MatMul<I, O>, Add<O>), and then if someone doesn't want bias, they can just MatMul<I, O> instead.

However, this would break linear's .npz compatibility with pytorch.

@coreylowman
Copy link
Owner Author

Maybe a better option is to include bias as a const BIAS: bool = true generic param.

@jafioti
Copy link
Contributor

jafioti commented Jul 20, 2022

I think having a Linear module is much better than just using a tuple. I feel like if you specifically want no bias, then just use MatMul<I, O>

@M1ngXU
Copy link
Contributor

M1ngXU commented Jul 21, 2022

I think having a Linear module is much better than just using a tuple. I feel like if you specifically want no bias, then just use MatMul<I, O>

i think it's something like this
type Linear<I, O> = (MatMul<I, O>, Add<O>)

(and maybe type LinearNoBias<I, O> = MatMul<I, O>)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants