-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
H3 / LongConvKernel - l_max=None isn't working #2
Comments
The H3Conv replaces the SSM with a fixed-length long convolution the size
of the input, so you need to set the l_max.
The SSM version of H3 has no fixed context length, so that’s why it’s not
necessary for that case.
…On Thu, Mar 9, 2023 at 9:08 AM Alexey Korepanov ***@***.***> wrote:
In H3 model here noted that
https://github.com/HazyResearch/safari/blob/1ff064eb43a455e217449314dfbbd83dcac7871c/src/models/sequence/h3_conv.py#L36
But it isn't working, because of
https://github.com/HazyResearch/safari/blob/1ff064eb43a455e217449314dfbbd83dcac7871c/src/models/sequence/long_conv_kernel.py#L56
Which is leading to torch.randn(int, int, None) error:
TypeError: randn(): argument 'size' must be tuple of ints, but found element of type NoneType at pos 3
So, are global kernels supported now? Or, to make it more global, should I
use the large l_max value?
—
Reply to this email directly, view it on GitHub
<#2>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABDDIISS53RDBV3INEJGOATW3HI3HANCNFSM6AAAAAAVVDARFQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
ok thank! |
Thanks, this feedback makes sense! Will add some more notes and
documentation next week (currently traveling).
…On Thu, Mar 9, 2023 at 4:04 PM Alexey Korepanov ***@***.***> wrote:
ok, it would be clearer if there was an assert or something like that.
this would be helpful for users like me who use just the model
implementation without any other framework things
—
Reply to this email directly, view it on GitHub
<#2 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABDDIITA4YQFTXTWTM2FXKTW3IZWJANCNFSM6AAAAAAVVDARFQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In H3 model here noted that
safari/src/models/sequence/h3_conv.py
Line 36 in 1ff064e
But it isn't working, because of
safari/src/models/sequence/long_conv_kernel.py
Line 56 in 1ff064e
Which is leading to
torch.randn(int, int, None)
error:So, are global kernels supported now? Or, to make it more global, should I use large
l_max
value?The text was updated successfully, but these errors were encountered: