Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
how does one get attention values out of the predictions? #143
thanks for this library! i've managed to try an simple model and i'm excited to do more.
i'm using is config:
and following the tutorial i can train and get predictions. awesome!
but how do i find the attention?
it doesn't appear to be anywhere in the results of
thanks for any help!
i've now gotten to the point that i've looked at the shapes of all the numpy arrays in
am i missing something here?
near as i can tell, attention is turned on in my model...
Hey, we've only recently gotten this as a feature request, so we haven't really done anything explicit for making the attention weights exported. You're on the right track with looking in the caffe2 workspace blobs as the quickest way to get this to actually work without making code changes, although I can't give you a great answer about where these will be in the workspace blobs as I haven't had time to look into it yet.
If you do something like this: https://github.com/facebookresearch/pytext/blob/master/pytext/__init__.py#L80
It's also possible that your model is not learning something particularly explainable from the attention inputs which is why the values are confusing; I'm not a particularly good person to answer whether this is likely to be valuable, but I've heard claims both ways.
thanks for that tip @bethebunny ; using the graph i was able to figure out which tensor it was
(i picked node 110).
i'm happy with this hack for now, but also happy to be directed on how to add this into the codebase via a pr so that it's much nicer in the future :)