Neuron View Inconsistency

[mojivalipour]

As you can see in the following example, the neuron view output is different than what you've provided in the readme of this repo. Please also check this one.

The current output:

The demanded output:

[jessevig]

Okay, I see. You actually need to click the plus sign that comes up when you hover over the tokens on the left. I'll make that more clear in the README.

[mojivalipour]

Cool, this is a nice feature, but why you don't integrate all of these things into one? It's really a useful library for attention models, but there are some integration and documentation problems in the current version, which make it unsuitable for mass usage.

[jessevig]

Thanks. I'm working on expanding the documentation.Do you have a specific suggestion for a different integration? The reason for the difference between the neuron view and the other views is that the neuron view needs access to the queries and keys, which are not available in the current version of transformers. I don't want to have the head view and model view rely on the forked version of transformers because this creates a dependency that needs to be updated with each new release of transformer models. The head view and model view are the more commonly used views.

[mojivalipour]

Yes, I see why you did this, but I think HuggingFace (@huggingface) are friendly if you raise an issue and inform them about your technology. I think it is good for them to use and integrate your repo into theirs. Their repo, which is based on Transformers, suffered from the lack of this kind of visualization support. This solution is the best idea if they (@clmnt, @julien-c, @LysandreJik, @n1t0, @Pierrci, @thomwolf, and @VictorSanh) cooperate.