You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TDB enables rapid exploration before needing to write code, with the ability to intervene in the forward pass and see how it affects a particular behavior. It can be used to answer questions like, "Why does the model output token A instead of token B for this prompt?" or "Why does attention head H attend to token T for this prompt?" It does so by identifying specific components (neurons, attention heads, autoencoder latents) that contribute to the behavior, showing automatically generated explanations of what causes those components to activate most strongly, and tracing connections between components to help discover circuits.
Other Information
This tool could be a very great guide to people working with the interpretability of LLM models. There are already a lot of LLM models in Keras-nlp and engineers might find it very useful while working on the deployment of the models to ensure the safety, reliability, intrepretability and control of the LLM models available here.
The text was updated successfully, but these errors were encountered:
Thanks for the suggestion! How do you envision Transformer Debugger to be incorporated into KerasNLP. Does it require the integration with a tool in our library? Or do we just need to create a guide?
Note that there is an on-going effort to integrate Learning Interpretability Tool (LIT) with KerasNLP. #1521 is an example of adding the .score() function for interpretability use cases.
Thanks for the suggestion! How do you envision Transformer Debugger to be incorporated into KerasNLP. Does it require the integration with a tool in our library? Or do we just need to create a guide?
Note that there is an ongoing effort to integrate Learning Interpretability Tool (LIT) with KerasNLP. #1521 is an example of adding the .score() function for interpretability use cases.
Thank you for pointing that out. I'm not an expert here, but LIT sounds like a very general approach while TDB is a very specific LLM approach. TDB could be a very long and lengthy feature to implement, so I'm all up to contribute in case there is any need.
Short Description
Paper
https://arxiv.org/pdf/2211.00593v1.pdf
Existing Implementations
Other Information
This tool could be a very great guide to people working with the interpretability of LLM models. There are already a lot of LLM models in Keras-nlp and engineers might find it very useful while working on the deployment of the models to ensure the safety, reliability, intrepretability and control of the LLM models available here.
The text was updated successfully, but these errors were encountered: