Model explainability is an important component of Responsible AI. The goal of the tutorial is to penetrate the opacity of the “black box” and allow Data Scientists, and even users, to understand why a decision was made by a model. While Researchers that call for Fairness and Transparency rank model transparency high on their list of Responsible AI objectives, LLMs are extremely large, have complex deep learning architecture and are particularly opaque with non-deterministic weights that lead to frequently non-uniform output that can be hard to understand. As Large Language Models increase in popularity and use cases, it is more important now than ever that Data Scientists learn techniques to demystify the black box and provide a level of understanding behind the “why” of a model’s output. We will explore some common strategies used to explain LLMs, situations where they work best and their limitations. The target audience is Responsible AI Practitioners and people interested in how Responsible AI can be applied to LLMs. While it would be beneficial to have a background in NLP and LLMs, it is certainly not required to understand and leverage the tutorial. Description and Outline:
The tutorial will cover-
Heatmaps and Visualization for word embeddings
Visualization of Model Parameters
Model Cards for Large Language Models and Explainability Tools
Parity Model Charts
Comparison of Counterfactual Models
The contents of the tutorial will be shown in a notebook that will be a part of a larger open-source repository of LLM Explainability and Visualization.
The success of commercial Generative AI has renewed widespread interest in Large Language Models. While this renewed interest has been met with increased funding to push the boundaries of this technology, there have been calls by prominent AI practitioners to stop advancement in this space due to fear of potential harms. These parties frequently raise concerns around model opacity. By using LLM explainability tools, their concerns can be potentially assuaged by offering visibility into model decision making. Alternatively, there are groups of people who are interested in accelerating progress in this space. Explainability tools can offer those parties a way to mitigate harms brought about by rapid progress - allowing them to more safely improve their models.
In 2018, OpenAI released a paper showing that advancements in AI have eclipsed Moore’s Law’s two year doubling time by 7x. This means that every 3.4 months, AI advancements roughly double. If the past year has been any indication, LLMs are advancing at particularly blinding speeds. Due to this aggressive scaling timeline of LLMs, we must meet that demand with an equally measured effort toward the advancement of explainability tools, lest the gap between responsible AI and technological advancement continue to widen.
These tutorials and samples look to build upon that literature using visualization methods while serving as a call to action for more researchers to work in this space.