Skip to content

Commit

Permalink
Merge pull request #598 from PAIR-code:website-0_4_1
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 417471993
  • Loading branch information
LIT team committed Dec 20, 2021
2 parents f347fad + 30eb119 commit 9fd2da3
Show file tree
Hide file tree
Showing 42 changed files with 33 additions and 33 deletions.
Binary file modified docs/assets/images/actor_to_actress.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/actress_relative_cav.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-coref-compare.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-coref-data.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-coref-metric-top.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-coref-metrics.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-coref-pred.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-coref-select.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-datatable-search.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-metrics-not.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-not-saliency.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-saliency.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-sim-search.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-t5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-toolbars.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit-tweet.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/assets/images/lit-workspaces.jpg
Binary file not shown.
Binary file added docs/assets/images/lit-workspaces.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit_data_table_annotated.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit_slice_editor_annotated.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit_tcav_screen_annotated.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/lit_tcav_settings_annotated.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/tcav_result.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/images/tcav_results_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/demos/coref.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<html>
<script>
window.location.replace("http://35.225.219.27/");
window.location.replace("http://34.86.17.245:5431/");
</script>
</html>
2 changes: 1 addition & 1 deletion docs/demos/glue.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<html>
<script>
window.location.replace("http://34.69.145.124/");
window.location.replace("http://34.86.17.245:5432/");
</script>
</html>
2 changes: 1 addition & 1 deletion docs/demos/images.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<html>
<script>
window.location.replace("http://104.155.142.190/");
window.location.replace("http://34.86.17.245:5433/");
</script>
</html>
6 changes: 3 additions & 3 deletions docs/demos/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -122,12 +122,12 @@
<div class="demo-card-title"><a href="https://colab.research.google.com/github/PAIR-code/lit/blob/main/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb" target="_blank">Notebook usage</a></div>
<div class="demo-card-tags"> <span class="demo-tag"> BERT </span> <span class="demo-tag"> binary classification </span> <span class="demo-tag"> notebooks </span>
</div>
<div class="demo-card-data-source-title">DATA SOURCES</div>
<div class="demo-card-data-source-title">DATA SOURCE</div>
<div class="demo-card-data-source">
Stanford Sentiment Treebank
<a href="" target="_blank">Stanford Sentiment Treebank</a>
</div>
<div class="demo-card-copy">Use LIT directly inside a Colab notebook. Explore binary classification for sentiment analysis using SST2 from the General Language Understanding Evaluation (GLUE) benchmark suite.</div>
<div class="demo-card-cta-button"><a href="/lithttps://colab.research.google.com/github/PAIR-code/lit/blob/main/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb"></a></div>
<div class="demo-card-cta-button"><a href="https://colab.research.google.com/github/PAIR-code/lit/blob/main/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb"></a></div>
</div>
<div class="demo-card mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--4-col-phone">
<div class="demo-card-title"><a href="/lit/demos/coref.html" target="_blank">Gender bias in coreference systems</a></div>
Expand Down
2 changes: 1 addition & 1 deletion docs/demos/lm.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<html>
<script>
window.location.replace("http://104.197.40.13/");
window.location.replace("http://34.86.17.245:5433/");
</script>
</html>
2 changes: 1 addition & 1 deletion docs/demos/penguins.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<html>
<script>
window.location.replace("http://35.222.78.90/");
window.location.replace("http://34.86.17.245:5435/");
</script>
</html>
2 changes: 1 addition & 1 deletion docs/demos/t5.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<html>
<script>
window.location.replace("http://34.68.143.28/");
window.location.replace("http://34.86.17.245:5436/");
</script>
</html>
4 changes: 2 additions & 2 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@
<li>Does my model behave consistently if I change things like textual style, verb tense, or pronoun gender?</li>
</ul>
<p>LIT contains many built-in capabilities but is also customizable, with the ability to add custom interpretability techniques, metrics calculations, counterfactual generators, visualizations, and more.</p>
<p>LIT also contains preliminary support for non-language models, working with tabular and image data. For a similar tool built to explore general-purpose machine learning models, check out the <a href="https://whatif-tool.dev">What-If Tool</a>.</p>
<p>In addition to language, LIT also includes preliminary support for models operating on tabular and image data. For a similar tool built to explore general-purpose machine learning models, check out the <a href="https://whatif-tool.dev">What-If Tool</a>.</p>
<p>LIT can be run as a standalone server, or inside of python notebook environments such as Colab, Jupyter, and Google Cloud Vertex AI Notebooks.</p>
</div>
<div class="spacer" style="height:50px;"></div>
Expand Down Expand Up @@ -155,7 +155,7 @@ <h3 class="home-card-title">Version 0.4</h3>
</div>
<div class="mdl-cell mdl-cell--6-col mdl-cell--4-col-tablet mdl-cell--8-col-phone">
<img class="home-card-image" src="/lit/assets/images/LIT_Contribute.png"/>
<div class="home-card-action">CODE</div>
<div class="home-card-action">DOCS</div>
<h3 class="home-card-title">Documentation</h3>
<div class="home-card-desc">LIT is open-source and easily extensible to new models, tasks, and more.</div>
<div class="home-card-cta-button"><a href="https://github.com/PAIR-code/lit/wiki" target="_blank" >View documentation</a><img class="external-arrow" src="/lit/assets/images/arrow-link-out.png"/></div>
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/generation/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ <h2>Debugging a Text Generator</h2>
from the T5 decoder. With one click, we retrieve the 25 nearest neighbors to our datapoint of interest from the training set and add them to the LIT UI for inspection. We can see through the search capability in the data table that the words “captain” and “former” appear 34 and 16 times in these examples–along with 3 occurrences of “replaced by” and two occurrences of “by former”. This suggests a strong prior toward our erroneous phrase from the training data most related to our datapoint of interest.</p>
<div class="mdl-cell mdl-cell--12-col mdl-cell--6-col-tablet mdl-cell--4-col-phone">
<img class="tutorial-image" src="/lit/assets/images/lit-datatable-search.png"/>
<div class="tutorial-caption">Above: An example of the existance of "by former" in a similar datapoint from the training set.</div>
<div class="tutorial-caption">Above: An example of the existence of "by former" in a similar datapoint from the training set.</div>
</div>

</div>
Expand Down
6 changes: 3 additions & 3 deletions docs/tutorials/tcav/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,9 @@ <h4>Create a Concept</h4>
<div class="tutorial-caption">1: Selecting a slice as a concept and 2: running TCAV with the specified options</div>
</div>
<h4>Interpreting TCAV scores</h4>
<p>Once we run TCAV, we see an entry in the table in the TCAV module for each concept tested. Each concept gets a CAV (“Concept Activation Vector”) score between 0 and 1 describing the concept’s effect on the prediction of the class in question. What you want to look at is where the blue bar (CAV score) is relative to the black line (reference point). The reference point indicates the effect that slices made of randomly-chosen datapoints outside of the concept being tested has on prediction of the class. For a well-calibrated classifier, the reference point will usually be near 0.5 (i.e. no effect).</p>
<p>A blue bar extending beyond or falling short of the black line means the concept is influencing the prediction. If the blue bar extends beyond the black line, the concept is positively influencing the prediction; if it falls short, it is negatively influencing.</p>
<p>In our example, the CAV score of ~.85 indicates that our “acting” concept has a strong positive effect on the prediction of this class. So we have found that this concept has a positive effect on predicting positive sentiment for our classifier.</p>
<p>Once we run TCAV, we see an entry in the table in the TCAV module for each concept tested. Each concept gets a CAV (“Concept Activation Vector”) score between 0 and 1 describing the concept’s effect on the prediction of the class in question. What matters is where the blue bar (CAV score) is relative to the black line (reference point). The reference point indicates the effect that slices made of randomly-chosen datapoints outside of the concept being tested has on prediction of the class. For a well-calibrated classifier, the reference point will usually be near 0.5 (i.e. no effect).</p>
<p>A blue bar extending right or left of the black line means the concept is influencing the prediction. If the blue bar extends to the right of the black line, the concept is positively influencing the prediction. Conversely, if the bar extended to the left, it is negatively influencing. In either case, the larger the bar, the greater the influence.</p>
<p>In our example, the CAV score of ~0.91 indicates that our “acting” concept has a strong positive effect on the prediction of this class. So we have found that this concept has a positive effect on predicting positive sentiment for our classifier.</p>
<div class="mdl-cell mdl-cell--12-col mdl-cell--6-col-tablet mdl-cell--4-col-phone">
<img class="tutorial-image" src="/lit/assets/images/tcav_result.png"/>
<div class="tutorial-caption">TCAV results for our “acting” concept</div>
Expand Down
6 changes: 3 additions & 3 deletions docs/tutorials/tour/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ <h2>A Quick Tour of the Language Interpretability Tool</h2>
<h3>Building blocks - modules, groups, and workspaces</h3>
<p><strong>Modules, groups, and workspaces</strong> form the building blocks of LIT. Modules are discrete windows in which you can perform a specific set of tasks or analyses. Workspaces display combinations of modules known as groups, so you can view different visualizations and interpretability methods side-by-side.</p>
<div class="mdl-cell mdl-cell--12-col mdl-cell--6-col-tablet mdl-cell--4-col-phone">
<img class="tutorial-image" src="/lit/assets/images/lit-workspaces.jpg"/>
<img class="tutorial-image" src="/lit/assets/images/lit-workspaces.png"/>
<div class="tutorial-caption">Above: Building blocks of the Language Interpretability Tool: (1) Modules, (2) Groups, (3) Static workspace, (4) Group-based workspace.</div>
</div>
<p>LIT is divided into two workspaces - a Main workspace in the upper half of the interface, and a Group-based workspace in the lower half.</p>
Expand All @@ -108,7 +108,7 @@ <h3>Toolbars</h3>
<ul>
<li>Select data points by relationship, or by slice.</li>
<li>Choose a feature to color data points, across all modules.</li>
<li>Track the datapoint you’re looking at, navigate to the next, mark a datapoint as a favorite, or clear your selection.</li>
<li>Track the datapoint you are looking at, navigate to the next, mark a datapoint as a favorite, or clear your selection.</li>
<li>Select the active models and dataset, including multiple models to compare.</li>
</ul>
<div class="mdl-cell mdl-cell--12-col mdl-cell--6-col-tablet mdl-cell--4-col-phone">
Expand All @@ -123,7 +123,7 @@ <h3>Using Modules</h3>
<div class="info-box-title">Visualizations that scale</div>
<div class="info-box-text">Visualizations and results within modules can scale depending on if you're looking at one data point, multiple data points, one or multiple models. For instance, turning on the “compare datapoints” toggle allows you to compare a selected datapoint (known as the reference datapoint) to others by presenting individual results side-by-side within relevant modules.</div>
</div>
<p>Now that you’re familiar with LIT’s interface, take LIT for a spin in our <a href="../../demos">demos</a> or explore different <a href="../">case studies</a>.</p>
<p>Now that you are familiar with LIT’s interface, take LIT for a spin in our <a href="../../demos">demos</a> or explore different <a href="../">case studies</a>.</p>

</div>
<div class="mdl-cell--4-col hide-me">
Expand Down
6 changes: 3 additions & 3 deletions website/src/demos.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ color: "#49596c"

{% include partials/demo-card c-title: "Image classification", link: "/demos/images.html",
c-data-source: "Imagenette", c-copy: "Analyze an image classification model with LIT, including muliple image salience techniques.", tags: "images, mutli-class classification", external:"true" %}
{% include partials/demo-card c-title: "Classification and regression models", link: "/demos/glue.html",

{% include partials/demo-card c-title: "Classification and regression models", link: "/demos/glue.html",
c-data-source: "Stanford Sentiment Treebank, Multi-Genre NLI Corpus, Semantic Textual Similarity Benchmark"
c-copy: "Use LIT with any of three tasks from the General Language Understanding Evaluation (GLUE) benchmark suite. This demo contains binary classification (for sentiment analysis, using SST2), multi-class classification (for textual entailment, using MultiNLI), and regression (for measuringtext similarity, using STS-B).", tags: "BERT, binary classification, multi-class classification, regression", external:"true" %}

{% include partials/demo-card c-title: "Notebook usage", link: "https://colab.research.google.com/github/PAIR-code/lit/blob/main/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb",
{% include partials/external-demo-card c-title: "Notebook usage", link: "https://colab.research.google.com/github/PAIR-code/lit/blob/main/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb",
c-data-source: "Stanford Sentiment Treebank"
c-copy: "Use LIT directly inside a Colab notebook. Explore binary classification for sentiment analysis using SST2 from the General Language Understanding Evaluation (GLUE) benchmark suite.", tags: "BERT, binary classification, notebooks", external:"true" %}

Expand Down
4 changes: 2 additions & 2 deletions website/src/demos/coref.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
layout: "layouts/redirect.liquid"
permalink: "/demos/coref.html"
url: "http://35.225.219.27/"
---
url: "http://34.86.17.245:5431/"
---
4 changes: 2 additions & 2 deletions website/src/demos/glue.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
layout: "layouts/redirect.liquid"
permalink: "/demos/glue.html"
url: "http://34.69.145.124/"
---
url: "http://34.86.17.245:5432/"
---
4 changes: 2 additions & 2 deletions website/src/demos/images.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
layout: "layouts/redirect.liquid"
permalink: "/demos/images.html"
url: "http://104.155.142.190/"
---
url: "http://34.86.17.245:5433/"
---
4 changes: 2 additions & 2 deletions website/src/demos/lm.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
layout: "layouts/redirect.liquid"
permalink: "/demos/lm.html"
url: "http://104.197.40.13/"
---
url: "http://34.86.17.245:5433/"
---
4 changes: 2 additions & 2 deletions website/src/demos/penguins.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
layout: "layouts/redirect.liquid"
permalink: "/demos/penguins.html"
url: "http://35.222.78.90/"
---
url: "http://34.86.17.245:5435/"
---
4 changes: 2 additions & 2 deletions website/src/demos/t5.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
layout: "layouts/redirect.liquid"
permalink: "/demos/t5.html"
url: "http://34.68.143.28/"
---
url: "http://34.86.17.245:5436/"
---

0 comments on commit 9fd2da3

Please sign in to comment.