Merge branch 'main' into task/gsk-2701-add-documentation-advanced-scan

.github/workflows/build-python.yml

@@ -72,7 +72,7 @@ jobs:
             pandas_v1: false
             langchain_minimal: false
           - python-version: "3.10"
-            os: macos-latest
+            os: macos-12
             pydantic_v1: false
             pandas_v1: false
             langchain_minimal: false
@@ -144,7 +144,7 @@ jobs:
           df -h
 
       - name: Re-install lightgbm from sources for MacOS
-        if: ${{ matrix.os == 'macos-latest' }}
+        if: ${{ matrix.os == 'macos-12' }}
         run: |
           pdm run pip uninstall lightgbm -y
           pdm run pip install --no-binary lightgbm lightgbm --config-settings=cmake.define.USE_OPENMP=OFF

.github/workflows/create-release.yml

@@ -60,7 +60,7 @@ jobs:
 
       - name: "@slack Release process started"
         id: slack
-        uses: slackapi/slack-github-action@v1.25.0
+        uses: slackapi/slack-github-action@v1.26.0
         with:
           channel-id: ${{ vars.SLACK_CHANNEL_ID }}
           slack-message: |-

.github/workflows/post-release.yml

@@ -117,7 +117,7 @@ jobs:
       - name: "@slack Share release process completion"
         # cancellable always() https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions
         if: ${{ !cancelled() }}
-        uses: slackapi/slack-github-action@v1.25.0
+        uses: slackapi/slack-github-action@v1.26.0
         env:
           SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
           ON_SUCCESS: |-

CONTRIBUTING.md

@@ -48,7 +48,9 @@ An awesome feature request addresses the following points:
 
 ## Style guide
 
-We use `black` to format the code and `isort` to sort the imports. You can automatically enforce this style using the pre-commit hooks. You can find the instructions to install the pre-commit hooks [here](https://docs.giskard.ai/en/latest/community/contribution_guidelines/dev-environment.html).
+We use `black` to format the code and `isort` to sort the imports. You can automatically enforce this style using the
+pre-commit hooks. You can find the instructions to install the pre-commit
+hooks [here](https://docs.giskard.ai/en/stable/community/contribution_guidelines/dev-environment.html).
 
 For documentation strings, Giskard follows the [numpydoc](https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard) style.
 

README.md

@@ -16,7 +16,7 @@
 
 </div>
 <h3 align="center">
-   <a href="https://docs.giskard.ai/en/latest/getting_started/index.html"><b>Docs</b></a> &bull;
+   <a href="https://docs.giskard.ai/en/stable/getting_started/index.html"><b>Docs</b></a> &bull;
    <a href="https://www.giskard.ai/knowledge-categories/news/?utm_source=github&utm_medium=github&utm_campaign=github_readme&utm_id=readmeblog"><b>Blog</b></a> &bull;
   <a href="https://www.giskard.ai/?utm_source=github&utm_medium=github&utm_campaign=github_readme&utm_id=readmeblog"><b>Website</b></a> &bull;
   <a href="https://gisk.ar/discord"><b>Discord</b></a>

docs/assets/scan_widget.html

@@ -131,7 +131,7 @@
                 <li>Drift between your training set and test set</li>
             </ul>
             <p class="mt-1">
-                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/performance_bias/index.html" target="_blank">guide on performance bias.</a>
+                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/performance_bias/index.html" target="_blank">guide on performance bias.</a>
             </p>
             
         </div>
@@ -1129,7 +1129,7 @@
                 <li>Use of complex models with large number of parameters that tend to overfit the training data</li>
             </ul>
             <p class="mt-1">
-                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/robustness/index.html" target="_blank">guide on robustness issues.</a>
+                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/robustness/index.html" target="_blank">guide on robustness issues.</a>
             </p>
             
         </div>
@@ -1257,7 +1257,7 @@
                     class</li>
             </ul>
             <p class="mt-1">
-                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/overconfidence/index.html" target="_blank">guide on overconfidence issues.</a>
+                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/overconfidence/index.html" target="_blank">guide on overconfidence issues.</a>
             </p>
             
         </div>
@@ -2233,7 +2233,7 @@
                 <li>Data noise: the training set contains anomalies that are unrelated to the underlying problem (data collection, measurement biases, data preprocessing issues, etc.)</li>
             </ul>
             <p class="mt-1">
-                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/spurious/index.html" target="_blank">guide on spurious correlation.</a>
+                To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/stable/knowledge/key_vulnerabilities/spurious/index.html" target="_blank">guide on spurious correlation.</a>
             </p>
             
         </div>
@@ -3026,4 +3026,4 @@
   }), z = {}, window.jQuery !== d && q(window.jQuery), "function" == typeof define && define.amd ? define([], n) : "object" == (typeof module === "undefined" ? "undefined" : _typeof(module)) && "object" == _typeof(module.exports) && (module.exports = n()), window.iFrameResize = window.iFrameResize || n());
 }();
 (function(){iFrameResize({ checkOrigin: false }, '#scan-140417288606800');})();
-</script>
+</script>

docs/conf.py

@@ -34,7 +34,7 @@
 
 autoclass_content = "both"
 
-# autodoc_mock_imports = ["giskard.ml_worker.generated"]
+autodoc_mock_imports = ["ragas", "langchain_core"]
 templates_path = ["_templates"]
 exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 

docs/getting_started/index.md

@@ -24,7 +24,7 @@ Testing Machine Learning applications can be tedious. Since AI models depend on
 
 To help you solve these challenges, Giskard library helps to:
 
-- **Scan your model to find hidden vulnerabilitie automatically**: The `giskard` scan automatically detects vulnerabilities
+- **Scan your model to find hidden vulnerabilities automatically**: The `giskard` scan automatically detects vulnerabilities
 such as performance bias, hallucination, prompt injection, data leakage, spurious correlation, overconfidence, etc.
   <br><br>
   <iframe src="https://htmlpreview.github.io/?https://gist.githubusercontent.com/AbSsEnT/a67354621807f3c3a332fca7d8b9a5c8/raw/588f027dc6b14c88c7393c50ff3086fe1122e2e9/LLM_QA_IPCC_scan_report.html" width="700" height="400"></iframe>
@@ -62,7 +62,3 @@ An Enterprise Hub for teams to collaborate on top of the open-source Giskard lib
 - 💬 **Collect business feedback** and **share your model results** with data scientists, QA teams and auditors.
   <br><br>
   ![](../assets/credit_scoring_comment.png)
-
-
-Get started **now** with our [demo HuggingFace Space](https://huggingface.co/spaces/giskardai/giskard) or
-by following [installation instructions](../getting_started/quickstart/index.md)! 🐢

docs/getting_started/quickstart/index.md

@@ -8,18 +8,17 @@
 quickstart_llm
 quickstart_tabular
 quickstart_nlp
+quickstart_vision
 ```
 
-
 :::::{grid} 1 1 2 2
 
-
-::::{grid-item-card} <br/><h3>📚  LLM Quickstart</h3>
+::::{grid-item-card} <br/><h3>📚 LLM Quickstart</h3>
 :text-align: center
 :link: quickstart_llm.ipynb
 ::::
 
-::::{grid-item-card} <br/><h3>📊  Tabular Quickstart</h3>
+::::{grid-item-card} <br/><h3>📊 Tabular Quickstart</h3>
 :text-align: center
 :link: quickstart_tabular.ipynb
 ::::
@@ -29,4 +28,9 @@ quickstart_nlp
 :link: quickstart_nlp.ipynb
 ::::
 
+::::{grid-item-card} <br/><h3>📸 Vision Quickstart</h3>
+:text-align: center
+:link: quickstart_vision.ipynb
+::::
+
 :::::

docs/getting_started/quickstart/quickstart_llm.ipynb

@@ -1191,7 +1191,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/latest/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
+    "If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/stable/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
    ]
   },
   {
@@ -1478,7 +1478,7 @@
        "<li>Overreliance on spurious correlations like the presence of specific word</li>\n",
        "<li>Use of complex models with large number of parameters that tend to overfit the training data</li>\n",
        "</ul>\n",
-       "<p>To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/robustness/index.html">guide on robustness issues</a>.</p>\n",
+       "<p>To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/robustness/index.html">guide on robustness issues</a>.</p>\n",
        "            </div>\n",
        "        </div>\n",
        "    </div>\n",

docs/getting_started/quickstart/quickstart_nlp.ipynb

@@ -177,7 +177,7 @@
     "id": "3PGiyUwAJpdK"
    },
    "source": [
-    "To prepare for the vulnerability scan, make sure to wrap your dataset using Giskard's Dataset class. More details [here](https://docs.giskard.ai/en/latest/open_source/scan/scan_nlp/index.html#step-1-wrap-your-dataset)."
+    "To prepare for the vulnerability scan, make sure to wrap your dataset using Giskard's Dataset class. More details [here](https://docs.giskard.ai/en/stable/open_source/scan/scan_nlp/index.html#step-1-wrap-your-dataset)."
    ]
   },
   {
@@ -258,7 +258,7 @@
     "id": "NwunAyRvJzOS"
    },
    "source": [
-    "To prepare for the vulnerability scan, make sure to wrap your model using Giskard's Model class. You can choose to either wrap the prediction function (preferred option) or the model object. More details [here](https://docs.giskard.ai/en/latest/open_source/scan/scan_nlp/index.html#step-2-wrap-your-model)."
+    "To prepare for the vulnerability scan, make sure to wrap your model using Giskard's Model class. You can choose to either wrap the prediction function (preferred option) or the model object. More details [here](https://docs.giskard.ai/en/stable/open_source/scan/scan_nlp/index.html#step-2-wrap-your-model)."
    ]
   },
   {
@@ -310,7 +310,7 @@
    "source": [
     "### Scan your model for vulnerabilities with Giskard\n",
     "\n",
-    "Giskard's scan allows you to detect vulnerabilities in your model automatically. These include performance biases, unrobustness, data leakage, stochasticity, underconfidence, ethical issues, and more. For detailed information about the scan feature, please refer to our [scan documentation](https://docs.giskard.ai/en/latest/open_source/scan/scan_nlp/index.html)."
+    "Giskard's scan allows you to detect vulnerabilities in your model automatically. These include performance biases, unrobustness, data leakage, stochasticity, underconfidence, ethical issues, and more. For detailed information about the scan feature, please refer to our [scan documentation](https://docs.giskard.ai/en/stable/open_source/scan/scan_nlp/index.html)."
    ]
   },
   {
@@ -329,7 +329,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/latest/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
+    "If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/stable/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
    ],
    "id": "9dd5baaaa6a7ee62"
   },
@@ -442,7 +442,7 @@
        "<li>Overreliance on spurious correlations like the presence of specific word</li>\n",
        "<li>Use of complex models with large number of parameters that tend to overfit the training data</li>\n",
        "</ul>\n",
-       "<p>To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/robustness/index.html">guide on robustness issues</a>.</p>\n",
+       "<p>To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/robustness/index.html">guide on robustness issues</a>.</p>\n",
        "            </div>\n",
        "        </div>\n",
        "    </div>\n",
@@ -1021,7 +1021,7 @@
        "<li>Data is reflecting some structural biases and societal prejudices</li>\n",
        "<li>Use of complex models with large number of parameters that tend to overfit the training data</li>\n",
        "</ul>\n",
-       "<p>To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/ethics/index.html">guide on unethical behaviour.</a></p>\n",
+       "<p>To learn more about causes and solutions, check our <a href="https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/ethics/index.html">guide on unethical behaviour.</a></p>\n",
        "            </div>\n",
        "        </div>\n",
        "    </div>\n",
@@ -1835,7 +1835,7 @@
     "* Slicing functions such as detectors of toxicity, hate, emotion, etc\n",
     "* Transformation functions such as generators of typos, paraphrase, style tune, etc\n",
     "\n",
-    "To create custom tests, refer to [this page](https://docs.giskard.ai/en/latest/open_source/customize_tests/test_model/index.html).\n",
+    "To create custom tests, refer to [this page](https://docs.giskard.ai/en/stable/open_source/customize_tests/test_model/index.html).\n",
     "\n",
     "For demo purposes, we will load a simple unit test (test_f1) that checks if the test F1 score is above the given threshold. For more examples of tests and functions, refer to the Giskard catalog."
    ]