Skip to content

Latest commit

 

History

History
2085 lines (1964 loc) · 183 KB

CHANGES.md

File metadata and controls

2085 lines (1964 loc) · 183 KB
Raw

Changes

This file tracks changes to the raiwidgets package. It should be updated in PRs that make changes, not separately. At the top of the file will always be a v-next entry to add changes that were made since the last release. Each set of changes should be grouped by

  • educational materials
  • new features
  • breaking changes
  • bug fixes and tests
  • other

Note that it is not required to have an entry for every pull request. Instead, please try to add only changes that are meaningful to users who read this file to understand what changed.

v-next

  • educational materials
  • new features
  • breaking changes
  • bug fixes and tests
  • other

v0.35.1

  • bug fixes and tests

    • Responsible AI Dashboard

      • upgrade nx to 12.10.1 by @imatiach-msft in #2561
      • remove pins on werkzeug, itsdangerous and markupsafe since they are currently not needed by @imatiach-msft in #2565
      • release rai-core-flask 0.7.6 by @tongyu-microsoft in #2566

    • RAI Vision

      • fix wrapper constructor in ErrorAnalysisManager and RAIVisionInsights load call for RAI Vision Dashboard by @imatiach-msft in #2560

    • RAI Vision and Text

      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.35.0 by @imatiach-msft in #2563
      • release responsibleai-vision 0.3.8 and responsibleai-text 0.2.7 by @imatiach-msft in #2564

v0.35.0

  • new features

    • RAI Vision

      • Exif Nested Metadata support for Image Dashboards by @Advitya17 in #2542
      • Nested Exif Metadata support + Gate fixes for py 3.7 and keras version by @Advitya17 in #2545
      • Release responsibleai_vision 0.3.7 with Nested Exif Metadata support by @Advitya17 in #2549
  • bug fixes and tests

    • Responsible AI Dashboard

      • fix bad whitespace in url format for azure compute instance base url by @imatiach-msft in #2537
      • release rai-core-flask 0.7.4 by @imatiach-msft in #2538
      • Gates fix to remove py 3.7 for EOL & AutoML 2.0.6 by @Advitya17 in #2543
      • update ipython and werkzeug dependencies for responsibleai and rai-core-flask packages by @imatiach-msft in #2551
      • release rai-core-flask 0.7.5 by @imatiach-msft in #2553
      • upgrade packages and CI to rai-core-flask==0.7.5, rai-utils==0.4.2 and rai-test-utils==0.4.2 by @imatiach-msft in #2556

    • RAIInsights

      • fix builds due to OneHotEncoder sparse parameter breaking change in scikit-learn by @imatiach-msft in #2539
      • update responsibleai and raiwidgets to erroranalysis 0.5.4 by @imatiach-msft in #2558

    • RAI Vision

      • fix label column incorrectly added by feature extractors in RAI Vision dashboard for automl models by @imatiach-msft in #2532
      • fix index into image_dimensions for drawing bounding boxes in object detection flyout by @imatiach-msft in #2550

    • RAI Vision and Text

      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.34.1 by @imatiach-msft in #2534
      • release responsibleai-vision 0.3.6 and responsibleai-text 0.2.6 by @imatiach-msft in #2535
      • fix release github actions for RAI vision and text packages by @imatiach-msft in #2536

    • Error analysis

    • RAIUtils

      • release rai-utils 0.4.2 by @imatiach-msft in #2555

    • RAI Test Utils

      • release rai-test-utils 0.4.2 by @imatiach-msft in #2554

v0.34.1

  • new features

    • Responsible AI Dashboard

      • Fix forecast flattening prediction list error by @jamesbchao in #2529

    • RAI Text

      • Add methods and constants for genai metrics by @kartik727 in #2524
      • Genai integration by @kartik727 in #2522
  • bug fixes and tests

    • Responsible AI Dashboard

      • fix CD.yml permissions errors for dashboard links on PRs by @imatiach-msft in #2525

    • RAI Vision and Text

      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.34.0 by @imatiach-msft in #2530

v0.34.0

  • new features

    • Responsible AI Dashboard

      • Private Link URL support by @Advitya17 in #2493
      • PL Support on raiwidgets by @Advitya17 in #2500
      • Flyout Bounding Box legend & accessibility fixes by @Advitya17 in #2497
      • update UI to support genai task for RAI text dashboard by @imatiach-msft in #2508
      • add genai metrics endpoint in UI for model overview metrics (#2517) by @imatiach-msft in #2520

    • RAI Text

      • Added metrics for genai text by @kartik727 in #2514
      • added generative text explainer by @mohsinposts in #2516
      • Add helper method for genai metrics by @kartik727 in #2519
  • bug fixes and tests

    • Responsible AI Dashboard

      • upgrade plotly to latest version 2.27.1 by @imatiach-msft in #2484
      • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20231212124106972 to main by @csigs in #2457
      • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 113458687 by @RubyZ10 in #2460

    • RAIInsights

      • update responsibleai to interpret-community 0.31.0 by @imatiach-msft in #2488
      • Release rai_core_flask 0.7.3 by @Advitya17 in #2496
      • fix builds by supporting sparse_output renamed parameter in OneHotEncoder in new scikit-learn version update by @imatiach-msft in #2507

    • RAI Vision

      • fix vision data explorer elements truncated in smaller screen by @imatiach-msft in #2478
      • fix duplicate categorical features added from image exif metadata in RAI Vision Dashboard by @imatiach-msft in #2483
      • Accessibility Fix for Metric Checkboxes by @Advitya17 in #2485
      • fix wrong metrics count missing in vision explorer of RAI Vision dashboard for object detection by @imatiach-msft in #2495
      • Class View Search Case Fix by @Advitya17 in #2498
      • Dropdown option fix by @Advitya17 in #2505

    • RAI Vision and Text

      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.33.0 by @imatiach-msft in #2477
      • release responsibleai-vision 0.3.5 and responsibleai-text 0.2.5 by @imatiach-msft in #2481
      • release tensorflow pin in RAI vision and text package tests which was added last year to fix huggingface transformer models by @imatiach-msft in #2472

    • RAIUtils

      • Fix retry function not stopping for long time after sending an interrupt by @kartik727 in #2486
  • other
    • Update MANIFEST.in for erroranalysis to include requirements-object-detection.txt by @rxm7706 in #2476
    • fix release script due to dependency test errors by @imatiach-msft in #2482
    • ignore error when uploading test dashboard from build for external forks by @imatiach-msft in #2487
    • ci: Use GITHUB_OUTPUT envvar instead of set-output command by @arunsathiya in #2489

v0.33.0

  • new features

    • Responsible AI Dashboard

      • add AccessibleDetailsList with resizable columns for accessibility by @imatiach-msft in #2470
  • bug fixes and tests

    • Responsible AI Dashboard

      • fix aria label for screen reader on scatter plot color axis picker by @imatiach-msft in #2440
      • fix RAI dashboard treat as categorical accessibility on new cohort create by @imatiach-msft in #2446
      • fix line-height override on add tab button for accessibility by @imatiach-msft in #2449
      • fix pivot focus indicator for accessibility by @imatiach-msft in #2450
      • fix confusion matrix highcharts heatmap keyboard focus order accessibility by @imatiach-msft in #2452
      • fix class importance weights callout truncated in small view for accessibility by @imatiach-msft in #2462
      • fix tooltip callout keyboard focus order for accessibility by @imatiach-msft in #2464
      • announce highcharts confusion matrix heatmap tooltip aria label for accessibility by @imatiach-msft in #2467
      • announce search results in RAI Vision dashboard toolbar for accessibility by @imatiach-msft in #2466

    • RAIInsights

      • remove python 3.6 from builds, tests and packages by @imatiach-msft in #2442
      • Upgrade setup python from v4 to v5 by @gaugup in #2454
      • release rai-core-flask 0.7.2 by @imatiach-msft in #2458
      • update raiwidgets to rai-core-flask v0.7.2 by @imatiach-msft in #2459
      • update numpy and numba dependencies by @imatiach-msft in #2473

    • RAI Vision

      • fix luminosity ratio of RAI Vision dashboard success and failure instances text to background for accessibility by @imatiach-msft in #2444
      • fix error when extracting exif metadata features from images in RAI Vision dashboard by @imatiach-msft in #2461
      • fix images missing alternate text in RAI Vision dashboard for accessibility by @imatiach-msft in #2463

    • RAI Vision and Text

      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.32.1 by @imatiach-msft in #2445
      • release responsibleai-vision 0.3.4 and responsibleai-text 0.2.4 by @imatiach-msft in #2447
      • fix socketio cors authentication for object detection and qa metrics requests by @imatiach-msft in #2455

    • Error analysis

      • Data Imputation Fix in erroranalysis by @Advitya17 in #2436
      • release erroranalysis 0.5.3 by @imatiach-msft in #2468
      • update responsibleai and raiwidgets to erroranalysis 0.5.3 by @imatiach-msft in #2471
  • other
    • update vscode settings for codeActionsOnSave to use new field instead of boolean by @imatiach-msft in #2465

v0.32.1

  • new features

    • Responsible AI Dashboard

      • UI Improvement for Aggregate Object Detection Labels by @Advitya17 in #2431
  • bug fixes and tests

    • Responsible AI Dashboard

      • Update raiwidgets to rai-core-flask v0.7.1 by @Advitya17 in #2437

    • RAI Vision

      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.32.0 by @imatiach-msft in #2420
      • release responsibleai-vision 0.3.3 and responsibleai-text 0.2.3 by @imatiach-msft in #2421
      • Added aria-labels to dropdowns by @Advitya17 in #2417
      • make elements under image explorer view tab keyboard accessible by @imatiach-msft in #2422
      • Added aria-labels for sliders and checkboxes/DetailsList by @Advitya17 in #2419
      • Added aria-labels to buttons by @Advitya17 in #2418
      • fix switch cohort window getting clipped when zoomed in for accessibility by @imatiach-msft in #2430
      • fix vision dashboard flyout accessibility on smaller screen size by @imatiach-msft in #2432
      • fix image downloader failing with automl format on deserialize due to missing label transformations by @imatiach-msft in #2435
      • fix save and switch button text overflowing for accessibility by @imatiach-msft in #2438
      • Object Detection Model Overview missing-class fix for Cohorts by @Advitya17 in #2429
  • other
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 111947035 by @RubyZ10 in #2433
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20231125124253772 to main by @csigs in #2428
    • Removed python 3.6 from CI-notebook gate by @Advitya17 in #2441
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 112224195 by @RubyZ10 in #2434

v0.32.0

  • new features

    • Responsible AI Dashboard

      • add socket implementation to RAI dashboard object detection and question answering metrics calls by @imatiach-msft in #2394
      • Dashboard Port Updates by @Advitya17 in #2406
  • bug fixes and tests

    • Responsible AI Dashboard

      • release rai-core-flask 0.7.0 by @imatiach-msft in #2398
      • update raiwidgets to rai-core-flask v0.7.0 by @imatiach-msft in #2401
      • Release new version of rai_core_flask by @Advitya17 in #2407
      • fallback to request if socket connection fails for object detection and question answering metrics for backcompat by @imatiach-msft in #2409
      • Side Panel Bug Fix by @Advitya17 in #2414

    • RAI Vision

      • OD Explanation Loader Fix by @Advitya17 in #2381
      • Optional support for IS_CROWD for OD COCO label format by @Advitya17 in #2395
      • Import & data loading fixes for IC & multilabel notebooks by @Advitya17 in #2410
      • fix RangeIndex error in error analysis for object detection models by @imatiach-msft in #2411
      • add average parameter to MeanAveragePrecision to specify micro or macro calculation by @imatiach-msft in #2412

    • Counterfactual

      • Update requirements.txt to upgrade dice-ml to 0.11.0 by @gaugup in #2397
      • Refer to predicted value from Counterfactual data by @gaugup in #2402

    • Error analysis

      • Accessibility fix for 'Error Analysis' header by @Advitya17 in #2415
  • other
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 107831016 by @RubyZ10 in #2393
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20231003114057604 to main by @csigs in #2384
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20231030114314322 to main by @csigs in #2400
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 109426168 by @RubyZ10 in #2408
    • fix build errors due to changed exception messages and UCI repository expired certificate by @imatiach-msft in #2413

v0.31.0

  • new features

    • Responsible AI Dashboard

      • Heatmap Support for Object Detection by @Advitya17 in #2303
      • Vision Explanation Dashboard render refactor by @Advitya17 in #2306
      • Single Container in Image Data Explorer for Object Detection by @Advitya17 in #2330
      • Error Labeling integration into Image Data Explorer for Agg Labels by @Advitya17 in #2334
      • Detection info inside Object Detection Flyout by @Advitya17 in #2339
      • Categorical Exif Metadata support for Vision Dashboard by @Advitya17 in #2348
      • Error Labeling Integration in Table View by @Advitya17 in #2350
      • Numerical Exif Metadata support for Image Dashboards by @Advitya17 in #2358
      • add local feature importance info button and include shap values in description for RAI text dashboard by @imatiach-msft in #2373
      • Error Labeling Integration in Class View by @Advitya17 in #2354
      • Sorting support for Object Detection labels by @Advitya17 in #2376
      • Forecasting: generate time series as cohorts by @romanlutz in #2355

    • RAI utils

      • Add pandas timestamp support to json serialization utility function in raiutils by @romanlutz in #2320

    • RAI tabular

      • Forecasting: consume locally served model by @romanlutz in #2319
  • bug fixes and tests

    • Responsible AI Dashboard

      • accessibility: add aria label to RAI text dashboard slider by @imatiach-msft in #2293
      • Added ariaLabel to cohort dropdown by @Advitya17 in #2296
      • UI Tests for Object Detection Widgets in Model Overview by @Advitya17 in #2295
      • move text UI individual feature importance tests by @imatiach-msft in #2299
      • RAI Text Dashboard a11y: make scrollable pane focusable and use label attribute on slider by @imatiach-msft in #2305
      • Heatmap e2e test by @Advitya17 in #2239
      • fix true y and pred y not appearing for QA text dashboard in table view by @imatiach-msft in #2336
      • fix failing object detection e2e jupyter notebook UI tests on linux by @imatiach-msft in #2341
      • Expanded Object Detection Flyout & Image Sizes by @Advitya17 in #2335
      • add model explanation data to test app for question answering example on squad dataset by @imatiach-msft in #2326
      • fix alignment of true and predicted labels in vision data explorer by @imatiach-msft in #2369
      • fix choose cohorts close or cancel updating feature cohorts selection in flyout by @imatiach-msft in #2366
      • remove duplicate text in feature importance graph of the RAI Text dashboard by @imatiach-msft in #2371
      • remove non-functional page size selector from RAI Vision dashboard table view by @imatiach-msft in #2370
      • Backend and OD Notebook Dataset fixes by @Advitya17 in #2360
      • Data Analysis Label Fix for OD by @Advitya17 in #2365
      • Table View Search Fix for Multilabel & OD by @Advitya17 in #2367
      • Enabling Heatmap test for Feature Cohort + MO test fixes by @Advitya17 in #2340

    • RAI Text

      • fix failing vision tests due to new RAI validation logic which doesn't allow non string dropped column names by @imatiach-msft in #2302
      • add text feature importance e2e unit tests for covid jupyter notebook by @imatiach-msft in #2323
      • add text feature importance e2e unit tests for dbpedia jupyter notebook by @imatiach-msft in #2313
      • fix error analysis for RAITextInsights displaying all incorrect instances for text labels by @imatiach-msft in #2325
      • add text feature importance e2e unit tests for blbooksgenre jupyter notebook by @imatiach-msft in #2327
      • fix question answering pipeline indexing error in shap due to varied token length after masking by @imatiach-msft in #2332
      • add RAITextInsights openai notebook example and tests by @imatiach-msft in #2345

    • RAI Vision

      • fix failing vision tests due to new RAI validation logic which doesn't allow non string dropped column names by @imatiach-msft in #2302
      • OD model type support in responsibleai_vision for EA by @Advitya17 in #2233
      • Test case for EA-OD flow by @Advitya17 in #2322
      • fix data analysis description and tooltips hidden for large screens by @imatiach-msft in #2368
      • fix flaky object detection vision notebook test by allowing range of feature cohorts to be specified by @imatiach-msft in #2374
      • Unit test for Exif support in the Image Dashboards by @Advitya17 in #2382

    • RAI Tabular

    • Error analysis

      • Hotfix to add VEM as soft dependency to erroranalysis by @Advitya17 in #2290
      • EA ModelTask Fix by @Advitya17 in #2316
  • other
    • update responsibleai-text and responsibleai-vision packages to responsibleai 0.30.0 by @imatiach-msft in #2300
    • Update requirements-linting.txt to add flake8-all-not-strings by @gaugup in #2276
    • Release erroranalysis 0.5.1 by @Advitya17 in #2304
    • Release Error Analysis 0.5.2 by @Advitya17 in #2317
    • increase verbosity for gated tests to help diagnose failures by @imatiach-msft in #2301
    • release responsibleai-vision 0.3.0 and responsibleai-text 0.2.0 by @imatiach-msft in #2315
    • Release responsibleai_vision 0.3.1 by @Advitya17 in #2321
    • Add create_complex_regression_pipeline to sklearn_model_utils.py by @gaugup in #2318
    • Remove pins on flake8 linter packages by @gaugup in #2328
    • release responsibleai-text 0.2.1 by @imatiach-msft in #2337
    • causal_manager.py: Move user config validation checks from compute() to add() by @gaugup in #2342
    • Fix links on README.md by @jamesbchao in #2349
    • Update README.md by @jamesbchao in #2352
    • Remove unreachable branch in test_rai_insights_save_and_load_scenarios.py by @gaugup in #2343
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 106178211 by @RubyZ10 in #2375
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20230928114108584 to main by @csigs in #2372
    • Add python 3.11 for erroranalysis, raiutils and rai-test-utils by @gaugup in #2385
    • Add 3.11 support to rai-core-flask by @gaugup in #2386
    • fix failing builds due to pytorch dependencies by @imatiach-msft in #2389
    • pin responsibleai text and vision packages to 0.30.0 by @romanlutz in #2387

v0.30.0

  • educational materials

    • RAI Text

      • Doc update for responsibleai_text by @Advitya17 in #2169
  • new features

    • Responsible AI Dashboard

      • [Question Answering] Cache support for Model Overview Metrics by @Advitya17 in #2166
      • Individual feature importance interpret QA - part1 by @vinuthakaranth in #2167
      • Individual feature importance interpret QA by @vinuthakaranth in #2186
      • Error Analysis UI & Template support for Object Detection by @Advitya17 in #2203
      • add multiclass statistics to text and tabular RAI dashboards by @imatiach-msft in #2222
      • Add forecasting to raiwidgets and add a forecasting notebook by @romanlutz in #2252

    • RAI Text

      • [Question Answering] Cache support for Model Overview Metrics by @Advitya17 in #2166
      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.29.0 by @imatiach-msft in #2172
      • add base values and importances for ending tokens to QA explanations by @imatiach-msft in #2178

    • RAI Vision

      • [Question Answering] Cache support for Model Overview Metrics by @Advitya17 in #2166
      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.29.0 by @imatiach-msft in #2172
      • [Object Detection] Model Overview Cache Support Extension for torchmetrics by @Advitya17 in #2170
      • OD model type support in raiutils for EA by @Advitya17 in #2227
      • GPU support for OD metrics by @Advitya17 in #2266

    • RAI Tabular

      • OD model type support in responsibleai for EA by @Advitya17 in #2230

    • Error analysis

      • Error Analysis backend support for Object Detection by @Advitya17 in #2225
  • bug fixes and tests

    • Responsible AI Dashboard

      • ModelAssessmentVision UI Test Template by @Advitya17 in #2174
      • Multiclass Image Classification Notebook for UI testing by @Advitya17 in #2175
      • Image Classification Notebook for UI testing #2175 by @Advitya17 in #2176
      • Vision Data Analysis / Data Balance E2E Tests + Refactor by @Advitya17 in #2183
      • Vision Model Overview E2E Tests - Refactor by @Advitya17 in #2185
      • OD ID Plot Hotfix by @Advitya17 in #2190
      • Fix heatmap by removing erroneous color configuration by @romanlutz in #2201
      • Vision Data Explorer Table & Class View tests by @Advitya17 in #2220
      • Vision Data Explorer Image Explorer Tests by @Advitya17 in #2188
      • Enabled EA UI testing for OD by @Advitya17 in #2251
      • CanvasTools OD Bounding Box Fix by @Advitya17 in #2267
      • Add forecasting to raiwidgets and add a forecasting notebook by @romanlutz in #2252
      • add e2e UI notebook tests to DBPedia text classification notebook by @imatiach-msft in #2265
      • Endpoint Interruption Logic for Model Overview with OD by @Advitya17 in #2268
      • Endpoint Interruption Logic for Model Overview with QA by @Advitya17 in #2274
      • add e2e UI notebook tests to blbooksgenre text classification notebook by @imatiach-msft in #2273
      • add RAITextInsights question answering notebook example and tests by @imatiach-msft in #2279
      • add e2e UI notebook tests to covid19 event multilabel text classification notebook by @imatiach-msft in #2282

    • RAI Vision

      • Require pillow>=10.0.0, catch drise failure due to low mask_res or num_masks by @romanlutz in #2181
      • Add retry logic to image reader by @kicha0 in #2206
      • fix error for automl object detection models when initializing error analysis and optimize explanation execution by @imatiach-msft in #2245
      • fix for aggregate parameter passed to torchmetrics by @imatiach-msft in #2235
      • pass transforms for optimized automl model explanations and improve error logging to include inner exception by @imatiach-msft in #2269

    • RAI Text

      • add RAITextInsights DBPedia notebook example and tests by @imatiach-msft in #2204
      • add RAITextInsights covid19 multilabel classification notebook example and tests by @imatiach-msft in #2221
      • add RAITextInsights blbooksgenre binary classification notebook example and tests by @imatiach-msft in #2226
      • Raise usererror on prediction failure. by @kicha0 in #2248
      • add RAITextInsights question answering notebook example and tests by @imatiach-msft in #2279
      • fix question answering error when running pipeline model on cuda by @imatiach-msft in #2291
      • add e2e UI notebook tests to covid19 event multilabel text classification notebook by @imatiach-msft in #2282

    • RAI Tabular

      • Change ValueError to UserConfigValidationException in casual_manager.py by @gaugup in #2209
      • Add input type validation for FeatureMetadata class by @gaugup in #2218
      • Move missing value validations before unique value validations by @gaugup in #2228
      • Pass categorical features after dropping dropped features in CounterfactualManager by @gaugup in #2241
      • RAIInsights: prevent failures on optional methods & fix feature range formatting for timestamps by @romanlutz in #2258
      • fix RAI explanation error for dropped categorical features after serialization by @imatiach-msft in #2261
      • Tabular Only: Address missing values as per each manager in responsibleai by @gaugup in #2263
      • responsibleai: Add prediction wrapper to missing value tests by @gaugup in #2270

    • Error Analysis

      • Update text insights and error_analysis_manager to handle pd.DataFrame predictions and prediction with type of string by @tongyu-microsoft in #2179
      • Error Analysis testing for IC & multilabel by @Advitya17 in #2205
      • change erroranalysis valueerror caused by invalid model prediction shape to user error by @imatiach-msft in #2254

    • Fairness Dashboard

      • Use only Fairlearn v0.7.0 due to breaking changes in v0.9.0 and lack of Python 3.7 support in v0.8.0 by @romanlutz in #2194
  • other
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 99116202 by @RubyZ10 in #2177
    • release responsibleai-vision 0.2.4 by @imatiach-msft in #2192
    • fix responsibleai-vision pypi release script by installing pycocotools from conda by @imatiach-msft in #2197
    • fix build CI failing due to old pip-tools failing with latest pip by @imatiach-msft in #2200
    • release scikit-learn upper bound in responsibleai package by @imatiach-msft in #2173
    • improve notebook test logging and verbosity to diagnose timeouts by @imatiach-msft in #2202
    • Widget spacing for OD MO widgets by @Advitya17 in #2213
    • Update _NOTICE.md file by @romanlutz in #2231
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20230811114026921 to main by @csigs in #2237
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 101597327 by @RubyZ10 in #2236
    • reduce PR execution time for tabular RAI dashboard changes by @imatiach-msft in #2238
    • Release new version of raiutils with OD support by @Advitya17 in #2244
    • Removed python 3.6 from python PR gate by @Advitya17 in #2249
    • fix ci-notebook-vision transient 6 hour CI timeouts by installing older pyarrow by @imatiach-msft in #2212
    • Update explainer_manager.py to change docs from MimicWrapper to MimicExplainer by @gaugup in #2240
    • refactor notebook examples into tabular, text and vision folders by @imatiach-msft in #2253
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 101886729 by @RubyZ10 in #2255
    • Added additional paths for vision gates by @Advitya17 in #2256
    • move image folder for tabular responsibleaidashboard under tabular folder by @imatiach-msft in #2257
    • release erroranalysis 0.4.5 by @imatiach-msft in #2262
    • fix CI errors due to new mlflow and pydantic dependencies by @imatiach-msft in #2259
    • update responsibleai and raiwidgets to erroranalysis 0.4.5 by @imatiach-msft in #2264
    • convert CI-python to use conda and install pytorch dependencies by @imatiach-msft in #2272
    • update responsibleai to interpret-community 0.30.0 by @imatiach-msft in #2275
    • Remove deprecated module cohort.py from raiwidgets by @gaugup in #2278
    • Turn off flag should_construct_pandas_query in PredictionsModelWrapperClassification by @gaugup in #2277
    • Update CODEOWNERS to include romanlutz for responsibleai by @romanlutz in #2280
    • OD model and data util functions in rai_test_utils by @Advitya17 in #2246
    • Release new version of rai test utils by @Advitya17 in #2284
    • Updated release gate for rai test utils to install OD dependencies by @Advitya17 in #2285
    • Add torch installation support for rai test utils release by @Advitya17 in #2286
    • Release new version of erroranalysis with OD support by @Advitya17 in #2288
    • remove vision e2e notebook tests from raiwidgets and responsibleai release workflow by @imatiach-msft in #2292
    • Rename release-rai.yml to release-rai-tabular.yml by @gaugup in #2283

v0.29.0

  • educational materials

    • Doc update for responsibleai_vision by @Advitya17 in #2164
    • [Object Detection] User Flows / Architectures by @Advitya17 in #2134
    • [Object Detection] DFD Documentation by @natalie-isak in #2138
    • [Object Detection] Explanation Documentation by @natalie-isak in #2136
    • [Object Detection] CanvasTools documentation by @Advitya17 in #2133
    • [Object Detection] Model Overview Object Detection Documentation by @Advitya17 in #2132

  • new features

    • Responsible AI Dashboard

      • [Object Detection] Cache support for Model Overview Metrics by @Advitya17 in #2158
      • [Image Explorer] CanvasTools Image Loading support for Object Detection by @Advitya17 in #2097
      • [Object Detection] [Image Explorer] Bounding Box Support with CanvasTools on Flyout by @Advitya17 in #2116

  • bug fixes and tests

    • Responsible AI Dashboard

      • update responsibleai-text and responsibleai-vision packages to latest responsibleai 0.28.0 release by @imatiach-msft in #2159
      • Robot page exception in DatasetUtils where jointDataset.metaDict[property] is undefined by @tongyu-microsoft in #2148
      • Remove console log by @RubyZ10 in #2125
      • Refactor- Add datasetCohorts to ModelAssessementContext && create datasetCohort && add compositeFilter in DatasetCohort by @RubyZ10 in #2119

    • RAIInsights

      • raise error instead of warning when a user has missing data and add c… by @hawestra in #2143
      • Add columns names to create_multiclass_classification_dataset by @gaugup in #2114
      • add validation target column type in the classification scenario by @hawestra in #2127
      • update econml to latest 0.14.1 release by @imatiach-msft in #2112
      • Update dice-ml to 0.10.0 by @gaugup in #2145

    • Causal

      • Fix individual causal plot not rendering for 5k points by @vinuthakaranth in #2113
      • Add loading spinner for causal policy tab by @gaugup in #2121

    • RAI Text

      • merge RAI Text repository to RAI toolbox by @imatiach-msft in #2118
      • add responsibleai-text package tests to build gate by @imatiach-msft in #2139
      • release responsibleai-text 0.1.4 by @imatiach-msft in #2163
      • add release scripts for responsibleai-text and responsibleai-vision pypi packages by @imatiach-msft in #2156

    • RAI Vision

      • add responsibleai-vision package to responsible-ai-toolbox by @imatiach-msft in #2135
      • add responsibleai-vision package to gated CI tests by @imatiach-msft in #2152
      • add retry logic to common vision utils to make tests more robust by @imatiach-msft in #2154
      • add device parameter to responsibleai_vision package and pass to model wrappers by @imatiach-msft in #2157
      • release responsibleai-vision 0.2.3 by @imatiach-msft in #2162

    • RAI Test Utils

      • Update rai_test_utils in erroranalysis and responsibleai to 0.3.0 by @gaugup in #2111
      • rai_test_utils: Convert all namespaces in __all__ list as strings by @gaugup in #2128

  • other

    • fix build failures due to lint errors for onelocbuild yml by @imatiach-msft in #2137
    • Skipping python 3.7 tests on MACOS by @gaugup in #2129
    • fix build failures due to new joblib 1.3.0 release failing on windows with python 3.7 by @imatiach-msft in #2155
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 97902894 by @RubyZ10 in #2160
    • Update the python version to 3.8 in release workflows by @gaugup in #2161
    • fix job names in release scripts by @imatiach-msft in #2165

v0.28.0

  • educational materials
    • Correct test command and add clarification on GPG Key to CONTRIBUTING.md by @ziqi-ma in #2082
  • new features

    • Responsible AI Dashboard

      • QA Scenario metrics template by @mehektulsyan in #2034
      • Reset ComboBox Option for Object Detection by @natalie-isak in #2049
      • Integrate Question answering metrics from python backend by @mehektulsyan in #2054
      • Display details of points and bubble for highcharts in full-screen mode by @vinuthakaranth in #2060
      • Show message to select bubble after bubble plot renders by @vinuthakaranth in #2070
      • Add dataset metadata to IDataset by @gaugup in #2071
      • enable confusion matrix for binary and multiclass text and vision RAI dashboards by @imatiach-msft in #2108
  • bug fixes and tests

    • Responsible AI Dashboard

      • remove QA statistics which were mistakenly added to multilabel metrics by @imatiach-msft in #2048
      • fix multilabel RAI Vision Dashboard table view labels squished by @imatiach-msft in #2057
      • [Big data] Few bug fixes and changes by @vinuthakaranth in #2056
      • fix outline bug and add contributors to readme by @hawestra in #2081
      • Fix background color not changing with theme switch by @hawestra in #2084
      • Revert "fix outline bug" by @gaugup in #2086
      • Fix table view multi-selection by @RubyZ10 in #2087
      • [Big Data] Fix create cohort, filter range for index and dataset by @vinuthakaranth in #2089
      • Fix background color and outline by @hawestra in #2092
      • [Big data] Add telemetry by @vinuthakaranth in #2093
      • Fix bug -"All data" cohort detection is based on localized strings by @RubyZ10 in #2094
      • Retain filter selections when editing filter by @RubyZ10 in #2096
      • Adding Class Name To OD Dropdown by @natalie-isak in #2098
      • Refactor cohortList to be less than 300 lines & add datasetCohorts to context by @RubyZ10 in #2103

    • RAIInsights

      • Use cohort filter constatns from erroranalysis/responsibleai in tests by @gaugup in #2047
      • fix build failures by pinning statsmodels to older version due to numpy incompatibility by @imatiach-msft in #2053
      • Raise warning in case the test data has missing values by @gaugup in #2065
      • Add user feedback to compute() function by @gaugup in #697
      • Remove un-needed requests dependency by @gaugup in #2083
      • Raise value error for min max computation of feature range by @kicha0 in #2072
      • Move common_utils.py into responsibleai_dashboard directory by @gaugup in #2095
      • Port over logic of load_adult_income_dataset from dice-ml to unblock gates by @gaugup in #2106
      • Fix dimension mismatch error when dropped_features contains cat_features by @tongyu-microsoft in #2099

    • Error Analysis

      • Prevent overwriting of dataset in erroranalysis by @gaugup in #2050
      • fix multilabel erroranalysis metrics dropdown appearing blank by @imatiach-msft in #2063
      • release erroranalysis 0.4.4 by @imatiach-msft in #2105
      • update responsibleai and raiwidgets to erroranalysis 0.4.4 by @imatiach-msft in #2107

    • RAI Utils

      • Referencing UserConfigValidationException from raiutils instead of responsibleai by @gaugup in #1932

    • RAI Test Utils

      • Add doc strings to public functions in rai_test_utils by @gaugup in #2055
      • Add Imputation to classification pipeline in rai_test_utils by @gaugup in https://github.com/microsoft/responsible-ai-toolbox/ pull/2064
      • Add is_valid_uuid to rai_test_utils by @gaugup in #2061
      • Create release 0.2.0 for rai_test_utils by @gaugup in #2066
      • Update requirements-dev.txt in responsibleai and erroranalysis with rai_test_utils 0.2.0 by @gaugup in #2067
      • Use is_valid_uuid from rai_test_utils by @gaugup in #2069
      • Add more dataset utilities in rai_test_utils by @gaugup in #2101

    • Interpret

      • Fix robot page exception on feature importance when dropped_feature is defined by @tongyu-microsoft in #2051
      • [Big data] Fix local explanations crash by @vinuthakaranth in #2068

  • other

    • Add pyproject.toml file for isort config by @gaugup in #2052
    • Remove flaky HomeBrew Install by @gaugup in #2058
    • Update codecov-action@v2 to codecov-action@v3 by @gaugup in #2062
    • Remove rai_core_flask from paths in CI-python.yml by @gaugup in #2059
    • Update python-linting.yml to remove isort parameters by @gaugup in #2073
    • Fix ci-notebook error on failing to retrieve superconduct data zip by @tongyu-microsoft in #2102

v0.27.0

  • educational materials
    • Update explanation-dashboard-employee-attrition.ipynb to remove deprecated iteritems() and use items() instead by @gaugup in #2030
  • new features

    • Responsible AI Dashboard

      • Drop Down For Object Detection by @natalie-isak in #2010
      • Renaming Data Characteristics Tab by @natalie-isak in #2032
      • Object Detection UX & Documentation updates by @Advitya17 in #2037
      • Ensuring backwards compat b/w OD scenario and other vision scenarios by @natalie-isak in #2039

    • Model Overview

      • [Model Overview] Object Detection Metrics support by @Advitya17 in #2025
  • bug fixes and tests

    • Responsible AI Dashboard

      • [Refactor] Add preBuiltCohort to datasetCohorts in modelAssessmentContext by @RubyZ10 in #2007
      • Fix bugs 1.Save and switch doesn't do anything in cohort edit panel 2.Unable to edit cohort filters in certain scenario by @RubyZ10 in #2040

    • RAIInsights

      • make train data optional on RAIBaseInsights class by @imatiach-msft in #2029
      • responsibleai: Add test case for dataset having boolean features by @gaugup in #2042

    • Error Analysis

      • fix indexing error when creating a filtered error analysis tree view with a dataset that contains categoricals by @imatiach-msft in #2026
      • add EBM and LightGBM with TreeShap methods to calculate error correlation in error analysis by @imatiach-msft in #2018
      • release erroranalysis v0.4.3 by @imatiach-msft in #2043
      • update responsibleai and raiwidgets to erroranalysis 0.4.3 by @imatiach-msft in #2045
      • fix transient test failures by increasing timeout for gbm shap method by @imatiach-msft in #2044

    • RAI Utils

      • Move import of cohort module from raiwidgets to raituils by @gaugup in #2027

    • Interpret

      • update responsibleai to interpret-community 0.29.0 by @imatiach-msft in #2031

  • other

    • Add python 3.10 tests by @gaugup in #2033
    • fix failing builds by upgrading pip-tools after latest pip release by @imatiach-msft in #2038
    • Add 3.10 classifier for python packages by @gaugup in #2035
    • Upgrade python from 3.7 to 3.10 in few workflows by @gaugup in #2041

v0.26.0

  • educational materials
    • Update notebooks for categorical features to pass through FeatureMetadata by @tongyu-microsoft in #2011
  • new features

    • Responsible AI Dashboard

      • Adding Object Detection Fridge Data by @natalie-isak in #1998

    • RAIInsights

      • Support forecasting in RAIInsights by @romanlutz in #1948

    • Model Overview

      • [Model Overview] Metrics Template support with OD-specific optional args by @Advitya17 in #2002
      • Object Detection Model Overview Flask template by @Advitya17 in #2004
      • Model Overview: Object Detection Widgets for Aggregate Methods, Class selection, & IoU Threshold by @Advitya17 in #1997
  • bug fixes and tests

    • Responsible AI Dashboard

      • Forecasting: UI code adjustments to renamed datetime_features and time_series_id_features by @romanlutz in #1993
      • [Refactor]1.Add DatasetCohort 2. Move FeatureFlights to core by @RubyZ10 in #2003
      • [Refactor]Add columnRanges and modelType to ModelAssessmentContext by @RubyZ10 in #2006

    • RAIInsights

      • Add validations for model predictions by @gaugup in #2008
      • Add validations if target column is included in FeatureMetadata by @gaugup in #2013
      • try pickling model instead of failing based on state functions and change to UserConfigValidationException by @imatiach-msft in #2014
      • update dependencies for rai-core-flask, raiwidgets and responsibleai packages by @imatiach-msft in #2016
      • Fix deprecated DataFrame indexer type in Error Analysis package by @JarvisG495 in #2019

    • Error Analysis

      • fix bug in tree view where slider params are not used by @imatiach-msft in #1990
      • fix error analysis showing float values with many zeros for some classification task nodes by @imatiach-msft in #1991
      • release erroranalysis v0.4.2 by @imatiach-msft in #2000
      • update raiwidgets and responsibleai to erroranalysis 0.4.2 by @imatiach-msft in #2001

v0.25.0

  • educational materials
    • Update README badges by @romanlutz in #1933
    • DOC improving several error strings by @romanlutz in #1943
    • Add rai_test_utils badge in README.md by @gaugup in #1953
    • improve installation instructions for onboarding to RAI toolbox repository by @imatiach-msft in #1962
    • fix outdated example link in Error Analysis README by @JarvisG495 in #1965
    • Update example notebook links in README docs by @JarvisG495 in #1970
    • Create README.md for python tests by @gaugup in #1984
  • new features

    • Responsible AI Dashboard

      • Forecasting RAI dashboard: expand test data, add categorical what if by @romanlutz in #1921

    • Dataset Explorer

      • [Large data Data explorer] Add individual data points - data explorer for large data. by @vinuthakaranth in #1919
      • add missing true and predicted labels to axis config for dataset explorer in multilabel text and vision scenarios by @imatiach-msft in #1960

    • Counterfactual

      • [Large data counterfactuals] Use data from local counterfactuals to display in counterfactuals panel by @vinuthakaranth in #1913

    • Interpret

      • [Large data feature importance] Add individual feature importance by @vinuthakaranth in #1929
  • bug fixes and tests

    • Responsible AI Dashboard

      • Fix typo in variable name disaggregatedAnalysisBaseCohortDislaimer by @gaugup in #1855
      • add real model for dev env to have better test data by @xuke444 in #1893
      • Remove setstate() out of render function in RegressionDistributionChart.tsx by @gaugup in #1922
      • Unify responsibleai constants by @romanlutz in #1925
      • Fix bubble chart render issue due to exception in SidePanel by @gaugup in #1941
      • Fix table view feature value && scroll bar by @RubyZ10 in #1954
      • [Large counterfactuals & explanations] Do not call local counterfactuals/explanations on unselecting the point by @vinuthakaranth in #1957
      • [Large data] Map classification outcome to string values, dither, count changes by @vinuthakaranth in #1955
      • add (none) category for empty predicted or true labels in multilabel vision dashboard for data characteristics by @imatiach-msft in #1961
      • fix CI notebook build via updated yarn command flags by @imatiach-msft in https://github.com/microsoft/responsible-ai-toolbox/pull/
      • upgrade tslib to 2.5.0 by @imatiach-msft in #1973
      • Forecasting: improve description in no-features case by @romanlutz in #1978
      • Forecasting: fix dropdown scroll issue by @romanlutz in #1979
      • Forecasting: restrict transformation names to 50 chars by @romanlutz in #1981
      • Forecasting: create a better UX for value inputs by @romanlutz in #1985
      • Forecasting: turn off Error Analysis by @romanlutz in #1987
      • Forecasting: fix button text to say "switch time series" by @romanlutz in #1988
      • Forecasting: validate characters for name field in transformation creation dialog by @romanlutz in #1986

    • RAIInsights

      • raiwidgets tests: Change post_data from tuple to list by @gaugup in #1920
      • Remove duplicate load() and save() for large data by @gaugup in #1930
      • Deprecate categorical_features argument in favor of feature_metadata.categorical_features, improve metadata validation, and add forecasting-related metadata by @romanlutz in #1934
      • Add number of samples in large data in meta.json by @gaugup in #1976

    • Data balance

      • [Bugfix] FeatureBalanceMeasures: skip computing measures for col if unique vals < 2 by @ms-kashyap in #1980

    • Model Overview

      • fix mean absolute error metric calculation in model overview panel by @imatiach-msft in #1950
      • Fix selection rate by @romanlutz in #1958

    • Dataset Explorer

      • Dataset explorer: Propagate number of bins to SDK backend API by @gaugup in #1940
      • Fix robot page exception on data explorer by @tongyu-microsoft in #1963
      • Add spline chart data fetch from SDK backend by @gaugup in #1959

    • Error Analysis

      • erroranalysis: Fix the scenario when samples in dataset and string_indexed_data don't match by @gaugup in #1915
      • improve error message for invalid features passed to matrix and tree views from API endpoint in dashboard by @imatiach-msft in #1917
      • improve erroranalysis error message when invalid features passed to matrix and tree methods by @imatiach-msft in #1918
      • Fix confusion matrix Heatmap bugs by @RubyZ10 in #1926
      • Fix focus indicator is not visible on all four sides for “Tree map” and "heat map" tabs. by @vinuthakaranth in #1947
      • release erroranalysis v0.4.1 by @imatiach-msft in #1966
      • update raiwidgets and responsibleai to erroranalysis 0.4.1 by @imatiach-msft in #1969

    • Interpret

      • Adding a limit on the number of samples used for computing global explanations by @gaugup in #1924

    • Causal

      • Move sort operation of out render() in CausalAggregateView by @gaugup in #1927
      • [Large data individual causal] Add individual causal what-if to large data by @vinuthakaranth in #1942

    • RAI Utils

      • Add random sampling module to raiutils by @gaugup in #1928
      • Move ModelTask to raiutils and create forecasting utils by @romanlutz in https://github.com/microsoft/responsible-ai-toolbox/pull/
      • Add SystemErrorException class to raiutils by @gaugup in #1951
      • Update version.py in raiutils to 0.4.0 by @gaugup in #1977
      • Update raiutils to 0.4.0 in raiwidgets, responsibleai, nlp_feature_extractors and erroranalysis by @gaugup in #1982

    • RAI Test Utils

      • Migrate responsibleai tests to use utilities in rai_test_utils - Part 1 by @gaugup in #1937
      • Add more test helper functions in rai_test_utils by @gaugup in #1938
      • Create 0.1.0 release for rai_test_utils by @gaugup in #1952
      • Migrate responsibleai tests to use utilities in rai_test_utils - Part 2 by @gaugup in #1956

  • other

    • Update workflow ymls to pin pip-tools to below 6.12.2 by @gaugup in #1931

v0.24.1

  • educational materials
    • Update some sample notebooks to include dropped_features by @tongyu-microsoft in #1911
  • new features

    • Responsible AI Dashboard

      • Add forecasting dashboard - UI - time series selection & forecast comparison by @romanlutz in #1894
      • Confusion Matrix by @t-wangjohn in #1794
      • Update release workflows to include forecasting npm package by @tongyu-microsoft in #1901
      • Add forecasting dashboard - UI - transformation creation, table, and comparison by @romanlutz in #1874

    • Counterfactual

      • [Big data] Counterfactuals - Part2 by @vinuthakaranth in #1832
  • bug fixes and tests

    • Responsible AI Dashboard

      • Remove

        tags and replace with by @romanlutz in #1900

      • raiwidgets: Fix url in setup.py by @romanlutz in #1906
      • Enable logarithmic scaling, reflect cohort change and add add Revert to bubble chart button by @vinuthakaranth in #1905

    • RAIInsights

      • Move import to top level from within the function by @gaugup in #1898
      • Raise UserConfigValidationException when treatment_feature is empty list by @gaugup in #1904

    • Data balance

      • Make DataBalanceManager.compute() throw warning instead of exception by @ms-kashyap in #1902

    • Error Analysis

      • fix small data erroring out on mutual info score for error analysis guidance by @imatiach-msft in #1907
      • release erroranalysis v0.4.0 by @imatiach-msft in #1909
      • update raiwidgets and responsibleai to erroranalysis 0.4.0 by @imatiach-msft in #1910
      • Fix model wrapper to identify model type, remove task type from error analysis by @tongyu-microsoft in #1912

    • Interpret

      • Fix unwrap error in individual feature importance by @hawestra in #1897

  • other

    • Exempt only files from max-lines rule that are already in violation, but not new files (part 2/2) by @romanlutz in #1899

v0.24.0

  • educational materials
    • Add more linting instructions to CONTRIBUTING.md by @gaugup in #1810
    • Update some sample notebooks to include dropped_features by @tongyu-microsoft in #1836
    • Fix notebook links and typos by @gaugup in #1863
  • new features

    • Responsible AI Dashboard

      • add binary classification support for RAI text and vision scenarios by @imatiach-msft in #1778
      • add multilabel support to RAI dashboard and multilabel text classification covid events dataset by @imatiach-msft in #1811
      • Enable dropped_features for error analysis, data explorer and explainer, counterfactual and causal managers by @tongyu-microsoft in #1824

    • Error Analysis

      • Always display tree branching decisions by @tongyu-microsoft in #1870
      • Show branching decisions on hover over on unselected tree nodes by @tongyu-microsoft in #1881

    • RAI test utils

      • Add rai_test_utils package by @gaugup in #1777
      • Add release-rai-test-utils.yml to release rai_test_utils package by @gaugup in #1797
  • bug fixes and tests

    • Responsible AI Dashboard

      • Update e2e visit() function to not refresh the page if provided name doesn't exist in RAINotebookNames by @tongyu-microsoft in #1792
      • Fix TypeError: Cannot read properties of undefined (reading 'sort') by @vinuthakaranth in #1798
      • Fix multiple Scroll bars appear on Model demo page after setting the page to 320*256px viewport by @vinuthakaranth in #1796
      • Fix ‘Close button’ and ‘Spin button’ present in new cohort dialog does not have any title or aria-label by @vinuthakaranth in #1806
      • Fix fast pass accessibility bugs by @vinuthakaranth in #1817
      • Add min-width for highchart and fix padding by @vinuthakaranth in #1818
      • Add user feature name to box plot hover over by @gaugup in #1819
      • update to latest 16.x node version for tests by @imatiach-msft in #1809
      • Simplify route generation logic for testing with flights by @romanlutz in #1530
      • Use "lower whisker" and "upper whisker" instead of misleading "minimum" and "maximum" on boxplot tooltips by @romanlutz in #1825
      • Remove unused flight "newSdkEndpoints" by @gaugup in #1831
      • Gate call to probability distribution end point by @gaugup in #1839
      • Disable partial dependence plots for large data scenarios by @gaugup in #1843
      • add multilabel text example and fix correct and incorrect groups in data analysis by @imatiach-msft in #1848
      • Add number of cohort samples inside generateMetrics() by @gaugup in #1849
      • add multilabel RAI vision example for fridge multilabel dataset by @imatiach-msft in #1844
      • update raiwidgets to rai-core-flask v0.5.0 by @imatiach-msft in #1851
      • Remove warning for 'dropped_features are not in use currently.' by @tongyu-microsoft in #1871
      • add multilabel model overview metrics by @imatiach-msft in #1882
      • Populate probability box chart data on mount by @romanlutz in #1883
      • Improve spacing in RAI dashboard by @romanlutz in #1875
      • release raiwidgets and responsibleai 0.24.0 by @imatiach-msft in #1891
      • Refactor metric computation in ModelOverview by @gaugup in #1850
      • Remove setState call from render function in ProbabilityDistributionChart.tsx by @gaugup in #1858
      • Forecasting: boilerplate only by @romanlutz in #1887

    • RAIInsights

      • Update requirements.txt to pin econml to 0.14.0 by @gaugup in #1813
      • Remove pip pinning to unblock RAI gates by @gaugup in #1826
      • Update requirements-linting.txt to pin isort by @gaugup in #1847
      • fix loading older versions of RAIInsights due to _FEATURE_METADATA key error by @imatiach-msft in #1857
      • Add better error message if all features are dropped from dataset by @tongyu-microsoft in #1867
      • Refer to prediction wrappers from ml-wrappers by @gaugup in #1830
      • Raise exception instead of logging warnings when model cannot be loaded by @gaugup in #1886

    • Counterfactual

      • [Metadata] Fix counterfactual error on unable to update feature value by @tongyu-microsoft in #1842
      • Add test_data to ICounterfactualData by @gaugup in #1892
      • Add test_data to CounterfactualData for large data scenario by @gaugup in #1885
      • [Big data] Counterfactuals - part 1 by @vinuthakaranth in #1888
      • Add missing property 'id' in ICounterfactualData by @gaugup in #1834

    • Causal

      • Add e2e for dependency plot Avg of abs value option by @vinuthakaranth in #1768
      • Remove redundant call to serialize_json_safe() in raiwidgets causal tests by @gaugup in #1840
      • Fetching global causal effects and global policy from SDK backend by @gaugup in #1753
      • Remove typo casual and correcting it to causal by @gaugup in #1846
      • Disable local causal chart for large data support by @gaugup in #1845
      • Add better error message if causal treatment features contain dropped features by @tongyu-microsoft in #1868

    • Error Analysis

      • Update erroranalysis tests to use new rai_test_utils package by @gaugup in #1800
      • Fix Heatmap tooltip theme by @vinuthakaranth in #1808
      • impute missing values prior to calculating mutual information in erroranalysis by @imatiach-msft in #1807
      • update raiwidgets and responsibleai to erroranalysis 0.3.13 by @imatiach-msft in #1822
      • Migrate debug_ml and matrix endpoints to compute_error_tree_on_dataset() and compute_matrix_on_dataset() by @gaugup in #1795
      • in error analysis, refactor error rate metric into function by @imatiach-msft in #1742
      • fix error in erroranalysis heatmap caused by max value over right side of max bin by @imatiach-msft in #1866

    • Dataset Explorer

      • Add dataset analysis chart view for large data by @gaugup in #1829

    • Model Overview

      • Fix Model overview - Dataset cohorts]: At 320*256 viewport content is getting truncated and graph is not visible. by @vinuthakaranth in #1801
      • Model overview: add box plot in the regression case by @romanlutz in #1532
      • Send the correct column name to SDK backend for model overview distribution charts by @gaugup in #1835
      • Derive fairness metrics from metric stats instead of cohort stats by @gaugup in #1854
      • Add common APIs for model overview metrics by @gaugup in #1862

    • Interpret

      • Get global explanations from SDK if endpoint available by @gaugup in #1820
      • Disabling local feature importance view for large data scenario by @gaugup in #1828
      • Remove dead space in aggregate feature importance by @tongyu-microsoft in #1869
      • update responsibleai to interpret-community 0.28.0 by @imatiach-msft in #1879

    • NLP Feature Extractors

      • remove the unnecessary dependency pins in nlp-feature-extractors package by @imatiach-msft in #1759
      • release nlp-feature-extractors 0.1.0 by @imatiach-msft in #1821
      • remove unused dependency and method in nlp-feature-extractors package by @imatiach-msft in #1823

    • RAI Utils

      • Port cohort.py from raiwidgets to raiutils by @gaugup in #1859
      • Update raiutils to 0.3.0 in raiwidgets, responsibleai, erroranalysis and nlp_feature_extractors by @gaugup in #1873

  • other

    • Add Ke as code owner to remaining folders by @vinuthakaranth in #1799
    • Update CODEOWNERS by @tongyu-microsoft in #1805
    • merge postga branch into main by @gaugup in #1815
    • Remove postga build trigger by @gaugup in #1816
    • upgrade setup-python github action from v2 to v4 and pin to older ubuntu image for python 3.6 to fix build error by @imatiach-msft in #1838
    • Revert "Update some sample notebooks to include dropped_features" by @tongyu-microsoft in #1837
    • fix install command for speech to text example by @imatiach-msft in #1856
    • update rai-core-flask by closing WSGIServer on separate thread to fix segfault errors and get the rai-core-flask release builds working again by @imatiach-msft in #1852
    • Update maintainers by @romanlutz in #1860
    • Update requirements.txt in raiwidgets and responsibleai to pin numpy upper bound by @gaugup in #1864
    • Update requirements-linting.txt to remove pin on isort by @gaugup in #1861
    • Turn off fail-fast for python tests by @romanlutz in #1878
    • Separate out rai_core_flask CI and add paths filter by @romanlutz in #1884
    • Remove unused definitions, commented out code, etc. by @romanlutz in #1876
    • Update CI-raiwidgets-pytest.yml to skip windows 3.8 tests by @gaugup in #1890
    • Exempt only files from max-lines rule that are already in violation, but not new files by @romanlutz in #1896
    • skip CI-Python gates when pull requests are unrelated by @romanlutz in #1895

v0.23.0

  • educational materials
    • fixed the link to developer blog for housing example by @aminadibi in #1713
    • Fix developer blog link in notebook by @imatiach-msft in #1715
    • readme-enhancements by @mesameki in #1704
  • bug fixes and tests

    • Responsible AI Dashboard

      • Update UT - use new ErrorCohort() to mock data by @tongyu-microsoft in #1711
      • [A11y] 1.Error analysis tree node text alignment 2. Add ariaLabel for new cohort panel 3. Model overview chart fix by @RubyZ10 in #1719
      • reduce file size for e2e by @xuke444 in #1723
      • fix max line line error for vision by @xuke444 in #1725
      • clean up fairness by @xuke444 in #1724
      • rollback local importance selection by @xuke444 in #1726
      • add fridge dataset to test app for RAI vision dashboard and fix bug with large images in flyout by @imatiach-msft in #1730
      • fix images in data characteristics panel hiding the success/failure instances by @imatiach-msft in #1731
      • add test watch mode by @xuke444 in #1728
      • refactor vision dashboard to remove getDerivedStateFromProps by @imatiach-msft in #1732
      • [A11y]Show Error analysis tree focus && Add Screen reader for 1. New cohort panel, 2. Add component callout 3. Callout tooltip by @RubyZ10 in #1727
      • [A11y] Fix bug - Screen reader is not announcing the associated information with axis selection radio buttons in data analysis chart view by @RubyZ10 in #1736
      • Fix cohort deletion not working by @vinuthakaranth in #1738
      • fix e2e watch mode by @xuke444 in #1740
      • fix an reference issue for chart lib by @xuke444 in #1741
      • fix chart that generates duplicate plotly data by @xuke444 in #1744
      • [A11y] Fix table view keyboard access by @RubyZ10 in #1739
      • add data characteristics search and fix switching between tabs in RAI vision dashboard by @imatiach-msft in #1745
      • fix RAI dashboard for vision models crashing when selecting individual datapoints in data analysis by @imatiach-msft in #1733
      • remove duplicate settings button in RAI dashboard for images by @imatiach-msft in #1747
      • in data characteristics of RAI vision dashboard, fix bug where labels to display is reset when changing number of rows by @imatiach-msft in #1748
      • [A11y][screen reader] Announce component added after adding a component by @RubyZ10 in #1751
      • [A11y] 1. Hide chart heading & 2. Fix Add component callout high contrast focus indicator by @RubyZ10 in #1758
      • Fix full name not visible for features in highcharts by @vinuthakaranth in #1761
      • Filter out undefined filters to resolve a robot page exception by @tongyu-microsoft in #1764

    • RAIInsights

      • Correct predict call to predict_proba by @gaugup in #1717
      • Port over cohort filtering tests for 'Predicted Y' cohort to responsibleai by @gaugup in #1709
      • Raise exception in case categorical feature list doesn't have all string features by @gaugup in #1720
      • Pass classes into the sklearn confusion_matrix by @gaugup in #1674
      • fix flaky test test_rai_insights_add_save_load_save on dataset fetch by @imatiach-msft in #1734
      • Add validations to catch empty train/test scenarios by @gaugup in #1749
      • responsibleai: Move tests into respective directories by @gaugup in #1746

    • Counterfactual

      • Update requirements.txt to upgrade dice-ml to 0.9 by @gaugup in #1782

    • Causal

      • Add attributes 'outcome', 'feature' and 'feature_value' to causal global effects by @gaugup in #1750
      • Move causal only function from core-ui to causality by @gaugup in #1752

    • Error Analysis

      • Change multiclass F1_Score metrics to Macro_F1_Score/Micro_F1_Score by @gaugup in #1710
      • Fix "Clear” button of Search field under 'Feature List' dialog is not Functional and Focus indicator is not visible on all four sides for “Tree map” and "heat map" tabs. by @vinuthakaranth in #1718
      • Rename 'Error' filter name to 'Regression error' by @gaugup in #1729
      • [A11y] announce the displayed search results information of Feature List "Search" edit field. by @RubyZ10 in #1757
      • fix error analysis cohort filter for string label values by @imatiach-msft in #1766
      • fix isnan exception in erroranalysis for newer versions of pandas and numpy when calling qcut by @imatiach-msft in #1784
      • update raiwidgets and responsibleai to erroranalysis 0.3.11 by @imatiach-msft in #1767

    • Dataset Explorer

      • clean up dataset explorer by @xuke444 in #1721

    • Model Overview

      • Fix [Keyboard navigation - Azure Machine Learning - Model overview - Dataset cohorts]: Focus indicator is overlapping with the "help me choose metrics" button. by @vinuthakaranth in #1716

    • Interpret

      • in RAI text dashboard, fix topK and maxK on slidebar not updating when selecting different text documents by @imatiach-msft in #1737
      • fix table view for image and text based on discussions for local importances and data analysis by @imatiach-msft in #1735
      • Fix dependency plot showing Average of abs value by @vinuthakaranth in #1765

    • NLP Feature Extractors

      • Bump protobuf from 3.17.3 to 3.18.3 in /nlp_feature_extractors by @dependabot in #1743

  • other

    • fix lint rules by @xuke444 in #1714
    • Add postga build trigger by @gaugup in #1755
    • Update flake8 to 4.0.1 in requirements-linting.txt to fix failing linting build by @gaugup in #1760

v0.22.0

  • educational materials
    • Notebook change showing the usage of identity feature and feature metadata by @gaugup in #1627
  • new features

    • Responsible AI Dashboard

      • Vision Dashboard Table View and initial explanation flyout by @jamesbchao in #1621
      • Vision dashboard backend without data characteristics tab by @jamesbchao in #1668
      • Vision dashboard data characteristics tab by @jamesbchao in #1675
      • Vision dashboard cohorts by @jamesbchao in #1686
      • Vision model overview metrics by @jamesbchao in #1688

    • Error Analysis

    • NLP Feature Extractors

      • Add the code for the nlp_feature_extractors to the RAI Toolbox by @ilmarinen in #1580
  • bug fixes and tests

    • Responsible AI Dashboard

      • [Accessibility] Data analysis scatter chart - use symbol and color to differentiate data series by @RubyZ10 in #1667
      • Bump moment-timezone from 0.5.34 to 0.5.35 by @dependabot in #1677
      • Add validations for dashboard endpoints by @gaugup in #1672
      • [Accessibility][reflow] Error analysis tree map & Data explorer & Feature importance Reflow by @RubyZ10 in #1678
      • [Accessibility][reflow] Error Analysis heat map & Model overview & Counterfactual & Causal analysis by @RubyZ10 in #1680
      • [localization] Localize high chart context menu items by @RubyZ10 in #1691
      • for RAI text dashboard, remove radio buttons and legend text when avg of abs value local importances selected by @imatiach-msft in #1683
      • Vision dashboard UI Fixups by @jamesbchao in #1687
      • touch ups from designer feedback to RAI dashboard for text data by @imatiach-msft in #1695

    • RAIInsights

      • Update econml to 0.13.1 by @gaugup in #1660
      • [responsibleai]: Add tests for classification outcome (multiclass) and regression error by @gaugup in #1662
      • JSON serialize the endpoint output and add tests by @gaugup in #1679
      • Cache predict() and predict_proba() output by @gaugup in #1702
      • Reduce the train and test dataset sizes in responsibleai to reduce test time by @gaugup in #1630

    • Counterfactual

    • Causal

      • Add validations for heterogeneity_features in causal manager by @gaugup in #1659
      • Fix localization for causal tooltips by @gaugup in #1690

    • Error Analysis

      • [Accessibility] Error analysis screen reader by @RubyZ10 in #1663
      • Add metric computation of mean prediction by @gaugup in #1693
      • Add support for 'True Y' filter for cohort filtering by @gaugup in #1685
      • Avoid additional predict call in case of ModelAnalyzer by @gaugup in #1701
      • release erroranalysis v0.3.10 by @imatiach-msft in #1706

    • Dataset Explorer

      • refactor table view into data analysis component by @imatiach-msft in #1673
      • add mini-table to feature importances view by @imatiach-msft in #1682

    • Model Overview

      • [Model Overview] Box plot data for classification probabilities from SDK endpoint by @tongyu-microsoft in #1689
      • Change 'Accuracy' to 'Accuracy score' in ModelOverview by @gaugup in #1700

    • Interpret

      • [Accessibility][reflow] Individual conditional expectation plot by @RubyZ10 in #1699

    • NLP Feature Extractors

      • update nlp-feature-extractors readme, dependencies and add a release script by @imatiach-msft in #1666
      • fix nlp-feature-extractors release by renaming requirements file by @imatiach-msft in #1670
      • add missing positive negative words csv file to attribute extractors by @imatiach-msft in #1671

  • other

    • [localization] LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20220907180147477 to main by @csigs in #1694
    • Update CODEOWNERS by @gaugup in #1696

v0.21.0

  • educational materials
    • [docs] Add Data Balance README by @ms-kashyap in #1567
    • Replace SynapseML links with RAIToolbox links by @ms-kashyap in #1592
  • new features

    • Responsible AI Dashboard

      • Data Balance UX Part 2: Add visualizations as a flight by @ms-kashyap in #1512
      • add new interpret-vision library skeleton by @jamesbchao in #1586
      • add support for multiclass local importance values for text by @imatiach-msft in #1601
      • Vision explanation dashboard -- Image explorer view by @jamesbchao in #1587

    • Error Analysis

      • Make generic class interface for cohort based filtering by @gaugup in #1597
  • bug fixes and tests

    • Responsible AI Dashboard

      • update release workflows to include interpret-text npm package by @imatiach-msft in #1582
      • Add RAI prefix for OSS UI telemetry events by @tongyu-microsoft in #1584
      • Bump moment from 2.29.2 to 2.29.4 by @dependabot in #1538
      • Bump eventsource from 1.0.7 to 1.1.1 by @dependabot in #1456
      • [Bugfix] Fix Dropdown bug and Update styling for feature balance measures by @ms-kashyap in #1583
      • Refactor tests in test_responsibleai_dashboard_input.py to have common success and failure criteria by @gaugup in #1591
      • update raiwidgets and responsibleai to erroranalysis 0.3.6 by @imatiach-msft in #1588
      • Fix tooltip location showing up incorrectly due to conflicting ids by @imatiach-msft in #1599
      • Update e2e test to expand correct predictions caret symbol by @vinuthakaranth in #1603
      • Updates to RAI text dashboard - update test app values to latest shap release, fix predicted label, limit selection to one row, fix random crash by @imatiach-msft in #1605
      • [300 line limit] Refactor AxisConfigDialog: create AxisConfigBinOptions and AxisConfigChoiceGroup component by @tongyu-microsoft in #1609
      • [300 line limit] Refactor CohortEditor, OverallMetricChart, set max-lines rule for core-ui package by @tongyu-microsoft in #1612
      • design updates for the RAI text dashboard local importances view by @imatiach-msft in #1611
      • [Accessibility] Fix all existing accessibility bugs by @RubyZ10 in #1613
      • Move CalculateBoxPlot.test.tsx to core-ui by @gaugup in #1620
      • Add feature metadata to IDataset and JointDataset by @gaugup in #1614
      • Remove console.log() statement from CohortEditor.tsx by @gaugup in #1628
      • update raiwidgets and responsibleai to erroranalysis 0.3.8 by @imatiach-msft in #1629
      • add new interpret-vision package to npm publish workflows in build and release gates by @imatiach-msft in #1615
      • [Accessibility] fix several accessibility bugs by @RubyZ10 in #1632
      • add UI tests for RAI text dashboard by @imatiach-msft in #1636
      • Fix typo for indices by @gaugup in #1640
      • Add extra space between Identity feature and the actual feature name by @gaugup in #1641
      • Use JointDataset API in ProcessPreBuiltCohort.ts by @gaugup in #1635
      • add wrapping for long text documents to text highlighting component in RAI dashboard for text by @imatiach-msft in #1643
      • add tests to RAI text dashboard importances part 2 by @imatiach-msft in #1642
      • add radio button tests for RAI text dashboard by @imatiach-msft in #1648
      • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 69976515 by @tongyu-microsoft in #1653
      • Add unit test for calculateLineData by @gaugup in #1654
      • Add test for uncovered condition in responsibleai_dashboard_input.py by @gaugup in #1638
      • update raiwidgets and responsibleai to erroranalysis 0.3.9 by @imatiach-msft in #1657

    • RAIInsights

      • responsibleai: Add FeatureMetadata class and related tests by @gaugup in #1604
      • Add categorical statistics computation in RAIInsights by @gaugup in #1598
      • Add cohort filtering capability in RAIInsights class by @gaugup in #1618
      • Inject FeatureMetadata into RAIInsights by @gaugup in #1616
      • responsibleai: Remove old manager namespaces by @gaugup in #1564
      • Fix docs in rai_base_insights.py by @gaugup in #1650
      • fix failing RAI Insights test due to change in EA _string_ind_data type by @imatiach-msft in #1658
      • Enable validations for FeatureMetadata when passed into RAIInsights by @gaugup in #1651

    • Counterfactual

      • [300 line limit] Refactor CounterfactualList by @tongyu-microsoft in #1619
      • Reduce the time taken by RAIInsights counterfactual tests by @gaugup in #1622
      • [300 line limit] Refactor CounterfactualChart by @tongyu-microsoft in #1624
      • Add identity feature value to counterfactual hover over by @gaugup in #1626
      • Minor refactoring in counterfactual package by @gaugup in #1634
      • [Accessibility] Add headings & change counterfactual text color by @RubyZ10 in #1646

    • Causal

      • Add backend functions in causal_manager.py to handle dashboard requests by @gaugup in #1579
      • SDK change for computing aggregate causal effects and policy tree per cohort by @gaugup in #1552
      • Add identity feature value to causal hover over by @gaugup in #1633

    • Error Analysis

      • Refactor categorical statistics computation in separate module by @gaugup in #1600
      • Add more classification metrics to erroranalysis by @gaugup in #1575
      • Improve error message when categorical string column is treated as numeric and error analysis is run by @imatiach-msft in #1562
      • fix bug in treeview filtering logic in error analysis package by @imatiach-msft in #1593
      • erroranalysis: Bug fix in classification outcome cohort filter by @gaugup in #1596
      • release erroranalysis 0.3.7 by @imatiach-msft in #1606
      • fix error analysis predictions analyzer failing due to new filtering refactor by @imatiach-msft in #1625
      • Make return type of string_ind_data in process_categoricals consistent by @gaugup in #1617
      • Handle 'Classification outcome' cohort filter for multiclass scenario in SDK by @gaugup in #1623
      • Handle 'Regression Error' cohort filter for regression scenario in SDK by @gaugup in #1644
      • [Accessibility] Error analysis tree node change 1. Bubble error rate colors; 2. Bubble text & fill by @RubyZ10 in #1649
      • release erroranalysis 0.3.9 by @imatiach-msft in #1655

    • Dataset Explorer

      • Add identity feature value to dataset explorer scatter plot by @gaugup in #1645

    • Model Overview

      • If feature groups is more than 10, short it to 10 with "Other" for the rest by @RubyZ10 in #1602
      • [Model Overview]Fix bug: x axis selection is not kept between tab switch by @RubyZ10 in #1607
      • [ModelOverview] Fix bug- On choosing a specific feature cohort and trying to choose feature cohort again - dropdown has all cohorts selected by @RubyZ10 in #1608
      • [ModelOverview]Fix bug- Model Overview: Clarification - Should state of 'use line chart' be persisted when switched from Probability -> Metrics -> Prob tab by @RubyZ10 in #1610
      • enable new Model Overview experience in OSS RAI Dashboard by default by @imatiach-msft in #1647

    • Interpret

      • Add backend functions in explainer_manager.py to handle dashboard requests by @gaugup in #1585
      • Add identity feature value to feature importance hover over by @gaugup in #1637

  • other

    • update CHANGES.md release notes to reflect delay in package release by @imatiach-msft in #1581
    • Remove strict pinnings from rai-core-flask requirements.txt. by @janjagusch in #1578
    • release rai-core-flask 0.5.0 by @imatiach-msft in #1589
    • update deprecated pypi publish github action to new supported version by @imatiach-msft in #1595
    • Update CODEOWNERS to add default reviewers for core-ui by @gaugup in #1639

v0.20.0

  • new features

    • Responsible AI Dashboard

      • Model overview: add legend to probability line chart by @romanlutz in #1492
      • Model overview: add cohort selection hint by @romanlutz in #1496
      • Model overview: wrap text after max line width by @romanlutz in #1503
      • Model overview: use spline instead of line chart by @romanlutz in #1508
      • Model overview: turn on heatmap colors by default by @romanlutz in #1510
      • Model overview: disable confirm button when no changes were made by @romanlutz in #1509
      • Add e2e tests for new model overview experience (part 1) by @romanlutz in #1467
      • Model overview: use unique key to avoid heatmap color mismatch by @romanlutz in #1517
      • Model overview: new cohorts should show up by @romanlutz in #1519
      • Model overview: bold cohort names in tooltip by @romanlutz in #1531
      • add text interpretability dashboard UI by @imatiach-msft in #1159
      • Data Balance UX Part 1: add data_balance_measures to IDataset by @ms-kashyap in #1511
      • refactor individual importances view to integrate local importances view for text data by @imatiach-msft in #1569

    • RAIInsights

      • Create DataBalanceManager + unit tests; clean up data balance conftest by @ms-kashyap in #1485
  • bug fixes and tests

    • Responsible AI Dashboard

      • [office-ui upgrade] Move components in causality, counterfactuals and dataset-explorer package to fluentui by @vinuthakaranth in #1481
      • [office-ui upgrade] Move components in error-analysis package to fluentui by @vinuthakaranth in #1484
      • [office-ui upgrade] Move components in interpret package to fluentui by @vinuthakaranth in #1495
      • [office-ui upgrade] Move components in fairness package to fluentui by @vinuthakaranth in #1494
      • [office-ui upgrade] Move components in model-assessment & dependencies to fluentui by @vinuthakaranth in #1497
      • [office-ui upgrade] Move ChoiceGroup & all other remaining dependencies to fluentui by @vinuthakaranth in #1501
      • [Bugfix] HeatmapHighChart: Prioritize configOverride over defaultOptions by @ms-kashyap in #1490
      • Remove download svg option due to highcharts limitations by @tongyu-microsoft in #1498
      • Upgrade highcharts from v9.3.0 to v10.1.0, remove highcharts bundle by setting it to external module by @tongyu-microsoft in #1506
      • Fix highcharts tooltip is clipped by @tongyu-microsoft in #1502
      • Bump shell-quote from 1.7.2 to 1.7.3 by @dependabot in #1514
      • [UI refactor] Remove all alert functions and set no-alert rule by @tongyu-microsoft in #1518
      • Use of "==" and "!=" confusing to non-programmers, update operators to strings by @tongyu-microsoft in #1525
      • [UI refactor] Remove .bind() part 1 - Remove .bind(this, true/false) by @tongyu-microsoft in #1523
      • Panel buttons should be uniform: change all buttons to "Apply"; all buttons should be at bottom by @tongyu-microsoft in #1524
      • Fix slider bug by @vinuthakaranth in #1526
      • Fix e2e-widget.js for the case without flight argument by @romanlutz in #1529
      • Make the local point selection drop down consistent in causal and counterfactual by @gaugup in #1539
      • [UI refactor] Remove .bind() part2 - Remove .bind in counterfactual folder and part of .bind() in other folders by @tongyu-microsoft in #1542
      • [UI refactor] Remove .bind() part3 - Remove .bind for SpinButton in AxisConfigDialog, MultiICEPlot and Fairness by @tongyu-microsoft in #1555
      • refactor text highlighting component to use Text and Label from FluentUI instead of styles on spans by @imatiach-msft in #1558
      • centralize plot colors to core ui by @imatiach-msft in #1559
      • create other cohort in model overview when there are more than 10 cohorts by @imatiach-msft in #1572
      • [Telemetry] Add button click event tracking for main menu and error analysis component by @tongyu-microsoft in #1570
      • Remove redundant lines of code in StatisticsUtils.ts by @gaugup in #1571
      • [Telemetry] Add button click event trackers for new model overview, data explorer, feature importances, counterfactual and causal by @tongyu-microsoft in #1574

    • RAIInsights

      • update responsibleai package to latest interpret-community 0.26.0 by @imatiach-msft in #1491
      • Update requirements.txt to bump up econml to 0.13.0 by @gaugup in #1499
      • remove version for flask by @xuke444 in #1515
      • Remove unused variables from create_classification_pipeline() by @gaugup in #1520
      • update raiwidgets and responsibleai to latest erroranalysis 0.3.3 release by @imatiach-msft in #1535
      • Add column and feature range meta data to meta file by @vinuthakaranth in #1541
      • update raiwidgets and responsibleai to erroranalysis 0.3.4 by @imatiach-msft in #1544
      • update packages to new raiutils 0.2.0 and rai-core-flask 0.4.0 releases by @imatiach-msft in #1550
      • Add hierarchy to rai exceptions by @gaugup in #1568
      • Add multiclass classification test in raiwidgets by @gaugup in #1576

    • Counterfactual

      • Update counterfactual serialization via dice-ml APIs by @gaugup in #1479
      • Fix unable to input decimal point in what if counterfactual feature text field by @tongyu-microsoft in #1459
      • Bold input text for values that have changed from the original reference values for counterfactual list by @tongyu-microsoft in #1516
      • Fix bug in deserializing counterfactual explainer by @gaugup in #1554
      • Add ID to counterfactual config and serialized counterfactual data by @gaugup in #1553

    • Causal

      • Enable skipped causal unit test by @gaugup in #1565

    • Error Analysis

      • add more docs to matrix filter aka heatmap in error analysis by @imatiach-msft in #1489
      • part 2: add pyspark support to tree surrogate model in error analysis by @imatiach-msft in #1388
      • use new getNativeModel method in get_surrogate_booster_pyspark to simplify erroranalysis code by @imatiach-msft in #1475
      • fix erroranalysis erroring on serialization with numpy int64 feature names by @imatiach-msft in #1537

    • Dataset Explorer

      • Reorder controls in data explorer legend by @tongyu-microsoft in #1493
      • Fill color for data explorer box plot by @tongyu-microsoft in #1504

  • other

    • Add issues template for bugs and feature requests by @imatiach-msft in #1486
    • Taking dependency on is_classifier() method and SKLearn constants class from raiutils by @gaugup in #1289
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 65085801 by @riedgar-ms in #1500
    • Split raiwidgets/e2e-notebooks test pipeline in two by @romanlutz in #1478
    • Derandomize notebooks by @romanlutz in #1507
    • Unify naming between notebooks and test cases by @romanlutz in #1505
    • Update CODEOWNERS by @romanlutz in #1513
    • Add License and Requirements to Source Distribtion by @janjagusch in #1522
    • Move references for _convert_to_list() and _convert_to_string_list_dict() to raiutils by @gaugup in #1534
    • Move references for serialize_json_safe() to raiutils by @gaugup in #1536
    • fix flaky test by adding retry logic to fetch openml dataset in notebook by @imatiach-msft in #1547
    • release rai-core-flask 0.4.0 by @imatiach-msft in #1546
    • release raiutils v0.2.0 by @imatiach-msft in #1545
    • update sphinx dependencies to fix jinja2 doc build error by @imatiach-msft in #1551
    • Update CODEOWNERS by @tongyu-microsoft in #1557
    • [E2ETests]Add flight && screenshots by @RubyZ10 in #1566
    • fix release pipeline failing on UI widget tests by @imatiach-msft in #1573
    • fix e2e UI tests incorrectly running tests with flights when no notebook name specified and reduce number of test jobs by @imatiach-msft in #1577

v0.19.0

  • educational materials
    • add info about PR reviews to the contributing docs file by @imatiach-msft in #1472
    • Adding extra statement in PR template for e2e by @vinuthakaranth in #1461
  • new features

    • Responsible AI Dashboard

      • Model overview: improve placeholder, put confirm/cancel buttons at the bottom, and hide toggle when needed by @romanlutz in #1433

    • RAIInsights

      • Add DataBalanceAnalysis module to ResponsibleAI package to be consumed by visualizations by @ms-kashyap in #1473
  • bug fixes and tests

    • Responsible AI Dashboard

      • Fix locators logic for string features - data explorer component by @vinuthakaranth in #1441
      • Fix cohort name conflict and not run few tests for AML by @vinuthakaranth in #1442
      • Few e2e tests changes to accommodate AML static tests by @vinuthakaranth in #1445
      • Fix locators logic for string features - data explorer and model statistics components by @vinuthakaranth in #1446
      • Add more unittests RAI dashboard input class by @gaugup in #1448
      • Update the way to get the length of elements obtained in e2e tests by @vinuthakaranth in #1450
      • Fix dataset explorer plot with count not displaying by @imatiach-msft in #1454
      • Add toggle for switching classes in binary classification case by @imatiach-msft in #1444
      • [office-ui upgrade] Move checkbox to fluentui by @vinuthakaranth in #1465
      • [office-ui upgrade] Move combobox to fluentui by @vinuthakaranth in #1469
      • [office-ui upgrade] Move components in core-ui packageto fluentui by @vinuthakaranth in #1471
      • Fix TypeError: Cannot read properties of undefined (reading 'treatAsCategorical')" by @vinuthakaranth in #1452
      • Update predicted class value to uneditable text in counterfactual panel by @tongyu-microsoft in #1458
      • Collapse correct prediction by default for individual feature importance by @tongyu-microsoft in #1474

    • RAIInsights

      • Update requirements.txt to pin dice-ml at 0.8 by @gaugup in #1470

  • other

    • Add more utilities into raiutils by @gaugup in #1295
    • Release raiutils v0.1.0 by @imatiach-msft in #1480
    • Update responsibleai to raiutils 0.1.0, raiwidgets and responsibleai to erroranalysis 0.3.2 by @imatiach-msft in #1483
    • Merge postbuild branch back into main by @romanlutz in #1466
    • Remove postbuild branch triggers by @romanlutz in #1476

v0.18.2

  • bug fixes and tests

    • Responsible AI Dashboard

      • Bug fixes on 'Set value' not copying over feature values correctly in what if counterfactual panel by @tongyu-microsoft in #1416
      • Fix description for model overview by @romanlutz in #1425
      • Fix math.min / max for array size more than 10^7 by @xuke444 in #1427

    • RAIInsights

      • Add warning in counterfactual manager when unable to load explainer by @gaugup in #1412

    • Counterfactual

      • Remove "Set Value" blurb in case it is not available in counterfactual panel by @gaugup in #1426
      • Add y-axis description to counterfactual feature importance chart by @gaugup in #1423

    • Causal

      • Fix causal UI strings according to classification/regression tasks by @gaugup in #1419
      • Add the user class name to causal UI strings by @gaugup in #1422
      • Upper bound SciKit-Learn to address freeze in causal by @riedgar-ms in #1432

    • Error Analysis

      • Fix error on machines with pyspark installed where passed dataframe is not spark pandas by @imatiach-msft in #1415
      • Fix failing to create error report when filter_features is empty list by @imatiach-msft in #1421

    • Interpret

      • Filter out missing values from what if dropdown to prevent explanation dashboard from crashing by @imatiach-msft in #1418
      • Fix dependency chart axis updating with incorrect values in explanation dashboard by @imatiach-msft in #1437

  • other

    • Add postbuild branch trigger by @romanlutz in #1417
    • Upgrade numpy to fix random segfault test failures by @imatiach-msft in #1424
    • Fix flaky notebook causing build failures by adding retry logic by @imatiach-msft in #1431
    • Fix codecov and widget test screenshot uploads by @imatiach-msft in #1428

v0.18.1

  • educational materials
    • Add pre-built cohort into adult census notebook by @gaugup in #1243
    • Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb by @gaugup in #1252
  • new features

    • Responsible AI Dashboard

      • Add a summary for disaggregated analysis showing ratio and difference by @romanlutz in #1367
      • Use global cohort for disaggregated analysis by @romanlutz in #1370
      • Add accuracy as multiclass metric by @romanlutz in #1371
      • Add a top-level pivot to model overview by @romanlutz in #1375
      • Add a toggle to model overview to turn heatmap colors on and off by @romanlutz in #1376
      • Add feature and metric configuration flyouts to model overview by @romanlutz in #1381
      • Add probability distribution line chart to model overview by @romanlutz in #1382
      • Provide feature flight support to dashboard, test environment, and Python widget by @romanlutz in #1385

    • Error Analysis

      • add pyspark support to tree surrogate model in error analysis by @imatiach-msft in #1251
  • bug fixes and tests

    • Responsible AI Dashboard

      • Fix bug on scatter points in dataset explorer; remove outlier for aggregate feature importance box chart by @tongyu-microsoft in #1364
      • Change procedural loops to functions map() and filter() by @gaugup in #1255
      • Create rai-e2e package for tests shared with AML by @vinuthakaranth in #1378
      • Create e2e package for tests shared with AML by @vinuthakaranth in #1383
      • fix race condition in tree view update code which can cause root stats to be ignored on selected cohort in RAI dashboard static view by @imatiach-msft in #1386
      • String updates for model overview by @romanlutz in #1387
      • Some fixes for data explorer, heatmap, counterfactual components by @tongyu-microsoft in #1389
      • disable causal treatment whatif in static view by @imatiach-msft in #1391
      • Fix feature selection state carry-over bug in model overview by @romanlutz in #1390

    • RAIInsights

      • Rename _validate_model_analysis_input_parameters() to _validate_rai_insights_input_parameters() by @gaugup in #1373

    • Counterfactual

      • Add missing desired range to counterfactual chart by @gaugup in #1366
      • Cache and serialize the counterfactual explainer by @gaugup in #1374
      • fix big data set for counter factual chart by @xuke444 in #1393

    • Causal

      • Handle the loading of causal models more gracefully by @gaugup in #1377
      • Alignment and style fixes for causal by @tongyu-microsoft in #1392

  • other

    • Add more context to download print statement in fetch_dataset by @gaugup in #1368
    • Update PR template to be more intuitive by @romanlutz in #1116
    • Update CODEOWNERS file by @romanlutz in #1369
    • update code owner by @xuke444 in #1372

v0.18.0

  • educational materials

    • Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb by @gaugup in #1195
    • Add supported models, data types and capability matrix to README.md responsibleai by @gaugup in #1259
    • make getting-started notebook a markdown file showing APIs by @imatiach-msft in #1223
    • fix readme link to fairness and interpretability example notebook by @imatiach-msft in #1282

  • new features

    • Responsible AI Dashboard

      • Replace dependence plot with highchart lib by @zhb000 in #1208
      • Add user defined cohort injection logic into raiwidgets by @gaugup in #1237
      • Add feature importance box & bar chart by @zhb000 in #1241
      • PreBuilt cohorts UX changes by @gaugup in #1242
      • Add individual causal scatter chart by @zhb000 in #1258
      • Add what-If scatter chart from highchart lib by @zhb000 in #1262
      • add new RAI Utils package for common utilities shared across RAI packages by @imatiach-msft in #1280
      • Add ICE chart by @zhb000 in #1283
      • Add highchart for Dataset Explorer by @zhb000 in #1286
      • Add disaggregated analysis table/heatmap by @romanlutz in #1332
      • Add disaggregated analysis table to Model Overview by @romanlutz in #1341

  • bug fixes and tests

    • Responsible AI Dashboard

      • update raiwidgets to rai-core-flask 0.2.5 release by @imatiach-msft in #1221
      • Add e2e tests for Housing decision making and multiclass dnn notebooks by @vinuthakaranth in #1212
      • Refactor highchart defaultOptions by @zhb000 in #1220
      • fix categorical what-if in RAI dashboard by @imatiach-msft in #1225
      • Add scatter highchart get coordinate for e2e by @zhb000 in #1226
      • update several required dependencies by @imatiach-msft in #1219
      • add ut for DashboardSettingDeleteButton by @xuke444 in #1231
      • Create pytest fixtures raiwidgets tests by @gaugup in #1232
      • Refactor dependence plot by @zhb000 in #1230
      • Add regression test for pre-defined cohorts in raiwidgets by @gaugup in #1249
      • Set bar color to align with plot style by @zhb000 in #1248
      • fix notebook build failures due to pywinpty dependency release failing in python 3.6 by @imatiach-msft in #1257
      • refactor tabs out of RAI dashboard into a separate component by @imatiach-msft in #1256
      • minor fix to url for responsibleai package in setup.py by @imatiach-msft in #1260
      • fix whitespace in values of adult census income dataset by @imatiach-msft in #1263
      • allow rai text insights to work with RAI dashboard by @imatiach-msft in #1269
      • remove duplicate code in explanation dashboard by @imatiach-msft in #1266
      • Allow duplicating cohorts multiple times by @romanlutz in #1274
      • Disable column header highlighting on hover in IndividualFeatureImportanceView by @romanlutz in #1272
      • Rename new cohorts from "Unsaved" to "Temporary cohort" by @romanlutz in #1273
      • Reorder "Sort by" controls in local feature importance chart by @zhb000 in #1281
      • Fix cohort info styling by @romanlutz in #1277
      • update docstring for explanation dashboard in regards to min number of rows by @imatiach-msft in #1271
      • make builds more reliable by adding retry logic to urlretrieve calls in notebooks by @imatiach-msft in #1218
      • upgrade pytest to 7.0.1, remove mock and upgrade pytest-mock to 3.6.1 by @imatiach-msft in #1287
      • fix jinja2 build error and remove deprecated codecov parameter by @imatiach-msft in #1293
      • Fix min/max special case in cohort filter creation with "in the range of" by @romanlutz in #1279
      • Rename 'Dashboard navigation' to 'Dashboard configuration' by @gaugup in #1291
      • Add raiutils to PR template by @gaugup in #1290
      • Fix heatmap bug by @zhb000 in #1297
      • Make "save and switch" work from cohort settings by @romanlutz in #1276
      • add retry logic to codecov step and only upload results for one python version and platform by @imatiach-msft in #1298
      • add github action to release raiutils to pypi by @imatiach-msft in #1294
      • Update requirements-linting.txt to add flake8-pytest-style by @gaugup in #1296
      • Fix sort by absolute value from local importance chart by @zhb000 in #1299
      • Rename "base cohort" to "global cohort" by @romanlutz in #1278
      • fix codecov comment not appearing on PRs by @imatiach-msft in #1302
      • take absolute value of error calculation for regression scenario by @imatiach-msft in #1301
      • Limit individual feature importance selection to up to 5 by @tongyu-microsoft in #1305
      • Add to_json() and from_json() methods to Cohort class by @gaugup in #1300
      • Add a highchart heatmap helper class by @romanlutz in #1307
      • Fix cohort setting string by @zhb000 in #1304
      • Fix Inconsistent font size on "All data" cohort by @zhb000 in #1303
      • Add a feature flag for the new model overview experience by @romanlutz in #1306
      • Clean up charts code by @romanlutz in #1313
      • Bump minimist from 1.2.5 to 1.2.6 by @dependabot in #1292
      • fix random node download failures by upgrading to latest github action with retry logic by @imatiach-msft in #1317
      • Add dataset cohort table to new ModelOverview experience by @romanlutz in #1314
      • Add installation instructions for raiwidgets to README by @imatiach-msft in #1320
      • refactor RAIInsights into RAIBaseInsights class for basic functionality by @imatiach-msft in #1284
      • Raise UserConfigValidationException in case no model but valid model serializer by @gaugup in #1325
      • show shift to an empty cohort in tree view as an empty node by @imatiach-msft in #1318
      • Bug fixing by @zhb000 in #1326
      • Add box outlier for dataset explorer by @zhb000 in #1323
      • Update string when no datapoint selected for local importance by @zhb000 in #1331
      • Fix Big empty space for featureImportance chart by @zhb000 in #1328
      • Change warning message to user exception for model type and task type mismatch by @gaugup in #1330
      • Limit each component description width up to 750px for readability by @tongyu-microsoft in #1336
      • block empty cohort creation in RAI Dashboard by @imatiach-msft in #1335
      • Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' by @tongyu-microsoft in #1339
      • All component title and descriptions should be aligned by @vinuthakaranth in #1346
      • Remove 5K limit blurb from local explanations tab by @gaugup in #1347
      • Bump moment from 2.28.0 to 2.29.2 by @dependabot in #1333 in #1350
      • disable turbo checking for large amount of data by @xuke444 in #1351
      • force re-render when chart type changes by @xuke444 in #1354
      • Show column chart for categorical feature in data explorer by @tongyu-microsoft in #1355
      • update fluentui by @xuke444 in #1356
      • update code owner by @xuke444 in #1308
      • update version to match studio by @xuke444 in #1357
      • A few UI alignment fixes by @tongyu-microsoft in #1359

    • RAIInsights

      • Add heterogeneity_model checks by @gaugup in #1210
      • DOC add type annotations to responsibleai package by @romanlutz in #1214
      • Add data validations to SDK defined cohorts by @gaugup in #1227
      • Pin markupsafe and itsdangerous to unblock gates by @gaugup in #1238
      • Make cohortData empty list in case no pre-defined cohorts are injected by @gaugup in #1247
      • Make _cohort.py module a public module by @gaugup in #1253

    • Counterfactual

      • Counterfactual Chart: Correct target description according to task_type by @gaugup in #1261
      • Counterfactual style refactor by @zhb000 in #1275
      • Add error message for counterfactual panel by @tongyu-microsoft in #1310
      • Change the counterfactual text color from black to grey by @gaugup in #1337
      • Expand the counterfactual flyout to cover the full page by @gaugup in #1315
      • Fix what if counterfactual header and description text misaligned by @zhb000 in #1316
      • Rename counterfactual style files to confirm with -.styles.ts by @gaugup in #1338
      • Sort features by default in counterfactual flyout by @gaugup in #1312
      • Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text by
      • Rename output column name in the counterfactual flyout by @gaugup in #1353

    • Error Analysis

      • fix tree api being called twice on initial load due to uninitialized context being used by @imatiach-msft in #1229
      • fix total metric changing with different num bins when using quantile binning on diabetes dataset by @imatiach-msft in #1233
      • erroranalysis version bump in raiwidgets to 0.1.31 by @imatiach-msft in #1245
      • add clear temporary cohort button to error analysis by @imatiach-msft in #1322
      • move the root all data statistics to ErrorReport and ErrorAnalysisData by @imatiach-msft in #1344
      • update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected by @imatiach-msft in #1349
      • Disable save as new cohort button if nothing is selected in error tree by @imatiach-msft in #1327
      • update erroranalysis to 0.2.1 by @imatiach-msft in #1334
      • move the root all data statistics to ErrorReport and ErrorAnalysisData (part 2) by @imatiach-msft in #1352

    • Interpret

      • update responsibleai to interpret-community 0.25.0 by @imatiach-msft in #1343

    • Causal

      • Individual causal style responsive by @zhb000 in #1268
      • Add test case for handling different types in causal manager by @gaugup in #1321
      • Rename causal style files to confirm with -.styles.ts by @gaugup in #1342

  • other

    • Remove widget tests from CI-notebook pipeline by @vinuthakaranth in #1213
    • add missing release steps causing rai-core-flask release errors by @imatiach-msft in #1216
    • fix release pipeline by adding pytorch packages for tests by @imatiach-msft in #1222
    • fix release error on unknown shell command when uploading to pypi by @imatiach-msft in #1224
    • Correct falsey to falsely by @gaugup in #1228
    • add a builddebug yarn command to build UX locally which can be debugged in browser e2e by @imatiach-msft in #1265

v0.17.0

  • educational materials
    • add e2e test steps to CONTRIBUTING.md file for developers/contributors
    • add steps for validating UX and SDK changes to documentation
    • replace boston housing dataset with california housing dataset in tests and example notebook due to ethical concerns and biases
    • update dnn pytorch notebook example to give more interesting results by increasing model epochs
  • new features

    • Responsible AI Dashboard

      • move performance metric chart, disaggregated metric table, and StatisticsUtils into core-ui
      • add highchart, all components will be moved to highchart instead of plotly gradually
      • add metrics to StatisticsUtils - F1 score, selection rate, mean absolute error
      • add cohort and filter definitions in raiwidgets SDK
      • enable highchart exporting functionality

    • Responsible AI Dashboard for Text

      • add new interpret-text library skeleton

    • Causal

      • add treatment bar chart
  • bug fixes and tests

    • Responsible AI Dashboard

      • add e2e UI tests for Causal Analysis - classification tests
      • add e2e UI tests for Feature Importance - aggregate importances
      • add e2e UI tests for Feature Importance - individual importances
      • add e2e UI tests for What-If Counterfactuals
      • add e2e UI tests for Data Explorer - diabetes-regression-model-debugging notebook
      • add e2e UI tests for Causal Analysis - diabetes-regression-model-debugging notebook
      • add e2e UI tests for Model Overview - diabetes-regression-model-debugging notebook
      • add e2e UI tests for responsibleaidashboard-housing-classification-model-debugging notebook
      • fix e2e UI What-if sort test
      • add e2e UI tests for Feature Importance - responsibleaidashboard-diabetes-decision-making notebook
      • add e2e UI tests for What-if, Model Overview and Data explorer - responsibleaidashboard-diabetes-decision-making notebook
      • add e2e UI tests for Aggregate Causal Analysis - diabetes-decision-making notebook
      • add multiclass classification dataset to UI test app and set up basic model-assessment tests for Model Overview
      • add notebook test for pytorch example notebook and add pytorch to CI
      • move to new fluentUI dependency
      • correctly annotate python warnings
      • separate out raiwidgets and UI tests into its own workflow, and added new pipeline to run RAIwidget e2e tests
      • add js and raiwidget code coverage
      • add upload step to e2e test for CI RAIWidgets Python Typescript pipeline
      • add upload step to e2e test to CI notebooks pipeline
      • return exception details to UI
      • add docstrings for locale parameter

    • RAIInsights

      • catch dirty data conditions in RAI Insights validation
      • update responsibleai to interpret-community 0.24.0
      • update responsibleai to interpret-community 0.24.1
      • update responsibleai to interpret-community 0.24.2
      • make responsibleai managers public and add back compat connectors
      • make explanations save behavior similar to other managers
      • change logic to walk categorical columns in validate_train_test_categories
      • cover missing test scenario in state_directory_management.py
      • default categorical feature to array
      • add error analysis test for regression task in RAIInsights
      • make causal manager add and compute behavior similar to other managers

    • Counterfactual

      • add feature importance description for counterfactual local importance
      • update Counterfactual explanation text description

    • Error Analysis

      • fix importances taking too long to calculate for large data in error analysis
      • add a perf test for validating training the surrogate error tree on large data and a minor optimization for traversing the tree
      • add filtering on index to error analysis backend for RAI dashboard cohort switch
      • remove unnecessary multiple calls to tree api
      • refactor tree view state to be private static in tree view renderer and not in parent
      • create github action to release erroranalysis
      • add excludes cohort filter test in erroranalysis
      • refactor matrix filter and area state to be private static
      • make search case insensitive in feature list
      • add more docs for error analysis surrogate tree
      • optimize tree traversal logic in error analysis to reduce execution time when running on several million rows

    • Interpret

      • allow ExplanationDashboard to show limited view for more than 1k features instead of erroring

    • Causal

      • improve documentation for nuisance_model and heterogeneity_model
      • implement list method in causal manager

  • other

    • fix pypi releases by reverting back to pypi token per package
    • fix dummy counterfactual feature importance values for UI test app
    • add pseudo language for localization test
    • integrate codecov into responsible-ai-toolbox repository
    • remove test warning due to use of deprecated np.object
    • add python 3.9 tests
    • fix ipython dependency in rai-core-flask and remove from raiwidgets
    • update setup.py files to carry python 3.9 classifiers
    • fix broken builds due to pip upgrade which broke pip-tools
    • remove mac temporarily from rai widgets step to fix failing builds
    • remove node-version from CI-python.yml
    • fix flaky flask test
    • improve description for responsibleai and raiwidgets pypi packages

v0.16.0

  • educational materials
    • major readme refactoring changes + filename changes for public release of RAI Dashboard
    • add missing doc for pred_y_dataset and min_child_samples
    • add npm and yarn installation instructions
  • new features

    • Counterfactual

      • counterfactual list style updates: highlight counterfactual examples columns with different color, and bold size
      • highlight first and last row of counterfactual list
      • implement list method for counterfactual manager

    • Error Analysis

      • add scrollbar area around matrix cells when there are many categories
      • add warning message for downsampled explanation data in ErrorAnalysis
  • bug fixes and tests

    • Responsible AI Dashboard

      • add e2e UI tests for Dataset Explorer - whisker plot
      • add e2e UI tests for Dataset Explorer - scatter plot
      • add e2e UI tests for Dataset Explorer - flyout
      • add e2e UI tests for Dataset Explorer - cohort scenarios
      • add e2e UI tests for Model Statistics - chart tests
      • refactor binding this with arrow func
      • add error message for duplicate cohort name
      • handle context cohort delete cases for data explorer

    • RAIInsights

      • throw error on mutating predict function in RAIInsights
      • use spilt orientation to save train and test dataframe to address duplicate indices condition in them
      • add .json extension to dtypes files and add tests
      • add predict and predict_proba outputs to RAIInsights save method
      • refactor RAIInsights load function into smaller functions
      • refactor classifier detection logic into common function
      • fix issue with missing directories
      • gracefully handle failure in loading model.pkl from disk
      • remove unused test dependencies responsibleai
      • add missing unit tests for _convert_to_list()
      • separate out save() and load() tests
      • add more unit tests for serialize_json_safe()
      • update responsibleai to interpret-community 0.23.0

    • Counterfactual

      • add e2e UI tests for what-if counterfactuals - creating counterfactuals

    • Error Analysis

      • fix coverage calculation for cell tooltips in heatmap for recall, precision and f1 score metrics
      • fix handling numpy types in heatmap for categories

    • Interpret

      • improve state management for explainer manager
      • test for save/load/save bug in Explanations
  • other
    • fix top broken image link in tour notebook on github
    • add isort to build pipelines
    • fix SubBarChart flaky test
    • update notebook tests for new notebooks
    • add flake8-bugbear for better python code contructs
    • add flake8-breakpoint to avoid code checkin with active breakpoints
    • remove csv files from source for cognitive services example notebooks
    • add yarn e2e-watch shortcut
    • replace existing user token with shared account token for releases

v0.15.1

  • educational materials
    • Notebooks updated for new class names (ResponsibleAIDashboard etc.)
  • new features
    • None
  • breaking changes

    • RAI Insights/Model Analysis

      • Rename explainers directory to generators when saving RAI Insights object
  • bug fixes

    • RAI Insights/Model Analysis

      • Fix Issue #1046, related to a save/load/save cycle when no explainer present
  • other
    • Extra tests for WhatIf counterfactuals
    • Checks on the UUID directory structure of RAI Insights

v0.15.0

  • new features

    • Model Analysis

      • move raw data into /data directory
      • perform schema validations during compute and load
      • rename model analysis to rai insights
      • rename ModelAnalysisDashboard to ResponsibleAIDashboard
      • directory based state management for model analysis tools
      • add .json extension to train and test dataset

    • Error Analysis

      • port ErrorReport schema validation for error analysis to responsibleai
      • add error analysis error importances (aka correlation of features with error) to model analyzer API
      • add support for special classification outcome filter to error analysis backend to work with model analysis filters

    • Counterfactual

      • remove save as new data point option for read-only mode on What-if counterfactual
      • split counterfactual output into individual json files
      • add support for non-string categorical values
  • bug fixes

    • Model Analysis

      • fix cohort text out of button box
      • fix bug when the tree node has 0 errors and N correct, the raw data table shows the opposite with N+1 errors and -1 correct.
      • when classes are not passed, sort them automatically in RAIInsights
      • fix composite filter issues

    • Counterfactual

      • fix sort feature columns by counterfactual

    • Error Analysis

      • fix map shift text not being clear, provide conditional dialog text based on current view
      • fix tree map connection line doubled after switching to dark theme

  • other

    • update readme links to be absolute instead of relative path
    • add more UX IFI tests for responsibleaitoolbox-classification-model-assessment notebook
    • several E2E UX test fixes
    • add E2E UX tests for Feature Importance - Cohort functionality
    • add test to render ModelAnalysisDashboard after save and load
    • add test to render ResponsibleAIDashboard after save and load
    • add E2E tests for What-if Counterfactuals - common functionalities & Y Axis Flyout
    • add separate github workflow for python linting

v0.14.0

  • educational materials
    • add Cognitive Service fairness assessment notebooks
  • new features

    • Error Analysis

      • add new accuracy and f1-score metrics in erroranalysis python backend and UI frontend
      • add importances to ErrorReport for error analysis
      • make ErrorReport, PredictionsAnalyzer and ModelAnalyzer public in erroranalysis package
      • display message bar after saving a cohort in error analysis dashboard

    • Counterfactual

      • implement save and load for counterfactual manager
      • make load method uniform across all RAI managers
      • perform schema validations on serialized counterfactual output

    • Interpret

      • upgrade interpret-community to 0.22.0
  • bug fixes

    • Model Analysis

      • remove drag and drop instructions in the dashboard navigation
      • add theme support for model statistics and dataset explorer
      • remove setup_explainer after ModelAnalysis load call in tests
      • add theme support for feature importances, causal, counterfactual tabs
      • hide opened feature list when clicking on the heatmap tab in model analysis
      • add localization for delete cohort dialog

    • Interpret

      • set is_run class variable in explainer manager

    • Causal

      • add init file to causal schema directory

    • Error Analysis

      • fix color for metrics that are not error metrics to be green in legend and heatmap nan cells
      • update features list in static view to have apply button removed and labels disabled for tree view parameters
      • update features list text for static case
      • fix running what if on categorical numeric values in error analysis dashboard
      • add theme support for error analysis view
      • fix for importances in static view causing failures when unspecified
      • fix feature list alignment issue

  • other

    • add documentation for the release process
    • add E2E test for model analysis, running from python jupyter notebook and validating in cypress
    • sort all python imports using isort
    • add aggregate feature importance tests for responsibleaitoolbox
    • add E2E individual feature importance tests for responsibleaitoolbox-classification-model-assessment notebook
    • fix image links in responsibleaitoolbox-classification-model-assessment.ipynb notebook to reference images in folder instead of using embedded images directly
    • rename causal tests

v0.13.0

  • new features

    • Model Analysis

      • add "Save amd switch" button to cohort creation
      • pass classes parameter to both error analysis dashboard and error analysis manager in model analysis
      • upgrade interpret-community to 0.20.0

    • Error Analysis

      • update erroranalysis to 0.1.24
  • bug fixes

    • Model Analysis

      • fix issue with widgets rendering with 8px height

    • Error Analysis

      • fix precision and recall score failing on string labels in error analysis
      • fix multiclass precision and recall calculation for NA zero values in heatmap
      • edit string in error analysis cohort creation
      • remove text in heatmap suggesting to select features when in static view
      • minor doc fix for examples in error analysis dashboard

    • Counterfactual

      • fix horizontal scroll bar disappears for small screen in what-if panel

    • Fairness

      • fix data issue caused robot page exception on fairnessDetails
  • other
    • fix broken link to getting started notebook
    • fix broken image link on main readme
    • fix the issues with sample notebooks markdown images
    • delete old notebooks
    • change tests to run on new notebooks, add new notebook to tests

v0.12.1

  • new features

    • Error Analysis

      • add disabled tree view and heatmap for static view
  • bug fixes

    • Error Analysis

      • fix class length causing labels to not be shown in simple view of Error Analysis
      • fix cohort filter tooltip behind cohort info panel

    • Causal

      • fix trace0 appearing on hover tooltip in plots

    • Counterfactual

      • handle cases where counterfactuals failed to compute
      • add tooltip for counterfactual info
  • other
    • some grammatical errors were corrected on documentation page

v0.12.0

  • new features

    • Model Analysis

      • add global cohort statistics

    • Error Analysis

      • add metric selector dropdown
      • add tree view parameters as sliders to features selector panel
      • add tooltip to heatmap categories and left-justify the heatmap

    • Counterfactual

      • upgrade dice-ml to 0.7.2
  • bug fixes

    • Model Analysis

      • fix cohort information in heatmap resizing in model analysis
      • log warning when regression model has predict_proba()

    • Error Analysis

      • fix error rate gradient for classification scenario
      • fix quantile binning toggle should stay on when making actions
      • fix multiple issues in instance view, including:
        • when selecting multiple rows in correct and incorrect panels, and going to all selected, if a point is removed in the middle all of the points below get removed
        • when removing points in the all selected panel, the count does not get updates in the correct/incorrect panels until they are selected
        • when using what if, make points not selectable
      • fix precision and recall calculation in the UI changing with number of bins
      • decrease the lower bound on first category for quantile binning
      • fix handling of numeric features names in pandas for error analysis tree view
      • reverse y axis in heatmap so that higher values are towards the top of the matrix
      • add warnings for 2d heatmap case and test for quantile binning with duplicate edges

    • Causal

      • sort causal data point by effect size
      • set treatment list spin range
      • fix causal global effect plot x-axis titles cut off
      • fix causal categorical treatment average gains text

    • Counterfactual

      • change current class label for regression task
      • raise exception when users configures less than required counterfactuals
      • show options for categorical features on counterfactual list

    • Fairness

      • add error handling for missing metrics in fairness
  • other
    • remove Axios library from dependencies
    • rename wrapped-flask to rai_core_flask
    • remove deprecated babelconfig option in workspace.json

v0.11.0

  • new features
    • table for list of available cohorts
    • added more metrics to ErrorAnalysisDashboard, including precision, recall and mean abs error
  • bug fixes
    • fix various UI usability bugs
    • fix for clicking custom scatter plot point

    • Error Analysis

      • fix selecting what if datapoint crashing dashboard
      • fix error in quantile binning when there are too many duplicate points
      • fix displayed error rate in all data cohort in cohort list
      • fix error in heatmap for classification case
      • fix features list not getting disabled in static view
      • update description text for static view
  • other
    • improve validation for model prediction
    • reorder counterfactual table columns
    • add absolute sorting for local importance charts
    • improve spacing and fonts across dashboard
    • fix typos in docstrings
    • improve wording of text on dashboard
    • add warnings for soft data limits
    • remove font override for all components
    • add code coverage to repository

v0.10.0

  • educational materials

    • update current model analysis classification notebook by adding treatment features for causal analysis

  • new features

  • breaking changes

  • bug fixes

    • Model Analysis:

      • change icon for ViewList in main menu
      • fix model statistics component not showing all cohorts
      • put causal component last, remove navigation
      • hide removal dialog on click
      • move cohort info and button to sticky menu, replace error stats with filters

    • Error Analysis:

      • refactor heatmap code and fix display issues for categoricals
      • fix incorrect values in heatmap when there are empty error cells
      • added quantile binning and number of bins functionality to erroranalysis package
      • added quantile binning and number of bins to UX and hooked up calls to backend
      • added tooltip to list of features for tree view
      • fix backend to support equals filter for integer categorical columns
      • fix backend to support include and exclude filters for integer categorical columns
      • update default value for matrix feature
      • fix cohort when switching tree view

    • Causal:

      • remove unnecessary z-index on what-if and causal components
      • update causal policy to support categorical features
      • format policy tree categorical features
      • fix description on how confounding features are handled
      • use lighter lines for causal plotly point graphs

    • Fairness:

      • incorporate uncertainty quantification in fairness widget v2

    • Counterfactuals:

      • upgrade dice-ml to 0.7
      • update counterfactual list to allow string values
      • extend unseen categorical check to counterfactuals
    • add dropdown to dataset explorer by default
    • fix removing what if point will keep a faded start on the chart
    • suppress distracting logging from flask when running widgets
    • fix large x and y axis going out of bounds
    • refactor multiple instances of convert to list method in several dashboards
    • add missing interpretability information and guidelines

  • other

    • fix message and check for environment in release pipeline

v0.9.4

  • bug fixes
    • fixed causal treatment policy display issue happening when switching between features

v0.9.3

  • educational materials
  • new features
    • causal treatment cost argument can be passed per feature/sample/category
    • target added to counterfactual tabular output
    • causal analysis now trains the model on training data, not both train and test
    • min_child_samples parameter for error analysis
  • breaking changes
  • bug fixes
    • improved text for causal what-if
    • fix for missing values in dataset
    • improvement for default explanation tab
    • fix for multiple feature importances detail view
    • fix for causal what-if slider being stuck
    • fixes for cohort selection
    • fixes for error analysis matrix view
    • fix for extra scroll bar in counterfactuals view
    • fix for train data input in classification notebook
    • fix for bug in counterfactual feature_to_vary parameter
    • fix for x and y axes of error analysis heatmap
    • fix for counterfactual predict
  • other
    • improved error messages for responsibleai ModelAnalysis
    • improved validation when not passing a model to ModelAnalysis
    • updated ModelAnalysis classification notebook to use separate train and test datasets

v0.9.2

  • educational materials
  • new features
    • Model Analysis:
      • add search box to counterfactuals panel
      • add support for feature to vary and permitted range in counterfactual manager
    • add matrix features and tree features to error report object in erroranalysis package
  • breaking changes
  • bug fixes
    • call flask service credential
    • Model Analysis:
      • fix hardcoded feature name in counterfactual panel
      • fix refresh on global cohort change
      • add tree_features and matrix_features params to static tree and heatmap
      • update sticky menu
      • change counter factual desired_range to tuple
    • fix aggregate feature importance chart for multiclass classification
    • fix serialize_json_safe not escaping special characters which caused dashboard to fail to render on pandas dataframe that had double quotes in string values
  • other
    • add dice-ml to intersphinx_mapping
    • add missing documentation for counterfactual manager
    • fix python responsibleai package test for windows

v0.9.1

  • educational materials
  • new features
  • breaking changes
    • error report in erroranalysis package had json_tree renamed to tree and json_matrix renamed to matrix
  • bug fixes
    • fix econML inputs
    • add categorical feature to causal whatif
    • fix counterfactual panel scroll bar
    • round precision in causal analysis tables
    • fix calculating feature importances after error report due to dataframe on analyzer being modified
  • other
    • dump information about the most time consuming tests

v0.9.0

  • educational materials
    • add Model Analysis notebook for regression
  • new features
    • local importance chart for counterfactual dashboard
    • add multi causal policy support
    • add delete tab dialog and remove inline widget for model assessment dashboard
    • add treatment policy and table to causal analysis dashboard
    • create new custom individual feature importance component
  • breaking changes
  • bug fixes
    • Model Analysis:
      • sticky layer for cohort and dashboard settings
      • restrict EA tree to 80% width
      • restrict ipykernel<0.6
      • fix flyout title and description
      • fix add button behavior by adding components in the appropriate spot
      • show add button only for components that can render
      • add validations for input parameters for Model Analysis class
      • validate treatment features in causal manager
      • explicitly pass model task to MimicExplainer in model analyzer
    • Error Analysis:
      • remove 100k row limit
      • fix zero nodes in tree view
      • fix search functionality in features list
      • fix displayed filter order in tree view to start from root node
      • raise warning for older version of pandas when feature names contain '-'
      • expand number of decimals shown in filter tooltips when values below 0.1
      • fix regression case for 1d heatmap matrix metrics being calculated incorrectly
    • set original data from dropdown for counterfactuals
    • use test data instead of train data for error analysis manager in model analyzer
    • causal analysis to keep all levels in dataframe outputs
    • pin networkx pip package to prevent installation of matplotlib
    • add target_column causal policy gain to UI control
  • other
    • unit tests for multiclass classification for causal analysis
    • add model analysis notebooks to notebook gate
    • add test for causal manager for None categoricals
    • fix example code for model assessment

v0.8.0

  • educational materials
  • new features
    • enabled categoricals on MimicExplainer for the explainer manager
    • updated causal constants to enable linear models and allow high cardinality features by default
    • support newest policy output format from econml
  • breaking changes
    • continuous_features in CounterfactualManager.add() is deprecated in favor of categorical_features in ModelAnalysis
  • bug fixes
    • fixed error analysis add after model analysis deserialization
  • other
    • improved tests for visualization dependencies

v0.7.0

  • educational materials
  • new features
    • added widget serialization for causal analysis
  • breaking changes
  • bug fixes
  • other

v0.6.1

  • educational materials
  • new features
  • breaking changes
  • bug fixes
  • other

v0.6.0

  • educational materials
  • new features
  • breaking changes
  • bug fixes
  • other

v0.5.0

  • educational materials
    • cleanup commented out explainers from interpretability notebook
  • new features
    • added support for causal, counterfactual, and error analysis tools in responsibleai
    • added model analysis widget to raiwidgets
    • added support for regression to error analysis
    • added policy tree to causal manager
  • breaking changes
    • renamed package raitools to responsibleai
    • renamed top-level class RAIAnalyzer to ModelAnalysis
    • removed support for fairness in responsibleai package
  • bug fixes
    • fixed bug in precision statistic calculation in dashboard
    • fixed bug in loading explanations with ModelAnalysis
    • fixed on predict method to use correct dtypes
    • fixed individual importances chart when no data available
  • other
    • create combined feature importances tab for local and global explanation in model assessment
    • add section headers to model assessment dashboard
    • add manual cohort creation to the model assessment dashboard

v0.4.0

  • educational materials
    • cleanup commented out explainers from interpretability notebook
  • new features
    • fix databricks environment with raiwidgets dashboards
    • updates to ErrorAnalysisDashboard:
      • improve the features list to use DetailsList
      • added tooltips to legend
      • initial static view for error analysis, specifically matrix filter and tree view
      • fixed tooltip node hover
      • fix indexing issue on cohort shift
      • add new simple dashboard implementation, which only takes in predictions and does not need model or dataset
    • updates to ModelAssessmentDashboard:
      • set up tabs on the left side
      • add model statistics tab
      • add pivot for error analysis
      • remove fullscreen button
    • refactor error analysis python code into separate common package
  • breaking changes
  • bug fixes
    • reduce the lower-bound on the lightgbm dependency
    • update greenlet and gevent dependencies to fix releases
  • other
    • use global context in interpret, error-analysis, and model-assessment
    • moved dataset explorer into a new project dataset-explorer
    • created new projects causality and counterfactuals
    • remove fairness v1 dashboard

v0.3.1

  • Bug fix:
    • fix public VM failing requests in rai-core-flask package, update raiwidgets to latest rai-core-flask package

v0.3.0

  • Bug fixes:
    • Fix all data cohort always appearing first in cohorts lists
    • Add number selected text under local explanation radio buttons
    • Fix the issue that categorical feature is not able to change in cohort editor.
    • Fix categorical feature in what-if tab

v0.2.2

  • Bug fix:
    • Fix categorical feature in ice plot
    • Left align tabs in explanation window of error analysis
    • Move the selected features and feature importances inside a scrollable pane
    • Show aggregate plots by default in dataset explorer
    • Fix indexing issue in heatmap when selecting all cells

v0.2.1

  • educational materials:
    • add imports for MimicExplainer and PFIExplainer in the interpretability-dashboard-employee-attrition notebook
  • features
    • migrate cohort and dataset utilities to core-ui
    • remove circular dependencies
    • add confirmation popup in ExplanationDashboard when cancelling editing a cohort.
    • fix clipped dropdown in FairnessDashboard
    • keep FairnessDashboard chart state in single model view
    • updates to ErrorAnalysisDashboard:
      • in large data view, change cohort state to use full data instead of downsampled explanation data
      • add true y and predicted y values
      • fix categorical labels handling in what if panel
      • add breadcrumb to error explorer view and minor style adjustments to navigation panel
      • fix indexing issues for all data cohort
      • add color column to local explanation view

v0.2.0

This is the first release of the raiwidgets package.

  • educational materials:
    • notebook analyze-categoricals-binary-classification-local.ipynb added
    • notebook erroranalysis-pretability-dashboard-breast-cancer.ipynb added
    • notebook erroranalysis-interpretability-dashboard-census.ipynb added
    • notebook fairness-credit.ipynb added
    • notebook fairness-dashboard-loan-allocation.ipynb added
    • notebook fairness-interpretability-dashboard-loan-allocation.ipynb added
    • notebook interpretability-dashboard-employee-attrition.ipynb added
  • features
    • FairnessDashboard changes compared to fairlearn's FairlearnDashboard which was migrated to raiwidgets:
      • Restructured performance metric selection as list rather than set of tiles for easier navigation.
      • Added fairness metric selection using a grouped list.
      • Added dropdowns to model comparison view and single model view to allow users to quickly switch between sensitive features, performance metrics, and fairness metrics.
      • Added metric table to single model view.
      • Added dropdown to switch between charts in single model view.
      • Simplified API to extract sensitive feature names directly from sensitive_features.
    • ErrorAnalysisDashboard: first release
    • migrated ExplanationDashboard from interpret-community to raiwidgets

We're skipping v0.1.0 to get all our packages onto the same version:

  • pypi:

    • raiwidgets: first real release

  • npm:

    • @responsible-ai/fairness: previously 0.1.*
    • @responsible-ai/interpret: previously 0.0.*
    • @responsible-ai/error-analysis: first real release
    • @responsible-ai/mlchartlib: previously 0.0.*
    • @responsible-ai/core-ui: previously 0.0.*
    • @responsible-ai/localization: previously 0.0.*