Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add wandb support #16

Merged
merged 20 commits into from
Jan 10, 2023
Merged

Add wandb support #16

merged 20 commits into from
Jan 10, 2023

Conversation

adamwawrzynski
Copy link
Collaborator

@adamwawrzynski adamwawrzynski commented Dec 30, 2022 •
edited

Description

This PR introduces support for Weights and Biases as experiment tracking tool. It adds custom callback class which logs artefacts from XAI algorithms to the W&B experiment. It also adds example script to run test training on MNIST dataset with wandb callback.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

  • manual testing

Checklist:

  • I have updated all related pyproject.toml files.
  • I have updated lock files.
  • I have set an Assignee and Reviewers.
  • I have labeled the PR correctly.
  • I have performed a self-review of my own code.
  • I have covered my code and changes with unit tests.
  • I have annotated types as extensively as possible.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • All code references and dependencies will work.

Add wandb callback and example
63ec4c2 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
@adamwawrzynski adamwawrzynski self-assigned this Dec 30, 2022
@adamwawrzynski adamwawrzynski added the enhancement New feature or request label Dec 30, 2022
Adam Wawrzyński added 7 commits December 30, 2022 12:00
Refine docstrings, remove unused optional fields
fd5bfc9 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Add CLI program to add explanations to existing W&B experiment run
3aa451c 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Filter out artifacts from other experiments in update program
6856268 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Remove commented out import
d39b29d 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Add support for providing JSON config with desired explainers and the…
60eb9ce 
…ir parameters

Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Update README, add 'Getting started' section with example, add table …
ea3b39e 
…of content

Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Add CLI tool, fix dependencies to be installed with wheel
3f69dc1 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
adamjankaczmarek
adamjankaczmarek reviewed Jan 4, 2023
View reviewed changes
README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
autoxai/callbacks/wandb_callback.py Show resolved Hide resolved
autoxai/callbacks/wandb_callback.py Outdated Show resolved Hide resolved
autoxai/callbacks/wandb_callback.py Outdated Show resolved Hide resolved
autoxai/callbacks/wandb_callback.py Outdated Show resolved Hide resolved
autoxai/explainer_helper.py Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
autoxai/callbacks/wandb_callback.py Show resolved Hide resolved
autoxai/callbacks/wandb_callback.py Show resolved Hide resolved
Adam Wawrzyński added 4 commits January 4, 2023 12:39
Add variants for dependencies based on Python version and platform, f…
b937d2b 
…ix installing from wheel

Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Move loggin images to at training start, rename arguments, minor fixes
31ffc19 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Remove specifying CUDA version of torch package in pyproject, update …
330d226 
…docs

Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Change default number of logged artifacts
e65f041 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
kamilrzechowski
kamilrzechowski reviewed Jan 4, 2023
View reviewed changes
autoxai/callbacks/wandb_callback.py Outdated Show resolved Hide resolved
autoxai/explainer_helper.py Outdated Show resolved Hide resolved
autoxai/cli/experiment_update.py Outdated Show resolved Hide resolved
autoxai/cli/experiment_update.py Outdated Show resolved Hide resolved
autoxai/callbacks/wandb_callback.py Outdated Show resolved Hide resolved
autoxai/explainer_helper.py Outdated Show resolved Hide resolved
Adam Wawrzyński added 4 commits January 9, 2023 13:27
Merge branch 'main' into feature/wandb_callback
282724f
Update wandb callback with context manager
430994c 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Add context manager to CLI app, add explainers to package __init__.py
6a30696 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Fix wrong variable name: rename prediction to target_label
288adf8 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Bump minor version
4576ece 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Fix displaying warnings in context manager
0b3d10a 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
kamilrzechowski
kamilrzechowski approved these changes Jan 9, 2023
View reviewed changes
Copy link
Collaborator

@kamilrzechowski kamilrzechowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Adam Wawrzyński added 2 commits January 10, 2023 08:41
Add type aliases, fix typo, remove unnecessary unpacking batch in ite…
3c48dd3 
…ration

Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
Rename type aliases
9a96c96 
Signed-off-by: Adam Wawrzyński <adam.wawrzynski@reasonfieldlab.com>
@adamwawrzynski adamwawrzynski merged commit b2c38fa into main Jan 10, 2023
@kamilrzechowski kamilrzechowski deleted the feature/wandb_callback branch January 10, 2023 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adamjankaczmarek adamjankaczmarek adamjankaczmarek left review comments

@kamilrzechowski kamilrzechowski kamilrzechowski approved these changes

@rpytel1 rpytel1 Awaiting requested review from rpytel1

Assignees

@adamwawrzynski adamwawrzynski

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@adamwawrzynski @adamjankaczmarek @kamilrzechowski