Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new SPI (GWTau) and benchmark generation script + workflow. #66

Merged
merged 17 commits into from
Apr 10, 2024

Conversation

jmoo2880
Copy link
Collaborator

In this PR, I've made the following changes:

  • Added a new manually executable Git Actions workflow for generating benchmark tables for SPI testing called run_datset_generation.yaml. Upon running (via Git Actions portal), the CML7.npy dataset will be loaded and SPIs will be computed on the same dataset for a fixed number of iterations (using a new Calculator each time). The mean and standard deviation for each SPI's output matrix (MPI) is then saved as a dictionary and uploaded as a .pkl file that can be downloaded as an artefact in Git Actions. Each time a new SPI is added, or a major change (e.g., dependency update) is made, the workflow can be manually triggered to create a new baseline set of SPI outputs.
  • Added a new pairwise distance measure - Gromov-Wasserstain (GWTau) distance. See here for more details about the specific implementation.
  • Added a new in-depth tutorial for integrating SPIs into the pyspi framework using GWTau as an example. See the docs for a full explanation of the validation/testing process of this SPI using the dataset provided in the original research paper.
  • Added new benchmark dataset tables which include reference outputs for the new GWTau SPI.
  • Updated the default (config.yaml) and fast (fast.yaml) subset configuration files to include GWTau.
  • Updated SPI output testing.

@jmoo2880 jmoo2880 changed the title Add new SPI and benchmark generation script + workflow. Add new SPI (GWTau) and benchmark generation script + workflow. Apr 10, 2024
@jmoo2880 jmoo2880 merged commit e2fe31d into main Apr 10, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant