Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for fast data types #5

Closed
mravanelli opened this issue Apr 28, 2020 · 1 comment
Closed

Add support for fast data types #5

mravanelli opened this issue Apr 28, 2020 · 1 comment
Assignees
@aheba
Labels
enhancement New feature or request

Comments

@mravanelli
Copy link
Collaborator

One option is the hdf5 data type.
Another is super fast data loading from one of our collaborators (I've forgotten which one).
@mravanelli mravanelli added the enhancement New feature or request label Apr 28, 2020
@nauman-daw
Copy link
Collaborator

Hi,

HDF5 v/s pkl

As we discussed earlier HDF5 format.

Here are a few observations I made during my x-vector PLDA experiment on Voxceleb1 dataset.
I thought of sharing this here as it may be useful in the future.

HDF5 takes less storage space as compared to pkl format.

8.7M	VoxCeleb1_enrol_rvectors.h5
 28M	VoxCeleb1_enrol_rvectors.pkl

8.7M	VoxCeleb1_test_rvectors.h5
28M	VoxCeleb1_test_rvectors.pkl

271M	VoxCeleb1_training_rvectors.h5
872M	VoxCeleb1_training_rvectors.pkl

However, I found the loading time is high in the case of hdf5.

HDF5:

(env_sdk) nauman@Naumans-MacBook-Air-2 sdk $ python plda_sdk.py
hdf5: loading Xvector Voxceleb-1 train file:  7.521356105804443
(env_sdk) nauman@Naumans-MacBook-Air-2 sdk $ python plda_sdk.py
hdf5: loading Xvector Voxceleb-1 train file:  5.630152940750122
(env_sdk) nauman@Naumans-MacBook-Air-2 sdk $ python plda_sdk.py
hdf5: loading Xvector Voxceleb-1 train file:  5.519376039505005

pkl:

(env_sb) nauman@Naumans-MacBook-Air-2 models (gauss-plda) $ python PLDA.py
pkl: loading Xvector Voxceleb-1 train file:  0.618098258972168
(env_sb) nauman@Naumans-MacBook-Air-2 models (gauss-plda) $ python PLDA.py
pkl: loading Xvector Voxceleb-1 train file:  0.6196680068969727
(env_sb) nauman@Naumans-MacBook-Air-2 models (gauss-plda) $ python PLDA.py
pkl: loading Xvector Voxceleb-1 train file:  0.6318840980529785

30stomercury pushed a commit that referenced this issue Sep 21, 2020
Merge pull request #5 from 30stomercury/dev
fb47a98 
add coverage penalty
@samuelazran samuelazran mentioned this issue May 16, 2021
Merged
2 tasks
ycemsubakan referenced this issue in ycemsubakan/speechbrain-1 Dec 29, 2022
@ycemsubakan
Merge pull request #5 from ycemsubakan/deep_class
222717c 
deep classifier + adaptation
anautsch pushed a commit that referenced this issue Mar 21, 2023
@ycemsubakan @fpaissan
starting the cleaning for the piq code (#5)
973c93d 
* starting the cleaning for the piq code

* L2I running on logspectra - checking exp results

* starting the migration of classes / functions to PIQ.py

* running with all checkpoints

* L2I both modalities w/ inp fidelity

* fixed paths; HF checkpoints

* Trying to fix L2I baseline for logspectra

* added new NMF decoder

* added new NMF decoder

* Changed NMF decoding

* returning non-negative W

* linters

* removed unused files

* moved from custom_models

* updated path

* remove instantiations in PIQ.py

* added docstring for a function

* as in clean branch

* Add docstrings for lobes

* Add docstring for NMFDecoderAudio

* pre-commit

* Fixed yamllint

* Restored input fidelity metric

* cosmetic

* Update README.md

* Update README.md

* Update README.md

* rename yaml

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Fixed README.md in classification

* removed double softmax

* Update README.md

* cosmetic

* Added docstrings for VQEmbedding

* Updated interpret README.md

* cosmetic

* cosmetic

* removed useless comments

* removed file

* cosmetic

* removed comments

* removed hard-coded time dimension

* cosmetic

* cosmetic

* cosmetic

---------

Co-authored-by: fpaissan <me@francescopaissan.it>
Co-authored-by: Francesco Paissan <46992226+fpaissan@users.noreply.github.com>
fpaissan added a commit to fpaissan/speechbrain that referenced this issue May 2, 2024
@fpaissan @ycemsubakan
Adding LMAC - needs refactor (speechbrain#5)
fb91eba 
* WHAM-ing the data

* AO on conv2d classifier

* added interpretability metrics

* fix debug steps -- updated

* minor to train_piq

* fix saving interpretations

* add wham! for L2I

* fix l2i eval

* add NCC

* cross correlation w/ batching

* checked crosscor

* finish finetuning script

* switch to l1

* linters

* add binarized oracle w/ BCE

* fix compute loss in finetuning while saving samples

* comparison script

* fix 0dB mixtures

* add original wav to comparison

* just path to new classifier

* just committing new checkpoint for L2I

* add NMF image logging for debug

* fix bug in viz L2I

* log the number of finetuning masks

* lower crosscor thr

* fix acc

* align L2I debugging w/ PIQ script

* fixed accuracy computation for L2I

* L2I with variable number of components (K=200)

* debugging l2i...

* update hparams

* fixed oracle source

* fixed wrong sources and running finetuning experiments..

* add AST as classifier

* hparams ast -- still not converging

* add ast augmentation

* update training script after merge

* with augmentations is better

* just pushing hparams

* classification with CE

* conv2d fix for CE

* playing with AST augmentation

* fixed thresholding

* starting to experiment with no wham noise stuff

* add wham noise option in classifier training, dot prod correlation in finetuning

* single mask training

* added zero grad

* added the entropy loss

* implemented a psi function for cnn14

* Update README.md

* added stft-mel transformation learning

* add latest eval setup - working on gradient-based

* removed unused brain -- was causing issues in weights loading..

* training l2i on this classifier

* add l2i eval -- removing mosaic; not well defined in the case of L2I

* removed old png file

* debugging eval weight loading..

* was always using vq

* fixed eval AO

* fixed eval -- now everything's fine also for L2I

* better numerical stability

* handling quantus assertionerror

* add saliency from captum

* updated smoothgrad for captum

* added norm to saliency

* IG from captum

* starting gradient-base eval on cnn14...

* commit before merge

* works on cnn14 -- but have a bad checkpoint

* fixed l2i as well

* fixed acc in l2i

* fix not listenable

* updated logging for eval

* a bit less verbose

* printing at sample level

* fix logging - was missing avg

* was messing up in the forward

* now running train_piq.py

* minor corrections

* fix l2i training with wham!

* fixed l2i computation

* linters

* add check for wham usage in eval

* add sample saving during eval

* bug fixes

* added predictions info to the logging

* fixed id for overlap test

* cutting sample before saving

* fixed l2i sampling rate

* fixed random seed so eval will match

* running on full set

* faithfulness fix

* remove pdb

* fix smoothgrad and IG

* fix nmf for pre-training

* removed nmf reconstructions

* truncated gaussian fix for smoothgrad

* fix nans in sensitivity

* better l2i psi network

* saving to a different folder. helps not overriding experiments..

* fix l2i

* fix csv logging of exps

* add guided backprop

* added gradcam. guided backprop and guided gradcam need debugging

* l2i encoder 1D

* mel only - ao

* eval for mel only

* changed logging to simple write

* hardcoded checkpoint - to run on cc

* save everything in one folder

* remove joblib import

* fixed eval?

* fix eval again..

* maybe now?

* trying on cc

* add eval_outdir

* runs full eval

* l2i with updated psi

* update gitignore

* l2i logging different loss values

* add us8k classifier

* us8k interpretations

* fixed guided backprop and guided gradcam

* add shap

* normalizing shap attributions

* adding us8k prepare in interp..

* eval on ID

* fixed backward compatibility

* added multiclass classification

* eval xplorer v1

* eval xplorer v2

* implemented multi label interpretation

* update the loss function in multilabel interpretations

* evaluation explorer - minor fixes

* add roar

* roar test

* just removing a print...

* add roar script

* adding the user study parsing script

* savefigs

* fix to roar hparam

* minor

* extract samples for user study

* fix bug roar

* fixed roar

* fix another copy-paste error

* MRT eval

* roar with random baseline

* fix np seed

* computes mrt metrics

* saving masks for mrt viz

* remove rand baseline roar

* abs

* gradcam eval

* fix class

* add mrt to l2i

* train piq us8k

* param in mrt_evaluator

* add viz

* adding the latest

* fixing path problems for multilabelstuff

* changed the loss function to output 10 masks

* more standard maskout term

* changed encoder loading to local

* added accuracy computation

* removed unnecessary evaluation methods

* added all ones mask and average energy computation

* fixed the bug for whitenoise

* pushing eval later

* l2i new ood

* removing useless files

* cleaning up classification as well

* removing useless hparams in interpret

* more useless files

* old linters

* fix paths

* fix paths

* update Cnn14

* restored old piq file

* wham on PIQ

---------

Co-authored-by: Cem Subakan <csubakan@gmail.com>
Co-authored-by: Francesco Paissan <fpaissan@cedar1.cedar.computecanada.ca>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aheba aheba

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nauman-daw @aheba @mravanelli