Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add huberts #57

Merged
merged 1 commit into from
Aug 29, 2023
Merged

add huberts #57

merged 1 commit into from
Aug 29, 2023

Conversation

bagustris
Copy link
Collaborator

@bagustris bagustris commented Aug 29, 2023

This adds five variants of Hubert: hubert, hubert_ft, hubert_large, hubert_xlarge, and hubert_xlarge_ft (Reference). Here are the results of the first four variants on RAVDESS dataset. This PR is to tackle #56.

# Hubert (base)
UG modelrunner: run: 0 epoch: 0: result: test: 0.742 UAR
DEBUG modelrunner: plotting confusion matrix to train_test_dev_svm_hubert__0_000_cnf
DEBUG runmanager: value for measure not found, using default: uar
DEBUG reporter: labels: ['angry' 'happy' 'neutral' 'sad']
DEBUG reporter: result per class (F1 score): [0.909, 0.8, 0.4, 0.849]
WARNING experiment: Save experiment: Can't pickle local object: Serialization of parametrized modules is only supported through state_dict(). See:
https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-a-general-checkpoint-for-inference-and-or-resuming-training
DEBUG experiment: Done, used 694.146 seconds
DONE
---
# Hubert_ft (finetuning)

DEBUG modelrunner: run: 0 epoch: 0: result: test: 0.891 UAR
DEBUG modelrunner: plotting confusion matrix to train_test_dev_svm_hubert__0_000_cnf
DEBUG runmanager: value for measure not found, using default: uar
DEBUG reporter: labels: ['angry' 'happy' 'neutral' 'sad']
DEBUG reporter: result per class (F1 score): [0.935, 0.814, 0.842, 0.892]
WARNING experiment: Save experiment: Can't pickle local object: Serialization of parametrized modules is only supported through state_dict(). See:
https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-a-general-checkpoint-for-inference-and-or-resuming-training
DEBUG experiment: Done, used 1998.011 seconds
DONE
-----
# Hubert_large

DEBUG modelrunner: run: 0 epoch: 0: result: test: 0.914 UAR
DEBUG modelrunner: plotting confusion matrix to train_test_dev_svm_hubert_large_ll60k__0_000_cnf
DEBUG runmanager: value for measure not found, using default: uar
DEBUG reporter: labels: ['angry' 'happy' 'neutral' 'sad']
DEBUG reporter: result per class (F1 score): [0.984, 0.881, 0.882, 0.882]
WARNING experiment: Save experiment: Can't pickle local object: Serialization of parametrized modules is only supported through state_dict(). See:
https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-a-general-checkpoint-for-inference-and-or-resuming-training
DEBUG experiment: Done, used 1867.607 seconds
DONE
---
# Hubert_xlarge
DEBUG modelrunner: run: 0 epoch: 0: result: test: 0.852 UAR
DEBUG modelrunner: plotting confusion matrix to train_test_dev_svm_hubert_xlarge__0_000_cnf
DEBUG runmanager: value for measure not found, using default: uar
DEBUG reporter: labels: ['angry' 'happy' 'neutral' 'sad']
DEBUG reporter: result per class (F1 score): [0.899, 0.821, 0.788, 0.879]
WARNING experiment: Save experiment: Can't pickle local object: Serialization of parametrized modules is only supported through state_dict(). See:
https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-a-general-checkpoint-for-inference-and-or-resuming-training
DEBUG experiment: Done, used 4410.192 seconds

I also modified the wav2vec feature file to adopt the style introduced in this PR, so it will be able to receive variants of wav2vec2 (not done yet).

Example of INI file for Ravdess (nkululeko.nkululeko)

[EXP]
root = ./
name = results/exp_ravdess_hubert
runs = 1
epochs = 1
save = True
[DATA]
databases = ['train', 'test', 'dev']
train = ./data/ravdess/ravdess_train.csv
train.type = csv
train.absolute_path = False
train.split_strategy = train
dev = ./data/ravdess/ravdess_dev.csv
dev.type = csv
dev.absolute_path = False
dev.split_strategy = train
test = ./data/ravdess/ravdess_test.csv
test.type = csv
test.absolute_path = False
test.split_strategy = test
target = emotion
labels = ['angry', 'happy', 'neutral', 'sad']
[FEATS]
type = ['hubert_xlarge']
scale = standard
[MODEL]
type = svm

@bagustris bagustris mentioned this pull request Aug 29, 2023
@felixbur felixbur merged commit 55b5c24 into felixbur:main Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants