New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pp/cpc ctc stats #5
Conversation
…ignments) for specidied "capture DS"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please double-check the 1:
.
It seems that right now half of training loop can be switched off, shall we have two functions?
Anyway, please merge once you verify the above.
cpc/criterion/soft_align.py
Outdated
captureRes = {} | ||
if 'pred' in captureOptions: | ||
# need to remove 'predict self loop' stuff from captured data | ||
captureRes['pred'] = predictions[:,:,:,1:] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am unsure about this [1:] at the end - is it to skip the self-loop?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I was removing it as it's not really a prediction, but forgot it would spoil alignment indices - fixed, now self loop is also saved in predictions
Yeah, another function seems better - I just didn't want to have entirely separate run file with constructing all stuff from args and went a bit too far in "everything together" direction; updated |
Added option to capture data for future visualizations (save them across different epochs during training, or just see what is produced after the training) - representations, CPC predictions and CPC-CTC alignments
Example run that saves data each 2 epochs for 33% of provided 'capture dataset':
python train.py --pathDB /pio/scratch/1/i283340/MGR/zs/ds2
--pathTrain /pio/scratch/1/i283340/MGR/zs/sometries/ds2part.txt
--pathVal /pio/scratch/1/i283340/MGR/zs/sometries/ds2part.txt
--pathCaptureDS /pio/scratch/1/i283340/MGR/zs/sometries/ds2part.txt
--captureDSfreq 33 --captureEachEpochs 2
--pathCaptureSave /pio/scratch/1/i283340/MGR/zs/capture/try2
--saveRepr --savePred --saveAlign
--pathCheckpoint /pio/scratch/1/i283340/MGR/zs/checkpoints/cpcctc_tests2
--file_extension .flac --n_process_loader 2 --max_size_loaded 40000000
--batchSizeGPU 16 --nPredicts 8 --CPCCTC --CPCCTCNumMatched 12
--CPCCTCSelfLoop --CPCCTCSkipBeg 1 --CPCCTCSkipEnd 2