CPC + Nullspace loading for downstream tasks

[jdzikowski]

This implements cpc adapter for nullspace and loading of a checkpoint with nullspace. Just add --load path/to/checkpoint/with/nullspace --nullspace to your train script and your good to go. Checkoints of CPC_ll6k with nullspace of various sizes can be found under /pio/scratch/1/i273233/linear_separability/cpc/cpc_official_phonemes_nullspace_X/checkpoint_9.pt

[janchorowski]

I don't understand what's happening here. The model works like that:

• there is an encoder
• it is followed by the autoregressive predictor
• we extract vectors from a middle layer of the autoregressive predictor

but you apply the null-space matrix to the output of the encoder, then feeding the modified values to the autoregressive predictor.

Shouldn't we first extract the features from a layer which works well, then use it to predict the speaker, then multiply it by the nullspace?