About start index

[art-jang]

Hi, thank you for your work.

I'm not sure whether the start index is False or True for KD loss calculation.

Could you let me know about it?

[ycmin95]

@JYJ-YeongJoon-Jang
We adopt the KD loss to distill the logits learnt by the BiLSTM layer to the convolution layer. Due to the spiky activations of CTC, we don't know whether blank labelled logits contain useful information, so we try distill with blank logits and without blank logits. Experimental results show that without blank logits can achieve better performance and we adopt this setting in this work.
There are some assumptions about this result: blank logits contain less information, or the unbanlanced ratio of blank and non-blank logits.

[art-jang]

Thank you for kind and fast reply :) I'll cite this paper ! best regards yeongjun jang

…

-----Original Message----- From: "Yuecong Min" ***@***.***> To: "ycmin95/VAC_CSLR" ***@***.***>; Cc: "JYJ-YeongJoon-Jang" ***@***.***>; "Mention" ***@***.***>; Sent: 2021-11-02 (화) 11:09:09 (UTC+09:00) Subject: Re: [ycmin95/VAC_CSLR] About start index (Issue #5) @JYJ-YeongJoon-Jang https://github.com/JYJ-YeongJoon-JangWe adopt the KD loss to distill the logits learnt by the BiLSTM layer to the convolution layer. Due to the spiky activations of CTC, we don't know whether blank labelled logits contain useful information, so we try distill with blank logits and without blank logits. Experimental results show that without blank logits can achieve better performance and we adopt this setting in this work https://github.com/ycmin95/VAC_CSLR/blob/main/slr_network.py#L104.There are some assumptions about this result: blank logits contain less information, or the unbanlanced ratio of blank and non-blank logits. —You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub #5 (comment), or unsubscribe https://github.com/notifications/unsubscribe-auth/AO3EB4VAPIW3EMO25HDUFFTUJ5B4FANCNFSM5HEXQR6A.Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign=notification-email&utm_medium=email&utm_source=github.