You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is especially salient due to the fact that this is so deep in the call stack (TransformerDecoderLayer <- TransformerDecoder <- Transformer <- GlobalTrackingTransformer <- GTRRunner <- ...).
This induces so many code paths it's nearly impossible to trace:
Both here and in the upstream Transformer.call, let's fix the variable names, disambiguate "target"/tgt, memory, and canonical key/query/value nomenclature.
The text was updated successfully, but these errors were encountered:
This function gives me nightmares:
https://github.com/talmolab/biogtr/blob/a47c48b84535632290ab7757d65abbfd787d69bc/biogtr/models/transformer.py#L444-L483
This is especially salient due to the fact that this is so deep in the call stack (
TransformerDecoderLayer
<-TransformerDecoder
<-Transformer
<-GlobalTrackingTransformer
<-GTRRunner
<- ...).This induces so many code paths it's nearly impossible to trace:
https://github.com/talmolab/biogtr/blob/a47c48b84535632290ab7757d65abbfd787d69bc/biogtr/models/transformer.py#L457-L458
https://github.com/talmolab/biogtr/blob/a47c48b84535632290ab7757d65abbfd787d69bc/biogtr/models/transformer.py#L460
Why??? It's just for self-attention, so why not just keep it in that conditional a few lines down??
https://github.com/talmolab/biogtr/blob/a47c48b84535632290ab7757d65abbfd787d69bc/biogtr/models/transformer.py#L462-L465
And surely we can come up with better names than
tgt
,tgt2
,tgt3
??https://github.com/talmolab/biogtr/blob/a47c48b84535632290ab7757d65abbfd787d69bc/biogtr/models/transformer.py#L467-L481
Both here and in the upstream
Transformer.call
, let's fix the variable names, disambiguate "target"/tgt
,memory
, and canonical key/query/value nomenclature.The text was updated successfully, but these errors were encountered: