Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

fix dialogpt dual usage of END_IDX #3256

Merged
merged 8 commits into from Dec 16, 2020
Merged

fix dialogpt dual usage of END_IDX #3256

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
6ade0eb
fix dialogpt dual usage of endoftext
jxmsML Nov 10, 2020
0946ef0
add null_idx = -1
jxmsML Nov 11, 2020
0a37ef0
dialog bs test
jxmsML Nov 17, 2020
7ee25bf
Set null_idx in model and decoder, add to dialogpt test
jxmsML Dec 8, 2020
3665e94
small formats
jxmsML Dec 8, 2020
38796fd
accidental delete old test
jxmsML Dec 8, 2020
590ceb7
reviewer comment
jxmsML Dec 15, 2020
1fec4f8
Merge branch 'master' into dialogptdebug
jxmsML Dec 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions parlai/agents/hugging_face/dialogpt.py
View file
Open in desktop
Expand Up @@ -26,6 +26,12 @@ class DialoGPTDecoder(GPT2Decoder):
This decoder is initialized with the pretrained model from Hugging Face.
"""

def __init__(self, opt, dict):
super().__init__(opt, dict)
if opt.get('batchsize', 1) == 1 and self.END_IDX == self.NULL_IDX:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when is the latter condition not going to be true? if you are inheriting from this model but changing things?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when -bs 1 --add_special_token True ? basically I only want to override the NULL_IDX if it's the same as END_IDX.

# get around the dual usage of end_idx that would otherwise mask endtoken during forward pass.
self.NULL_IDX = -1

def _init_from_pretrained(self, opt):
# load model
model_sz = opt['gpt2_size']
Expand Down
1 change: 0 additions & 1 deletion parlai/agents/hugging_face/dict.py
View file
Open in desktop
Expand Up @@ -20,7 +20,6 @@
)

SPECIAL_TOKENS = {"bos_token": "<bos>", "eos_token": "<eos>", "pad_token": "<pad>"}

NO_OP = "x"


Expand Down
1 change: 1 addition & 0 deletions parlai/agents/hugging_face/gpt2.py
View file
Open in desktop
Expand Up @@ -108,6 +108,7 @@ def forward(self, input, encoder_state, incr_state=None):
model_input = input[:, -1:]
attention_mask = torch.cat([encoder_state, input], dim=-1) != self.NULL_IDX

model_input = model_input.clamp_(min=0)
transformer_outputs = self.transformer(
model_input,
past=incr_state,
Expand Down