Skip to content

Remove Duplicate Declaration of Loss Function#38

Merged
iofu728 merged 1 commit intomicrosoft:mainfrom
Speuce:main
Dec 30, 2023
Merged

Remove Duplicate Declaration of Loss Function#38
iofu728 merged 1 commit intomicrosoft:mainfrom
Speuce:main

Conversation

@Speuce
Copy link
Copy Markdown
Contributor

@Speuce Speuce commented Dec 28, 2023

I noticed there is an unnecessary duplicate declaration of loss_fct here.

Relevant code:

        loss_fct = torch.nn.CrossEntropyLoss(reduction="none")
        shift_logits = response.logits[..., :-1, :].contiguous()
        shift_labels = input_ids[..., past_length + 1 : end].contiguous()
        # Flatten the tokens
        active = (attention_mask[:, past_length:end] == 1)[..., :-1].view(-1)
        active_logits = shift_logits.view(-1, shift_logits.size(-1))[active]
        active_labels = shift_labels.view(-1)[active]
        loss_fct = torch.nn.CrossEntropyLoss(reduction="none")
        loss = loss_fct(active_logits, active_labels)

As you can see, loss_fct is not used before it is declared for a second time, therefore it is safe to remove the first declaration.

@iofu728 iofu728 self-requested a review December 30, 2023 06:37
@iofu728 iofu728 self-assigned this Dec 30, 2023
@iofu728 iofu728 added the style some problem of code style label Dec 30, 2023
@iofu728
Copy link
Copy Markdown
Contributor

iofu728 commented Dec 30, 2023

Hi @Speuce,

Thanks for your help. I'll merge this PR.

@iofu728 iofu728 merged commit ef1813d into microsoft:main Dec 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

style some problem of code style

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants