Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
Patrick-Star125 committed Dec 5, 2023
1 parent 6e29bdd commit ec7a0af
Showing 1 changed file with 1 addition and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,7 @@ Paddle需要扩充API,新增 AdaptiveLogSoftmaxWithLoss API,

adaptive_log_softmax_with_loss的计算分步骤如下

$$\text{head_output} = \text{linear}(\text{input}, \text{head_weight}, \text{head_bias})$$

$$\text{head_logprob} = \text{log_softmax}(\text{head_output}, \text{axis}=1)$$

$$\text{output} += \text{take_along_axis}(\text{head_logprob}, \text{gather_inds.unsqueeze(1)}, \text{axis}=1).\text{squeeze()}$$

$$\text{loss} = -\text{output.mean()}$$
![image](https://github.com/PaddlePaddle/community/assets/69072522/3d43f3e9-deb0-4d52-96be-2cd85a104b90)

## 3、意义

Expand Down

0 comments on commit ec7a0af

Please sign in to comment.