Smoothl1 loss and Python API. #1870
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Noplz
reviewed
Apr 24, 2017
paddle/gserver/layers/CostLayer.h
Outdated
| @@ -91,8 +91,8 @@ class MultiClassCrossEntropy : public CostLayer { | |||
| * | |||
| * [1] Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, | |||
| * Richard Schwartz, and John Makhoul. Fast and robust neural | |||
| * network joint models for statistical machine translation. | |||
| * In Proceedings of the ACL 2014 Conference. | |||
| * network joint models for statistical machine translation. * In | |||
Noplz
reviewed
Apr 24, 2017
| if (std::fabs(cost[j]) >= 1) cost[j] = (0 < cost[j]) - (cost[j] < 0); | ||
| real val = out[j] - lbl[j]; | ||
| if (std::fabs(val) < 1) { | ||
| grad[j] += val; |
There was a problem hiding this comment.
这里是不是应该是grad[i] += val; 上面循环中去掉grad += dim grad的维度应该与numSamples的数量一致吧。之前在循环里加了dim，因为check了dim必须是1，如果dim不是1就错了。
There was a problem hiding this comment.
grad的大小为：numSamples * dim，代表第一个input对应的gradient.
There was a problem hiding this comment.
可以看https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/gserver/layers/CostLayer.cpp 这个文件 60行->68行-> 227行 -> 244行。
grad和out的维度一致。
Noplz
approved these changes
Apr 24, 2017
| @@ -1685,7 +1685,7 @@ TEST(Layer, smooth_l1) { | |||
| config.layerConfig.add_inputs(); | |||
|
|
|||
| for (auto useGpu : {false, true}) { | |||
| testLayerGrad(config, "smooth_l1", 100, false, useGpu, false, 2.0); | |||
| testLayerGrad(config, "smooth_l1", 100, false, useGpu, false); | |||
| } | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
Can you extend the loss test to have more than 1 dim output ?
config.inputDefs.push_back({INPUT_DATA, "layer_0", 1, 0});
config.inputDefs.push_back({INPUT_DATA_TARGET, "layer_1", 1, 0});
Change the size of input and label to be 10 in order to make sure it works with more than 1 dim
luotao1
requested changes
Apr 25, 2017
| @@ -498,6 +498,12 @@ hsigmoid | |||
| :members: hsigmoid | |||
| :noindex: | |||
|
|
|||
| smooth_l1 | |||
There was a problem hiding this comment.
请加到doc/api/v2/config/layer.rst里面，v1里面的不用改了。
luotao1
approved these changes
Apr 25, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #1859
@pengwangucla @gaoyuan please help to review. Thanks!