Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about configuration setup #17

Closed
ztqwerty opened this issue Apr 11, 2018 · 2 comments

Comments

@ztqwerty
Copy link

@ztqwerty ztqwerty commented Apr 11, 2018

Hi Bily大神,

I have a question about your configuration setup. In MODEL_CONFIG, there is an item called adjust_response_config, and you added a comment in siamese_model.py:

Adjust score, this is required to make training possible

Can you please provide some more explanation on the principle / reason behind that? Whether and when is it necessary to adjust the train_bias and scale parameters?

@bilylee

This comment has been minimized.

Copy link
Owner

@bilylee bilylee commented Apr 11, 2018

Hi,

Score adjustment is simply normalizing response to the range (e.g. [-10, 10]) where sigmoid is not saturated. Without it, the scale of the response will be on the order of 1e3 and the gradients vanish.

For most of the time, set the scale to 1e-3 and the train_bias to True should be fine.

@ztqwerty

This comment has been minimized.

Copy link
Author

@ztqwerty ztqwerty commented Apr 12, 2018

Thanks for the quick response. Appreciate it!

@ztqwerty ztqwerty closed this Apr 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.