Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confusion about computing the gradient of predictor? #12

Closed
wenerhg opened this issue Mar 11, 2019 · 2 comments
Closed

Confusion about computing the gradient of predictor? #12

wenerhg opened this issue Mar 11, 2019 · 2 comments

Comments

@wenerhg
Copy link

wenerhg commented Mar 11, 2019

In 'NAO/cnn/encoder/encoder.py', the gradient of predictor is computed as follow:
new_arch_outputs = self.encoder_outputs - self.params['predict_lambda'] * grads_on_outputs

However, I think the gradient should be computed as described in paper:

Is there an error with computing the gradient of predictor in code?

@renqianluo
Copy link
Owner

@wenerhg In the paper, for the general purpose, we denote f as the performance which is the larger the better, so we move towards the positive direction of the gradient, which shows as '+'. In the code, we use error rate as f which is the lower the better, so we need to move towards the negative direction, which shows as '-'.

@renqianluo
Copy link
Owner

@wenerhg in other words, if you use accuracy as f in code, you should move towards the positive direction and use '+'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants