You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi dragen:
I have a question about the autograd mechanism of pytorch. It seems to me that pytorch doesn't support high-order gradient since it builds the graph during forward pass, while computes the gradients by reverse passing the graph without expanding it, thus the gradient variable actually has no link to the origin variable so we cant further compute its gradient w.r.t. the origin var.. However in the above part of the code, where the meta update is conducted, seems no special mechanism is used so I think the update doesn't consider high-order gradient formed by K times inner loop.
To further demonstrate the problem, I write a simple demo.
The text was updated successfully, but these errors were encountered:
My bad. After further reading the original paper and I find the first-order approximation mentioned in Sec 5.2. Experiment results show that the approximation is good enough as well as easy to implement. :)
Thank u for the great code.
MAML-Pytorch/maml.py
Line 399 in 75a7d75
Hi dragen:
I have a question about the autograd mechanism of pytorch. It seems to me that pytorch doesn't support high-order gradient since it builds the graph during forward pass, while computes the gradients by reverse passing the graph without expanding it, thus the gradient variable actually has no link to the origin variable so we cant further compute its gradient w.r.t. the origin var.. However in the above part of the code, where the meta update is conducted, seems no special mechanism is used so I think the update doesn't consider high-order gradient formed by K times inner loop.
To further demonstrate the problem, I write a simple demo.
The text was updated successfully, but these errors were encountered: