-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Auto-initializing all instances of var as differetiable #2
Comments
Please note that In this code snippet, |
UPDATEPlease see the update in my other comment bellow. The code Dear dcoeurjo, note the line 15 in your code, where you declare the variable var energy(const std::vector<var> &pointsx,
const std::vector<var> &pointsy,
const var &x,
const var &y)
{
var sum=0.0;
//initialize sum as a placeholder
dCpp::initPlaceHolder(sum);
for (auto i=0; i<pointsx.size(); ++i)
sum += (pointsx[i] - x)* (pointsx[i] - x) + (pointsy[i] - y)* (pointsy[i] - y);
return sum;
} The function With the edited code, both the call to Please Note that currently all variables have to be initialized in order to be differentiable, so they can be used either as a constant, or as a differentiable variable. This may change in the next release (it has changed, see the update bellow). I am closing this issue, but if you have any other questions, please ask. SideNoteAlso Note, that your code as it is now, also performs differentiation with respect to the elements of for (auto i=0; i<4; ++i)
{
dCpp::init(px[i]);
dCpp::init(py[i]);
} or make |
Excellent. thanks a lot, that makes sense. (thanks also for the comment on thanks again for you quick reply and your nice code ;) |
Closure of the issueYour original code should now work as expected. Thank you for opening this issue, the specifics of the update can be found bellow. The title of the issue was edited to reflect this update. UPDATEAll instances of BeforePreviously one had to initialize placeholders (see the var sum=0.0;
dCpp::initPlaceHolder(sum); NowAll instances of var sum=0.0; This means all instances of NoteYou cannot differentiate a variable with respect to an var x=0.0;
dCpp::init(x); |
dear @ZigaSajovic, thanks again for your feedbacks. There is still something that puzzles me a bit (I've updated my dCpp clone to the last release). Here you have two codes that should do the same thing: https://gist.github.com/dcoeurjo/c5b9b1cc62b094807ff7ab8aa42f12a4 The problem is the following: we have four (fixed) points (corners of a unit square) and a fifth one. I want to move the fifth one such that it minimizes the sum of squared distances to the four anchor points. In this setting, the problem is very simple (explicit gradient w.r.t. the position of the fifth point). It is just a toy example here. So I use the gradients from dCpp in a gradient descent scheme to end up to the (global) minimum which is exactly
When the anchor points are std::vector ( Would you have any idea ? |
To whom it may concerndCpp is a flexible tool, allowing implementation and analysis of programs through operational calculus.
Please forgive me for the sparse tutorial in the ReadMe on the title page of this repository. The dCpp project is a flexible tool, meant to be used in conjunction with the above two papers. Finally, I extend my gratitude to @dcoeurjo , as this discussion, along with his code and my corrections will serve as a great read for anyone starting out with dCpp, where most of the issues one might encounter in the beginning are resolved. dear @dcoeurjo
x += -lambda*e.d(&x).id;
y += -lambda*e.d(&y).id; NoteThe section bellow (on initializations before/in the loop body) concerns the general case, where the variables x and y might have been differentiated with respect to some other variable in the body of the loop. In the specific case of @dcoeurjo code (after my edits, full code can be found bellow), it makes no difference, as the above expression simply adds a constant to x and y, x += -lambda*e.d(&x).id;
y += -lambda*e.d(&y).id; which does not change their derivatives (it remains equal to 1, as addition of a constant does not alter it) and the function energy also does not perform any differentiation of x and y with respect to other variable (assigning x or y as a result of a differentiable operation). Therefore their only derivatives are still derivatives with respect to themselves, and equal to 1, as before the body of the loop is executed (or right after initialization). I will still include instructions on the general case, for completeness sake. If you want each iteration to be differentiated separately (as would seem to fit your needs, as your goal is a simply gradient descent simulation, where all differentiations are to be done independently), initialize them each iteration //non relevant code for demonstration is omitted
var x(50.1);
var y(50.1);
//non relevant code for demonstration is omitted
while (norm > 0.0000000001)
{
e = energy(px,py,x,y);
//non relevant code for demonstration is omitted
x += -lambda*e.d(&x).id;
y += -lambda*e.d(&y).id;
norm = e.d(&x).id*e.d(&x).id+e.d(&y).id*e.d(&y).id;
dCpp::init(x);
dCpp::init(y);
} If you want the gradient to depend on all iterations, initialize at the beginning only (see //non relevant code for demonstration is omitted
var x(50.1);
var y(50.1);
dCpp::init(x);
dCpp::init(y);
//non relevant code for demonstration is omitted
while (norm > 0.0000000001)
{
e = energy(px,py,x,y);
//non relevant code for demonstration is omitted
x += -lambda*e.d(&x).id;
y += -lambda*e.d(&y).id;
norm = e.d(&x).id*e.d(&x).id+e.d(&y).id*e.d(&y).id;
}
std::vector<var> px = {0.0, 1.0, 1.0, 0.0};
std::vector<var> py = {0.0, 0.0, 1.0, 1.0};
dCpp::initSpace(1); You see, the space is set to order 0 at the time of decleration of px and py. Therefore their order is set to 0. Simply permute the lines dCpp::initSpace(1);
std::vector<var> px = {0.0, 1.0, 1.0, 0.0};
std::vector<var> py = {0.0, 0.0, 1.0, 1.0}; Along with the corrections I mentioned above. For clarity, I am appending both codes in their correct form here, assuming you wish to differentiate each iteration separately (as you are performing gradient descent) barycenter.cppint main()
{
std::vector<double> px = {0.0, 1.0, 1.0, 0.0};
std::vector<double> py = {0.0, 0.0, 1.0, 1.0};
dCpp::initSpace(1);
var x(50.1);
var y(50.1);
dCpp::init(x);
dCpp::init(y);
double lambda=0.1;
var e;
double norm = 1.0;
while (norm > 0.0000000001)
{
e = energy(px,py,x,y);
std::cout<< "x= "<<x.id<<" y= "<<y.id<<std::endl;
std::cout<<"e = "<<e.id<<std::endl;
std::cout<<"de/dx = "<<e.d(&x).id<<std::endl;
std::cout<<"de/dy = "<<e.d(&y).id<<std::endl<<std::endl;
x += -lambda*e.d(&x).id;
y += -lambda*e.d(&y).id;
norm = e.d(&x).id*e.d(&x).id+e.d(&y).id*e.d(&y).id;
// the following two lines have no effect in your particular case
// as you do not perform operations that change the derivative of x and y
dCpp::init(x);
dCpp::init(y);
}
return 0;
} barycenter2.cppint main()
{
dCpp::initSpace(1);
std::vector<var> px = {0.0, 1.0, 1.0, 0.0};
std::vector<var> py = {0.0, 0.0, 1.0, 1.0};
var x(50.1);
var y(50.1);
dCpp::init(x);
dCpp::init(y);
double lambda=0.1;
var e;
double norm = 1.0;
while (norm > 0.0000000001)
{
e = energy(px,py,x,y);
std::cout<< "x= "<<x.id<<" y= "<<y.id<<std::endl;
std::cout<<"e = "<<e.id<<std::endl;
std::cout<<"de/dx = "<<e.d(&x).id<<std::endl;
std::cout<<"de/dy = "<<e.d(&y).id<<std::endl<<std::endl;
x += -lambda*e.d(&x).id;
y += -lambda*e.d(&y).id;
norm = e.d(&x).id*e.d(&x).id+e.d(&y).id*e.d(&y).id;
// the following two lines have no effect in your particular case
// as you do not perform operations that change the derivative of x and y
dCpp::init(x);
dCpp::init(y);
}
return 0;
} Which outputs When would
|
Thanks a lot ! I'm an auto. diff. newbie and I'm not sure to fully get your point on Thanks again for your time and the great tool you share. |
I edited my post above with a Note, explaining why it makes no difference in your code. Thank you for the follow up question, as it made me clarify some points further. Please let me now if the edit clarified the remaining confusion. To answer your question, if you performed any differentiable operations (operations changing their derivative) on variables with respect to which you are differentiating within the body of the loop, and wish for the differentiation to be independent (each iteration separately, independently), than you should re-initialize. In your case (after my edit), the only change to x and y you make, are adding/subtracting a constant from them. This does not change their derivatives, so you do not have to re-initialize them. |
Fantastic ! |
Hi,
I'm playing a bit with your nice AD tool and I'm having a bug with C++ operator and expressions.
Please have a look to https://gist.github.com/dcoeurjo/ce2b7f5e16edd348b7e4ca061ae6ceb5
For short: I have four points in the plane and I compute a kind of energy (sum of squared euclidean distance to a anchor point) and I want to differentiate this energy w.r.t. the anchor point.
If I expand the for loop when computing the energy (line 28), everything looks fine. When I use a for loop (line 17), then the differentiation fails (returning
0.0
for de/dx for instance). In terms of C++ operators, both energy expressions look similar.Would you have any idea ?
The text was updated successfully, but these errors were encountered: