-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Benchmarks for Dircon performance #115
Comments
There are some benchmark tests done in #169. We can see that a single change to the traj opt problem doesn't necessarily improve the solve time. |
I compared ipopt with snopt using the walking example (https://github.com/DAIRLab/dairlib/blob/5f5205cbfc9466531bcefc10ad524222fd0998fb/examples/Cassie/run_dircon_walking.cc) in my local branch. Ipopt setting:
Snopt setting:
where Solve time and the optimal cost:
* Variable scaling is not implemented for ipopt yet. The benchmark should be exactly the same as the one without variable scaling. Ipopt spent more than 90% of the time evaluating the nonlinear constraints |
Very useful results, and discouraging for ipopt. Did ipopt really need all that time to converge to good solution? |
From the log file, it looks like ipopt could terminate a bit earlier if we changed the setting?
I think ipopt's performance is not so bad in this benchmark test. Without any scaling, snopt probably won't find a solution if we give it more time to solve. With only constraint scaling, snopt was stuck in a local bad solution. If we implement variable scaling for ipopt in drake, it's possible that ipopt can perform decently against snopt, maybe |
While we are modifying Dircon, we benchmark the performance here for future reference.
CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
Program: run_dircon_squatting.cc
Commit: ea86f21
Snopt solve time: 125.76 seconds
Constraint scaling
Commit: 1645804
Snopt solve time: 56.16 seconds
Cost: 8370.99
Constraint scaling + variable scaling
CPU: i7-8750H
Commit: af0fc3b
Snopt solve time: 93.7 seconds
Cost: 8233.8
Constraint scaling + variable scaling + sparsity pattern
CPU: i7-8750H
Commit: dc79d8e
Snopt solve time: 138.3 seconds
Cost: 8233.43
2fb9d4d vs 8e44186
Run command:
./bazel-bin/examples/Cassie/run_dircon_walking --store_data=true --tol=1e-5
The solver will find a local minimum, and then solve the problem again with a new initial guess:
./bazel-bin/examples/Cassie/run_dircon_walking --store_data=true --tol=1e-5 --init_file="z.csv"
It solves much slower in 2fb9d4d than in 8e44186. In snopt log, we can see that although snopt is taking unit step size, the cost doesn't get improved much after a step. Also, if we look at the gradient of the dynamic constraint, we can notice that the columns related to the slack variables have values about 1000 times smaller than other columns.
The text was updated successfully, but these errors were encountered: