-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
benchmarking infrastructure #61
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
// The linear solver is 100 iterations of CGLS. | ||
suite.benchmark( | ||
"NonlinearFactorGraph, Intel, 5 Gauss-Newton steps, 100 CGLS steps", | ||
settings: .iterations(1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Results are going to be quite noisy if you only run a single iteration. Is it not practical to run it for longer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is benchmarking a whole solver, and it's pretty slow right now (~12 seconds).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, thanks for the clarification.
settings: .iterations(1) | ||
) { | ||
var val = intelDataset.initialGuess | ||
for _ in 0..<5 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Usually, it's better to avoid loops in benchmarks. Instead, you can put just the loop body as a benchmark, and then customize number of iterations with .iterations
(or better leave it off to be automatically detected by the framework).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This loop is part of the solver logic that we are benchmarking, so I think it makes sense to have it in the benchmark. Eventually we're going to have an API for the solver and the benchmark body will look like solve(graph, initialGuess)
and this loop will be hidden inside the solve
function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also in the final solver this should be a while error < tol && i < max_iteration
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM and merge at will :)
This adds a benchmark for Pose2SLAM using NonlinearFactorGraph, Gauss-Newton, and CGLS.
I added a section of the readme showing how to run it.
Since this adds an external dependency, I committed a
Package.resolved
that pins us to a specific commit.swift package update
updates this to newer commits.