benchmarking infrastructure #61
Conversation
| // The linear solver is 100 iterations of CGLS. | ||
| suite.benchmark( | ||
| "NonlinearFactorGraph, Intel, 5 Gauss-Newton steps, 100 CGLS steps", | ||
| settings: .iterations(1) |
There was a problem hiding this comment.
Results are going to be quite noisy if you only run a single iteration. Is it not practical to run it for longer?
There was a problem hiding this comment.
This is benchmarking a whole solver, and it's pretty slow right now (~12 seconds).
There was a problem hiding this comment.
I see, thanks for the clarification.
| settings: .iterations(1) | ||
| ) { | ||
| var val = intelDataset.initialGuess | ||
| for _ in 0..<5 { |
There was a problem hiding this comment.
Usually, it's better to avoid loops in benchmarks. Instead, you can put just the loop body as a benchmark, and then customize number of iterations with .iterations (or better leave it off to be automatically detected by the framework).
There was a problem hiding this comment.
This loop is part of the solver logic that we are benchmarking, so I think it makes sense to have it in the benchmark. Eventually we're going to have an API for the solver and the benchmark body will look like solve(graph, initialGuess) and this loop will be hidden inside the solve function.
There was a problem hiding this comment.
Also in the final solver this should be a while error < tol && i < max_iteration
This adds a benchmark for Pose2SLAM using NonlinearFactorGraph, Gauss-Newton, and CGLS.
I added a section of the readme showing how to run it.
Since this adds an external dependency, I committed a
Package.resolvedthat pins us to a specific commit.swift package updateupdates this to newer commits.