Difference between 'cargo bench' and 'cargo criterion'

[nbigaouette]

I'd like to use cargo-criterion instead of cargo bench, but it seems that running cargo criterion does not add in the report a comparison with the previous run in case of regression (and maybe improvement?)

Is there something extra that I missed that needs to be done to get this comparison in the report? Or is it not supported?

Thanks!

[jtran]

I'm running into this problem also. I had always thought cargo-criterion was a drop-in replacement and a superset in terms of functionality. After some digging, I believe the data is saved, but it's in a different path and uses serde_cbor. Look for .cbor files in target/criterion/data/. It's a binary file format instead of JSON. I haven't yet extracted the data in these files to verify it.

[xpe]

@bheisler and others : Did I overlook a overview or summary of how cargo criterion is different than cargo bench? (I'd expect this is somewhat of a moving target.)

Here is the relevant section in the README at present:

Features

• Charts: Uses gnuplot or plotters to generate detailed graphs of benchmark results
• Reports: In addition to the reports generated by Criterion.rs, cargo-criterion generates a historical report showing the performance of a function over time.
• Configurable: cargo-criterion's plot generation can be configured using a criterion.toml file.

This doesn't directly answer the question of comparing against cargo bench. Seems like some low hanging fruit to help explain to people.