Benchmarking against previous package version should be allowed ideally by downloading/checking out the specific version and comparing both runs. This requires elastic/package-spec#446 so we can checkout the commit since packages in the registry do not have the required testing assets.