Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APIv2: benchmarks: inconsistent benchdata source #898

Open
dsnet opened this issue Jul 11, 2019 · 1 comment

Comments

Projects
None yet
2 participants
@dsnet
Copy link
Member

commented Jul 11, 2019

The .cache/benchdata is populated in two ways:

It seems that they download different sets of data files.

However, the benchmarks/bench_test.go source walks over the contents of .cache/benchdata and blindly loads all files there to use as the datasets. This means that our benchmarks are not consistent, and dependent on whether we ran the integration test, the download_benchdata.bash script, or both.

@dsnet

This comment has been minimized.

Copy link
Member Author

commented Jul 11, 2019

The benchdata directory is 451MiB. One consideration is to move it out of .cache and perhaps into benchmarks/.benchdata. Doing so will allow Travis-CI to avoid transferring that data every time the integration test is run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.