New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add /eval/<id>/builds endpoint #529
Conversation
This endpoint allows efficient retrieval of all the builds in an evaluation, without making a request for each single build.
a82e253
to
3ab5d29
Compare
Any feedback on this one? I am happy to pursue a different approach if this change is not wanted / should be implemented in a different way. |
I like this! Can this produce JSON like api in #440? If not, do you know how difficult it would be to support that? Hardly a barrier for entry but such an endpoint seems especially compelling when talking about thousands of builds -- many more than a human would ever issue requests for individually. |
@dtzWill yes the format should be the same as that API. It should return exactly the same as querying the builds of the evaluation and then getting the build details for each build with the JSON api. This new endpoint is only necessary to avoid making 40k requests to get the info for a single evaluation. |
If this would result in too much load on the hydra server, we could also provide a cached version of this is part of a channel (just like store-paths) |
cc @edolstra |
@bennofs mhm, if we cannot get this in, maybe a travis cron job in nix-index would do. Uploading built index files to the github release tab is not too hard: https://github.com/Mic92/cntr/blob/master/.travis.yml#L39 |
The way this works right now is not wonderful for hydra.nixos.org, taking almost 9 minutes and producing 80M of data:
I'm not sure what to do about that, but just for what its worth. |
Also the data includes magic numbers, like build status numbers: https://github.com/NixOS/hydra/blob/master/src/sql/hydra.sql#L202-L215 |
What information is actually needed from builds for
|
The information is very similar to other queries that are fast. Running this with debug produces an extremely long and convoluted output. I suspect this can be orders of magnitude faster with a purpose built query. |
This endpoint allows efficient retrieval of all the builds in an
evaluation, without making a request for each single build.