Skip to content

Provide build step information #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 15, 2022
Merged

Provide build step information #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3b86459
Add the build steps to the build/<id>/api/get-info endpoint.
iwanders Apr 19, 2022
b2c79e0
Add the <build>/api/get-info endpoint to the api spec.
iwanders Apr 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
213 changes: 197 additions & 16 deletions hydra-api.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -507,6 +507,33 @@ paths:
schema:
$ref: '#/components/schemas/Error'

/build/{build-id}/api/get-info:
get:
summary: Retrieves info about this build id, providing information about the steps in it.
parameters:
- name: build-id
in: path
description: build identifier
required: true
schema:
type: integer
responses:
'200':
description: build
content:
application/json:
schema:
$ref: '#/components/schemas/BuildInfo'
examples:
build-success:
$ref: '#/components/examples/build-get-info-success'
'404':
description: build couldn't be found
content:
application/json:
schema:
$ref: '#/components/schemas/Error'

/build/{build-id}/constituents:
get:
summary: Retrieves a build's constituent jobs
Expand Down Expand Up @@ -673,7 +700,7 @@ components:
nullable: true
description: the path to the file to evaluate
type: string
errormsg:
errormsg: &schema_errormsg
nullable: true
description: contains the stderr output of the nix-instantiate command
type: string
Expand Down Expand Up @@ -813,7 +840,7 @@ components:
checkinterval:
description: interval in seconds at which to check the jobset inputs
type: integer
haserrormsg:
haserrormsg: &schema_haserrormsg
description: true if the evaluation had errors
type: boolean
nrscheduled:
Expand Down Expand Up @@ -896,66 +923,68 @@ components:
description: The nix store path
type: string

Build:
Build:
type: object
properties:
id:
type: integer
starttime:
starttime: &schema_starttime
description: time when build started
type: integer
stoptime:
stoptime: &schema_stoptime
description: time when build ended
type: integer
timestamp:
timestamp: &schema_timestamp
description: time when the build was first created
type: integer
jobsetevals:
description: list of evaluations this build is part of
type: array
items:
type: integer
finished:
finished: &schema_finished
description: true when the build is finished
type: boolean
nixname:
nixname: &schema_nixname
description: name from the build's derivation
type: string
buildstatus:
buildstatus: &schema_status
nullable: true # should only be null if finished is false
description: |
Indicates the build status:</br>
<ul>
<li>0 : succeeded</li>
<li>1 : failed</li>
<li>2 : dependency failed</li>
<li>2 : dependency failed (builds only)</li>
<li>3 : aborted</li>
<li>4 : canceled by the user</li>
<li>6 : failed with output</li>
<li>6 : failed with output (builds only)</li>
<li>7 : timed out</li>
<li>8 : cached failure (steps only)</li>
<li>9 : aborted</li>
<li>10 : log size limit exceeded</li>
<li>11 : output size limit exceeded</li>
<li>12 : not deterministic</li>
<li>* : failed</li>
</ul>
<strong>Note:</strong>buildstatus should only be `null` if `finished` is false.
type: integer
jobset:
jobset: &schema_jobset
description: jobset this build belongs to
type: string
priority:
description: determines the priority with which this build will be executed (higher value means higher priority)
type: integer
job:
job: &schema_job
description: nix attribute from the nixexprpath
type: string
drvpath:
drvpath: &schema_drvpath
description: filename of the drv
type: string
system:
system: &schema_system
description: system this build was done for
type: string
project:
project: &schema_project
description: project this build belongs to
type: string
buildproducts:
Expand Down Expand Up @@ -986,6 +1015,99 @@ components:
unit:
type: string
description: unit of the measured build metric
BuildInfo:
properties:
id:
description: The build id.
type: integer
buildId:
description: Same as id, exists for backwards compatibility.
type: integer
finished:
<<: *schema_finished
steps:
description: List of steps that make up this build.
type: array
items:
type: object
additionalProperties:
$ref: '#/components/schemas/BuildStep'
job:
<<: *schema_job
system:
<<: *schema_system
buildstatus:
<<: *schema_status
jobset:
<<: *schema_jobset
project:
<<: *schema_project
timestamp:
<<: *schema_timestamp
nixname:
<<: *schema_nixname
drvPath:
description: Same as drvpath, exists for backwards compatibility.
type: string
outPath:
description: The nix store path for the output of this build.
type: string

BuildStep:
type: object
properties:
stepnr:
description: The step number, this number can be used when retrieving logs for a particular step.
type: integer
starttime:
<<: *schema_starttime
stoptime:
<<: *schema_stoptime
status:
<<: *schema_status
system:
<<: *schema_system
propagatedfrom:
description: Null if the result was not propagated.
nullable: true
type: object
additionalProperties:
$ref: '#/components/schemas/Build'
errormsg:
<<: *schema_errormsg
haserrormsg:
<<: *schema_haserrormsg
machine:
description: Machine on which this step was executed.
type: string
busy:
description: |
Indicates the current stage this build step is in:</br>
<ul>
<li>0 : not being worked on (status populated)</li>
<li>1 : preparing</li>
<li>10 : connecting</li>
<li>20 : sending inputs</li>
<li>30 : building</li>
<li>40 : receiving outputs</li>
<li>50 : post processing</li>
<li>* : unknown</li>
</ul>
type: integer
drvpath:
<<: *schema_drvpath
build:
description: Id of the build this step is a part of.
type: integer
timesbuilt:
nullable: true
type: integer
type:
nullable: true
overhead:
nullable: true
isnondeterministic:
nullable: true

examples:
projects-success:
Expand Down Expand Up @@ -1110,3 +1232,62 @@ components:
project: example-hello
starttime: 1588365711
timestamp: 1588365711

build-get-info-success:
value:
id: 5
buildstatus: 2 # actually a failed build, to show the propagated error.
nixname: hello-2.10
finished: 1
timestamp: 1588365711
system: x86_64-linux
job: hello
jobset: hello
project: example-hello
steps:
-
timesbuilt: null
errormsg: null
haserrormsg: false
overhead: null
type: 0
drvpath: /nix/store/ab9zv2y5gm8hr6g318p0s6kaclwj4slr-hello-2.10.drv
build: 1
machine: localhost
busy: 1
status: 4
isnondeterministic: null
starttime: 1649950241
propagatedfrom: null
system: x86_64-linux
stepnr: 1
stoptime: 1649950294
-
timesbuilt: null
errormsg: null
haserrormsg: false
overhead: null
type: 0
drvpath: /nix/store/hhl1dql437h1glbqv43629f1pmnlcfoo-package.drv
build: 1
machine: localhost
busy: 0
status: 8 # Cached failure, will have propagated result.
isnondeterministic: null
starttime: 1649950241
propagatedfrom:
jobset: hello
nixname: hello
system: x86_64-linux
job: sdk.x86_64-linux
project: hello
id: 3 # since this is a build, this denotes the build id the propagation result came from.
timestamp: 1649456722
finished: 1

system: x86_64-linux
stepnr: 2
stoptime: 1649950294
buildId: 5
drvPath: /nix/store/ab9zv2y5gm8hr6g318p0s6kaclwj4slr-hello-2.10.drv
outPath: /nix/store/y26qxcq1gg2hrqpxdc58b2fghv2bhxjg-hello-2.10
4 changes: 3 additions & 1 deletion src/lib/Hydra/Controller/API.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ sub buildToHash {
system => $build->system,
nixname => $build->nixname,
finished => $build->finished,
timestamp => $build->timestamp
timestamp => $build->timestamp,
buildstatus => undef,
priority => undef
};

if($build->finished) {
Expand Down
40 changes: 40 additions & 0 deletions src/lib/Hydra/Controller/Build.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use warnings;
use base 'Hydra::Base::Controller::NixChannel';
use Hydra::Helper::Nix;
use Hydra::Helper::CatalystUtils;
use Hydra::Controller::API;
use File::Basename;
use File::stat;
use Data::Dump qw(dump);
Expand Down Expand Up @@ -569,13 +570,52 @@ sub bump : Chained('buildChain') PathPart('bump') {
}


sub buildStepToHash {
my ($buildstep) = @_;
return {
build => $buildstep->get_column('build'),
busy => $buildstep->busy,
drvpath => $buildstep->drvpath,
errormsg => $buildstep->errormsg,
isnondeterministic => $buildstep->isnondeterministic,
machine => $buildstep->machine,
overhead => $buildstep->overhead,
# The propagated from field will hold a Build type if it was propagated, we'd like to display that info, so we
# convert the that record to a hash here and inline it, we already have the data on hand and it saves clients a
# request to obtain the actual reason why something happened.
propagatedfrom => defined($buildstep->propagatedfrom) ? Hydra::Controller::API::buildToHash($buildstep->propagatedfrom) : undef,
starttime => $buildstep->starttime,
status => $buildstep->status,
stepnr => $buildstep->stepnr,
stoptime => $buildstep->stoptime,
system => $buildstep->system,
timesbuilt => $buildstep->timesbuilt,
type => $buildstep->type,
haserrormsg => defined($buildstep->errormsg) && $buildstep->errormsg ne "" ? JSON::MaybeXS::true : JSON::MaybeXS::false
};
}


sub get_info : Chained('buildChain') PathPart('api/get-info') Args(0) {
my ($self, $c) = @_;
my $build = $c->stash->{build};

# Since this is the detailed info of the build, lets start with populating it with
# the info we can obtain form the build.
$c->stash->{json} = Hydra::Controller::API::buildToHash($build);

# Provide the original get-info endpoint attributes.
$c->stash->{json}->{buildId} = $build->id;
$c->stash->{json}->{drvPath} = $build->drvpath;
my $out = getMainOutput($build);
$c->stash->{json}->{outPath} = $out->path if defined $out;

# Finally, provide information about all the buildsteps that made up this build.
my @buildsteps = $build->buildsteps->search({}, {order_by => "stepnr asc"});
my @buildsteplist;
push @buildsteplist, buildStepToHash($_) foreach @buildsteps;
$c->stash->{json}->{steps} = \@buildsteplist;

$c->forward('View::JSON');
}

Expand Down