New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build.snapcraft.io should honour target-architectures when that lands in snapcraft #556

Closed
popey opened this Issue Mar 28, 2017 · 19 comments

Comments

Projects
None yet
10 participants
@popey

popey commented Mar 28, 2017

My yaml has:-
architectures: [amd64]
Because I know it fails to build on armhf, and I'm only testing on amd64, so pointless wasting cpu cycles on 40 min armhf builds.
I realise build is explicitly limited to amd64 and armhf (and currently not i386), but it would be nice if I could force not to build on armhf for example.

@evandandrea evandandrea changed the title from build.snapcraft.io ignore architectures in yaml to build.snapcraft.io ignores "architectures" field in snapcraft.yaml Apr 21, 2017

@ghost ghost added the Status: Proposal label Apr 24, 2017

@ghost ghost removed the Status: Proposal label Apr 25, 2017

@flexiondotorg

This comment has been minimized.

Show comment
Hide comment
@flexiondotorg

flexiondotorg May 24, 2017

I am running into the same issue for a different use case. A snapcraft.yaml that ingests a tarball from CI and said tarball is amd64 only, but there are now builds in the store for armhf and amd64.

flexiondotorg commented May 24, 2017

I am running into the same issue for a different use case. A snapcraft.yaml that ingests a tarball from CI and said tarball is amd64 only, but there are now builds in the store for armhf and amd64.

@evandandrea

This comment has been minimized.

Show comment
Hide comment
@evandandrea

evandandrea May 25, 2017

Collaborator

Also encountered here

Collaborator

evandandrea commented May 25, 2017

Also encountered here

@cjwatson

This comment has been minimized.

Show comment
Hide comment
@cjwatson

cjwatson May 25, 2017

Collaborator

Sadly this isn't what the architectures field means in snapcraft.yaml. Setting that field declares that you're building a single multi-architecture snap that works on that set of architectures: you get a *_multi.snap with that set of architectures in its meta/snap.yaml. That's not at all what's wanted here.

I agree that we should have the feature of being able to declare what architectures you want the snap to be built for, but it can't be done with the existing architectures field.

Collaborator

cjwatson commented May 25, 2017

Sadly this isn't what the architectures field means in snapcraft.yaml. Setting that field declares that you're building a single multi-architecture snap that works on that set of architectures: you get a *_multi.snap with that set of architectures in its meta/snap.yaml. That's not at all what's wanted here.

I agree that we should have the feature of being able to declare what architectures you want the snap to be built for, but it can't be done with the existing architectures field.

@evandandrea

This comment has been minimized.

Show comment
Hide comment
@evandandrea

evandandrea May 31, 2017

Collaborator

Sorry for the confusion. I just spoke with Sergio and Snapcraft will gain an option for specifying what architectures a snap should be built on:
https://lists.ubuntu.com/archives/snappy-app-devel/2016-April/000654.html

Collaborator

evandandrea commented May 31, 2017

Sorry for the confusion. I just spoke with Sergio and Snapcraft will gain an option for specifying what architectures a snap should be built on:
https://lists.ubuntu.com/archives/snappy-app-devel/2016-April/000654.html

@evandandrea evandandrea changed the title from build.snapcraft.io ignores "architectures" field in snapcraft.yaml to build.snapcraft.io should honour target-architectures when that lands in snapcraft May 31, 2017

@evandandrea

This comment has been minimized.

Show comment
Hide comment
@evandandrea

evandandrea Jun 5, 2017

Collaborator

Another reason for letting the developer limit the target architectures in the snapcraft.yaml, anbox's build badge shows as "failing" because it's (seemingly) not intended to work on ARM.

Collaborator

evandandrea commented Jun 5, 2017

Another reason for letting the developer limit the target architectures in the snapcraft.yaml, anbox's build badge shows as "failing" because it's (seemingly) not intended to work on ARM.

@evandandrea

This comment has been minimized.

Show comment
Hide comment
@evandandrea

evandandrea Jun 5, 2017

Collaborator

@sergiusens, can you estimate when this might land in snapcraft?

Collaborator

evandandrea commented Jun 5, 2017

@sergiusens, can you estimate when this might land in snapcraft?

@sergiusens

This comment has been minimized.

Show comment
Hide comment
@sergiusens

sergiusens Jun 5, 2017

sergiusens commented Jun 5, 2017

@elopio

This comment has been minimized.

Show comment
Hide comment
@elopio

elopio Nov 3, 2017

Contributor

For reference, this is the discussion in the forum:
https://forum.snapcraft.io/t/snapcraft-build-on-hint-for-builders/939

Contributor

elopio commented Nov 3, 2017

For reference, this is the discussion in the forum:
https://forum.snapcraft.io/t/snapcraft-build-on-hint-for-builders/939

@matthewpaulthomas

This comment has been minimized.

Show comment
Hide comment
@matthewpaulthomas

matthewpaulthomas Nov 6, 2017

Collaborator

Looks like the current dependency is snapcraft#1681.

Collaborator

matthewpaulthomas commented Nov 6, 2017

Looks like the current dependency is snapcraft#1681.

@jdxcode

This comment has been minimized.

Show comment
Hide comment
@jdxcode

jdxcode May 3, 2018

because https://github.com/snapcore/snapcraft/issues/1681 is merged does that mean this is available now?

jdxcode commented May 3, 2018

because https://github.com/snapcore/snapcraft/issues/1681 is merged does that mean this is available now?

@sergiusens

This comment has been minimized.

Show comment
Hide comment
@sergiusens

sergiusens May 3, 2018

sergiusens commented May 3, 2018

@cjwatson

This comment has been minimized.

Show comment
Hide comment
@cjwatson

cjwatson May 3, 2018

Collaborator

Can somebody please file this against Launchpad? We should keep this open on build.snapcraft.io as well, as I expect we'll need to expose a slightly different API for it to use once Launchpad is calculating the correct set of architectures, but the bulk of the remaining work lives in Launchpad and should be tracked there.

Collaborator

cjwatson commented May 3, 2018

Can somebody please file this against Launchpad? We should keep this open on build.snapcraft.io as well, as I expect we'll need to expose a slightly different API for it to use once Launchpad is calculating the correct set of architectures, but the bulk of the remaining work lives in Launchpad and should be tracked there.

@evandandrea

This comment has been minimized.

Show comment
Hide comment
@evandandrea
Collaborator

evandandrea commented May 10, 2018

@cjwatson

This comment has been minimized.

Show comment
Hide comment
@cjwatson

cjwatson Aug 6, 2018

Collaborator

The Launchpad parts of this are almost finished, and I'm working on the build.snapcraft.io parts now. However, I need a small amount of design input. @matthewpaulthomas, could you have a look over the following and indicate how this should be presented? Let me know if anything here is unclear.

This task calls for honouring metadata in snapcraft.yaml that indicates, among other things, which architectures a snap should be built for. As a result, before it can dispatch builds, Launchpad has to consult the relevant code hosting service - in this case, GitHub - to fetch snapcraft.yaml. But having HTTP request handlers that block while they go off and talk to some other service for an indefinite period of time is generally bad news; so the operation of requesting builds has to turn from synchronous HTTP requests ("request build for this snap on this architecture", which inserts a row directly into the Launchpad database) into asynchronous HTTP requests ("request all necessary builds for this snap", which schedules a job that consults GitHub, parses snapcraft.yaml, inserts database rows, and makes webhook deliveries back to build.snapcraft.io to indicate that it's done so).

The asynchronous operations here should normally only take a couple of seconds, but we can't guarantee that: for instance, our job runners might be backlogged, or GitHub might be slow. It's also possible for the asynchronous operations to fail: for instance, GitHub might be down, or the repository's snapcraft.yaml might be malformed.

From the user's point of view, when a build was requested (either by finishing initial configuration of the repository in BSI, by using "Build manually now", or automatically because the poller script noticed a commit to the relevant branch in their GitHub repository), the next change in the UI would previously have been that new rows would appear on the repo page, one for each architecture being built. After this change, we should instead indicate in some way that their request is in progress, so that it doesn't look like the site is ignoring them. If their request fails, we should tell them why.

My initial proposal would be to do something like this, although of course please do suggest something else if it seems better:

  • Show pending and recently-failed build requests in the "Latest builds" and "Previous builds" tables, sorted chronologically.
    • Should a pending build request cause the most recent set of actual builds to move down to "Previous builds", or should that only happen once the actual build rows show up?
    • I think it makes sense to present build requests in the same table as builds; anything else seems too fussy, and the "Build trigger" (apparently not in the design spec?) and "Result" columns still make sense more or less as-is. The "Architecture" and "Duration" columns should probably just be empty. What about "Number"? Build requests do have an ID, but it isn't from the same sequence as builds, so that could be confusing; but in the case of a failed request it probably does make sense to show the ID in some way because it's what we'd need to investigate the failure.
    • Exactly how should failed build requests be presented? We don't have a build log to link to in this case, but we do have an error message that we could show. (Nothing else in the builds tables currently shows error messages.)
  • For the time being, don't show completed build requests. In the longer term, they would be useful: we could group builds under their originating request, which would essentially be part of the implementation of build sets (see e.g. #902 and #983); but that's a larger chunk of work and we don't need to tackle it as part of this.
Collaborator

cjwatson commented Aug 6, 2018

The Launchpad parts of this are almost finished, and I'm working on the build.snapcraft.io parts now. However, I need a small amount of design input. @matthewpaulthomas, could you have a look over the following and indicate how this should be presented? Let me know if anything here is unclear.

This task calls for honouring metadata in snapcraft.yaml that indicates, among other things, which architectures a snap should be built for. As a result, before it can dispatch builds, Launchpad has to consult the relevant code hosting service - in this case, GitHub - to fetch snapcraft.yaml. But having HTTP request handlers that block while they go off and talk to some other service for an indefinite period of time is generally bad news; so the operation of requesting builds has to turn from synchronous HTTP requests ("request build for this snap on this architecture", which inserts a row directly into the Launchpad database) into asynchronous HTTP requests ("request all necessary builds for this snap", which schedules a job that consults GitHub, parses snapcraft.yaml, inserts database rows, and makes webhook deliveries back to build.snapcraft.io to indicate that it's done so).

The asynchronous operations here should normally only take a couple of seconds, but we can't guarantee that: for instance, our job runners might be backlogged, or GitHub might be slow. It's also possible for the asynchronous operations to fail: for instance, GitHub might be down, or the repository's snapcraft.yaml might be malformed.

From the user's point of view, when a build was requested (either by finishing initial configuration of the repository in BSI, by using "Build manually now", or automatically because the poller script noticed a commit to the relevant branch in their GitHub repository), the next change in the UI would previously have been that new rows would appear on the repo page, one for each architecture being built. After this change, we should instead indicate in some way that their request is in progress, so that it doesn't look like the site is ignoring them. If their request fails, we should tell them why.

My initial proposal would be to do something like this, although of course please do suggest something else if it seems better:

  • Show pending and recently-failed build requests in the "Latest builds" and "Previous builds" tables, sorted chronologically.
    • Should a pending build request cause the most recent set of actual builds to move down to "Previous builds", or should that only happen once the actual build rows show up?
    • I think it makes sense to present build requests in the same table as builds; anything else seems too fussy, and the "Build trigger" (apparently not in the design spec?) and "Result" columns still make sense more or less as-is. The "Architecture" and "Duration" columns should probably just be empty. What about "Number"? Build requests do have an ID, but it isn't from the same sequence as builds, so that could be confusing; but in the case of a failed request it probably does make sense to show the ID in some way because it's what we'd need to investigate the failure.
    • Exactly how should failed build requests be presented? We don't have a build log to link to in this case, but we do have an error message that we could show. (Nothing else in the builds tables currently shows error messages.)
  • For the time being, don't show completed build requests. In the longer term, they would be useful: we could group builds under their originating request, which would essentially be part of the implementation of build sets (see e.g. #902 and #983); but that's a larger chunk of work and we don't need to tackle it as part of this.
@matthewpaulthomas

This comment has been minimized.

Show comment
Hide comment
@matthewpaulthomas

matthewpaulthomas Aug 6, 2018

Collaborator

Should a pending build request cause the most recent set of actual builds to move down to "Previous builds", or should that only happen once the actual build rows show up?

That’s a tricky question. One objective of the “Latest builds” table is to highlight whether the developer needs to fix anything in the repo (a compilation error, for example, or trying to build on an inappropriate architecture). If the build request failed because the snapcraft.yaml is malformed, that’s something you need to fix, so I’d think that failure should occupy the “Latest builds” table, pushing previously-requested builds out. But if the build request failed because GitHub or Launchpad was down, that isn’t something the developer needs to fix, so then I’d prefer to keep the latest completed builds in the “Latest builds” table.

Is it practical to distinguish between those two types? If not, is one type much more common than the other?

What about "Number"? Build requests do have an ID, but it isn't from the same sequence as builds, so that could be confusing; but in the case of a failed request it probably does make sense to show the ID in some way because it's what we'd need to investigate the failure.

How long is a build request ID? If it was prefixed with “Request ”, would it fit in the current column width?

Exactly how should failed build requests be presented? We don't have a build log to link to in this case, but we do have an error message that we could show. (Nothing else in the builds tables currently shows error messages.)

How long is a typical error message of this kind? Can you provide an example?

Collaborator

matthewpaulthomas commented Aug 6, 2018

Should a pending build request cause the most recent set of actual builds to move down to "Previous builds", or should that only happen once the actual build rows show up?

That’s a tricky question. One objective of the “Latest builds” table is to highlight whether the developer needs to fix anything in the repo (a compilation error, for example, or trying to build on an inappropriate architecture). If the build request failed because the snapcraft.yaml is malformed, that’s something you need to fix, so I’d think that failure should occupy the “Latest builds” table, pushing previously-requested builds out. But if the build request failed because GitHub or Launchpad was down, that isn’t something the developer needs to fix, so then I’d prefer to keep the latest completed builds in the “Latest builds” table.

Is it practical to distinguish between those two types? If not, is one type much more common than the other?

What about "Number"? Build requests do have an ID, but it isn't from the same sequence as builds, so that could be confusing; but in the case of a failed request it probably does make sense to show the ID in some way because it's what we'd need to investigate the failure.

How long is a build request ID? If it was prefixed with “Request ”, would it fit in the current column width?

Exactly how should failed build requests be presented? We don't have a build log to link to in this case, but we do have an error message that we could show. (Nothing else in the builds tables currently shows error messages.)

How long is a typical error message of this kind? Can you provide an example?

@cjwatson

This comment has been minimized.

Show comment
Hide comment
@cjwatson

cjwatson Aug 7, 2018

Collaborator

Thanks for looking into this.

One objective of the “Latest builds” table is to highlight whether the developer needs to fix anything in the repo (a compilation error, for example, or trying to build on an inappropriate architecture).

I can see the use of distinguishing between temporary and permanent errors, much like HTTP 4xx being roughly "you screwed up" and 5xx being "we screwed up". The current build request interface doesn't allow for doing that in a reliable way, and I'd need to think about whether it's practical to extend it to do so. The main difficulty is that, while some permanent errors will be generated directly by our systems and could be tagged as such, some of them will be generated by e.g. GitHub instead (for instance, if the developer renames the repository without reconfiguring BSI); we can anticipate some of those, but I'm not certain that we'll be able to reliably anticipate all the possible permanent errors.

Most of the Launchpad support for build requests is on production, but it isn't in widespread use yet, so I don't have enough data to give accurate answers about which kind of failure is more common. My gut feel based on stray cron mail and the like is that permanent errors will be rather more common than temporary ones, and it certainly ought to be that way if both we and GitHub are doing a good job of maintaining our services, but that's just a guess.

One other consideration is that if the build request fails for a temporary reason, then the developer probably still needs to do something, even if it's only trying to request the build again: it may well not be retried automatically. (I acknowledge that this is a slightly weak argument on its own.)

How long is a build request ID? If it was prefixed with “Request ”, would it fit in the current column width?

I hadn't been considering surfacing them in the UI when I designed the data structures, so the build request ID is just the ID column of the Job table in Launchpad, and there are rather a lot of jobs of all shapes and sizes. As a result, build request IDs on production are currently eight digits, as opposed to the six-digit build IDs, and I think that would overflow the current column width. That's not an intrinsic limitation: we could make up and backfill some other ID just for these jobs in order to make them more manageably-sized, although it would be a fiddly operation.

However, once they start being used, we should probably expect build request IDs to approach within an order of magnitude of build IDs over time, since successful build requests will currently result in between one and six builds. I think that might eventually end up being quite tight anyway. Maybe some kind of symbol or icon instead of "Request "?

How long is a typical error message of this kind? Can you provide an example?

In some cases it could be multi-line, although we could divide that up into a primary message and a detail field if need be. Some examples of permanent errors:

And temporary errors will typically be something like:

(I'm not set on the current text, and now that I actually type out examples it's clear some of these aren't ideal; the repetition in the typical temporary error case is particularly unfortunate. I do feel that if we're issuing what amounts to a parse error we need to give the developer details about exactly what we're objecting to, though.)

Collaborator

cjwatson commented Aug 7, 2018

Thanks for looking into this.

One objective of the “Latest builds” table is to highlight whether the developer needs to fix anything in the repo (a compilation error, for example, or trying to build on an inappropriate architecture).

I can see the use of distinguishing between temporary and permanent errors, much like HTTP 4xx being roughly "you screwed up" and 5xx being "we screwed up". The current build request interface doesn't allow for doing that in a reliable way, and I'd need to think about whether it's practical to extend it to do so. The main difficulty is that, while some permanent errors will be generated directly by our systems and could be tagged as such, some of them will be generated by e.g. GitHub instead (for instance, if the developer renames the repository without reconfiguring BSI); we can anticipate some of those, but I'm not certain that we'll be able to reliably anticipate all the possible permanent errors.

Most of the Launchpad support for build requests is on production, but it isn't in widespread use yet, so I don't have enough data to give accurate answers about which kind of failure is more common. My gut feel based on stray cron mail and the like is that permanent errors will be rather more common than temporary ones, and it certainly ought to be that way if both we and GitHub are doing a good job of maintaining our services, but that's just a guess.

One other consideration is that if the build request fails for a temporary reason, then the developer probably still needs to do something, even if it's only trying to request the build again: it may well not be retried automatically. (I acknowledge that this is a slightly weak argument on its own.)

How long is a build request ID? If it was prefixed with “Request ”, would it fit in the current column width?

I hadn't been considering surfacing them in the UI when I designed the data structures, so the build request ID is just the ID column of the Job table in Launchpad, and there are rather a lot of jobs of all shapes and sizes. As a result, build request IDs on production are currently eight digits, as opposed to the six-digit build IDs, and I think that would overflow the current column width. That's not an intrinsic limitation: we could make up and backfill some other ID just for these jobs in order to make them more manageably-sized, although it would be a fiddly operation.

However, once they start being used, we should probably expect build request IDs to approach within an order of magnitude of build IDs over time, since successful build requests will currently result in between one and six builds. I think that might eventually end up being quite tight anyway. Maybe some kind of symbol or icon instead of "Request "?

How long is a typical error message of this kind? Can you provide an example?

In some cases it could be multi-line, although we could divide that up into a primary message and a detail field if need be. Some examples of permanent errors:

And temporary errors will typically be something like:

(I'm not set on the current text, and now that I actually type out examples it's clear some of these aren't ideal; the repetition in the typical temporary error case is particularly unfortunate. I do feel that if we're issuing what amounts to a parse error we need to give the developer details about exactly what we're objecting to, though.)

@cjwatson

This comment has been minimized.

Show comment
Hide comment
@cjwatson

cjwatson Aug 7, 2018

Collaborator

Oh, also, note that the question of whether pending build requests should push previous builds out of the "Latest builds" table is slightly different from the question of whether previously-failed build requests should do so.

Collaborator

cjwatson commented Aug 7, 2018

Oh, also, note that the question of whether pending build requests should push previous builds out of the "Latest builds" table is slightly different from the question of whether previously-failed build requests should do so.

@matthewpaulthomas matthewpaulthomas self-assigned this Aug 9, 2018

@matthewpaulthomas

This comment has been minimized.

Show comment
Hide comment
@matthewpaulthomas

matthewpaulthomas Aug 10, 2018

Collaborator

@cjwatson Ok, how about this:

The “Latest builds” table should describe, for each of the last-known-specified architectures, the latest requested build in that architecture, whether or not it has completed or even started.

  • If it has not yet started, the “Number” cell should contain only “Requested”.
  • If the request failed, the table should collapse to one row, with the “Architecture”, “Duration”, and “Result” cells merged to display the request ID and error message, for example, “(Request #11858014) Can’t find snapcraft.yaml in https://github.com/example/test master”.

This assumes that you don’t need to see a request ID unless the build request actually fails.

Collaborator

matthewpaulthomas commented Aug 10, 2018

@cjwatson Ok, how about this:

The “Latest builds” table should describe, for each of the last-known-specified architectures, the latest requested build in that architecture, whether or not it has completed or even started.

  • If it has not yet started, the “Number” cell should contain only “Requested”.
  • If the request failed, the table should collapse to one row, with the “Architecture”, “Duration”, and “Result” cells merged to display the request ID and error message, for example, “(Request #11858014) Can’t find snapcraft.yaml in https://github.com/example/test master”.

This assumes that you don’t need to see a request ID unless the build request actually fails.

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Sep 27, 2018

Colin Watson
Move "Build trigger" column before "Architecture"
The "Build trigger" column is currently missing from the design
specification.  Design work for canonical-websites#556 calls for failed build requests to
have their request ID and error message rendered spanning the
"Architecture", "Duration", and "Result" columns; since failed build
requests still have a meaningful build trigger, having "Build trigger"
in the middle of this group of columns makes it hard to comply with that
otherwise-reasonable suggestion.

Moving "Build trigger" two columns to the left resolves this difficulty,
and seems to make just as much sense.  In fact, once we start using
build requests, it arguably makes more sense (because the build trigger
is known before the set of built-on architectures is known).

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Oct 1, 2018

Colin Watson
Simplify internalGetSnapBuilds return value
`internalGetSnapBuilds` previously returned a somewhat mangled
`Collection` (a type used by the Launchpad web service client that
automatically handles batching and iteration of large collections).
However, this didn't make much sense here because the mangling meant
that the automatic batching wouldn't work quite right, and the callers
all just picked out the `entries` array anyway; it will make even less
sense as a `Collection` after upcoming work for canonical-websites#556.

We now just return an array instead.

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Oct 1, 2018

Colin Watson
Prepare UI for upcoming build request changes
We will shortly need to change new snap build requests to be dispatched
more asynchronously, in order that Launchpad can fetch `snapcraft.yaml`
from the repository and decide which architectures the snap should be
built for.  This means that a snap can be in a state where it has
pending build requests (different from pending builds: for instance,
they don't have an architecture), and also where it has failed build
requests in its history (for example, because `snapcraft.yaml` is
malformed in such a way that Launchpad can't determine which
architectures it should be built for).

This prepares the UI layer for that work by ensuring that it can
understand the modified server responses and render build requests in
the builds list.  The server side hasn't been changed yet (that will be
a follow-up PR), so this should result in no functional change for the
moment.

Build requests will be interleaved with builds in server responses, and
the UI tells them apart by looking at the `resource_type_link`.  This
wasn't my first choice for designing the internal API, but it turns out
to be the simplest option: in particular, if we want to be able to
support sensible pagination of the combined build and build request
history in future then we need to be iterating over a single collection.

Part of canonical-websites#556.

cjwatson added a commit that referenced this issue Oct 2, 2018

Simplify internalGetSnapBuilds return value (#1151)
`internalGetSnapBuilds` previously returned a somewhat mangled
`Collection` (a type used by the Launchpad web service client that
automatically handles batching and iteration of large collections).
However, this didn't make much sense here because the mangling meant
that the automatic batching wouldn't work quite right, and the callers
all just picked out the `entries` array anyway; it will make even less
sense as a `Collection` after upcoming work for #556.

We now just return an array instead.

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Oct 2, 2018

Colin Watson
Prepare UI for upcoming build request changes
We will shortly need to change new snap build requests to be dispatched
more asynchronously, in order that Launchpad can fetch `snapcraft.yaml`
from the repository and decide which architectures the snap should be
built for.  This means that a snap can be in a state where it has
pending build requests (different from pending builds: for instance,
they don't have an architecture), and also where it has failed build
requests in its history (for example, because `snapcraft.yaml` is
malformed in such a way that Launchpad can't determine which
architectures it should be built for).

This prepares the UI layer for that work by ensuring that it can
understand the modified server responses and render build requests in
the builds list.  The server side hasn't been changed yet (that will be
a follow-up PR), so this should result in no functional change for the
moment.

Build requests will be interleaved with builds in server responses, and
the UI tells them apart by looking at the `resource_type_link`.  This
wasn't my first choice for designing the internal API, but it turns out
to be the simplest option: in particular, if we want to be able to
support sensible pagination of the combined build and build request
history in future then we need to be iterating over a single collection.

Part of canonical-websites#556.

cjwatson added a commit that referenced this issue Oct 2, 2018

Prepare UI for upcoming build request changes (#1152)
We will shortly need to change new snap build requests to be dispatched
more asynchronously, in order that Launchpad can fetch `snapcraft.yaml`
from the repository and decide which architectures the snap should be
built for.  This means that a snap can be in a state where it has
pending build requests (different from pending builds: for instance,
they don't have an architecture), and also where it has failed build
requests in its history (for example, because `snapcraft.yaml` is
malformed in such a way that Launchpad can't determine which
architectures it should be built for).

This prepares the UI layer for that work by ensuring that it can
understand the modified server responses and render build requests in
the builds list.  The server side hasn't been changed yet (that will be
a follow-up PR), so this should result in no functional change for the
moment.

Build requests will be interleaved with builds in server responses, and
the UI tells them apart by looking at the `resource_type_link`.  This
wasn't my first choice for designing the internal API, but it turns out
to be the simplest option: in particular, if we want to be able to
support sensible pagination of the combined build and build request
history in future then we need to be iterating over a single collection.

Part of #556.

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Oct 2, 2018

Colin Watson
Return pending build requests from getSnapBuilds
This allows the UI to show the user that there's a pending build
request, rather than appearing unresponsive in the interval between a
set of builds being requested and the request being processed.

The database gains a new `build_request_annotation` table, allowing us
to keep track of build reasons even when the process of creating new
builds goes through a build request.

I'd like to return failed build requests as well, but there are some
significant complications involved in making that work efficiently in
conjunction with pagination, mainly on the Launchpad side.  I don't
think that needs to block the rest of this work.

Part of canonical-websites#556.
@cjwatson

This comment has been minimized.

Show comment
Hide comment
@cjwatson

cjwatson Oct 2, 2018

Collaborator

@matthewpaulthomas Thanks. There are minor wording issues (a build request can have completed and dispatched builds for individual architectures which in turn haven't started yet, and in that case I think we should display the individual builds as before, not just "Requested"), but I get the gist and am finishing off the implementation now.

Displaying failed build requests turns out to be unexpectedly difficult due to some issues with passing appropriately-paginated lists around between various APIs, but that doesn't need to block this issue and we can deal with it separately.

Collaborator

cjwatson commented Oct 2, 2018

@matthewpaulthomas Thanks. There are minor wording issues (a build request can have completed and dispatched builds for individual architectures which in turn haven't started yet, and in that case I think we should display the individual builds as before, not just "Requested"), but I get the gist and am finishing off the implementation now.

Displaying failed build requests turns out to be unexpectedly difficult due to some issues with passing appropriately-paginated lists around between various APIs, but that doesn't need to block this issue and we can deal with it separately.

cjwatson added a commit that referenced this issue Oct 3, 2018

Return pending build requests from getSnapBuilds (#1154)
This allows the UI to show the user that there's a pending build
request, rather than appearing unresponsive in the interval between a
set of builds being requested and the request being processed.

The database gains a new `build_request_annotation` table, allowing us
to keep track of build reasons even when the process of creating new
builds goes through a build request.

I'd like to return failed build requests as well, but there are some
significant complications involved in making that work efficiently in
conjunction with pagination, mainly on the Launchpad side.  I don't
think that needs to block the rest of this work.

Part of #556.

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Oct 3, 2018

Colin Watson
Extend webhook handler to support build requests
Launchpad sends a webhook when each build is created, and we now respond
to that by recording the link between the build and its parent build
request.

As is generally appropriate for webhooks, this is non-essential: if the
webhook delivery goes missing, we'll miss out on some metrics and won't
be able to present the correct build reason, but everything else will
still work.

Part of canonical-websites#556.

cjwatson added a commit that referenced this issue Oct 3, 2018

Extend webhook handler to support build requests (#1156)
Launchpad sends a webhook when each build is created, and we now respond
to that by recording the link between the build and its parent build
request.

As is generally appropriate for webhooks, this is non-essential: if the
webhook delivery goes missing, we'll miss out on some metrics and won't
be able to present the correct build reason, but everything else will
still work.

Part of #556.

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Oct 3, 2018

Colin Watson
Extend poller to check pending build requests
If a build request is pending for a snap, we skip triggering new builds
just as we do if there have been recent builds.

Part of canonical-websites#556.

cjwatson added a commit that referenced this issue Oct 4, 2018

Extend poller to check pending build requests (#1157)
If a build request is pending for a snap, we skip triggering new builds
just as we do if there have been recent builds.

Part of #556.

cjwatson pushed a commit to cjwatson/build.snapcraft.io that referenced this issue Oct 4, 2018

Colin Watson
Request new builds using build requests
This makes Launchpad responsible for asynchronously figuring out which
architectures the snap should be built for, thereby allowing us to
support Snapcraft's `architectures` keyword in `snapcraft.yaml`.

Fixes canonical-websites#556.

cjwatson added a commit that referenced this issue Oct 5, 2018

Request new builds using build requests (#1158)
This makes Launchpad responsible for asynchronously figuring out which
architectures the snap should be built for, thereby allowing us to
support Snapcraft's `architectures` keyword in `snapcraft.yaml`.

Fixes #556.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment