Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history 
commit b381c87
Merge: 2289275 baf8b9a
Author: Tao ZHOU <angeloudy@gmail.com>
Date:   Thu Aug 8 10:13:30 2019 +1000

    Merge branch 'master' into support-glob-recursive

commit baf8b9a
Merge: c54138c e60b114
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Mon Aug 5 23:14:34 2019 +0200

    Merge pull request buildbot#4941 from PhilippSelenium/PhilippSelenium-patch-ubuntu18.04

    Philipp selenium patch ubuntu18.04

commit c54138c
Merge: da38451 3b4a0cc
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Mon Aug 5 23:13:37 2019 +0200

    Merge pull request buildbot#4927 from rajdeepbharati/b4c

    Add webpage to display all builds for a revision

commit e60b114
Author: PhilippSelenium <31542906+PhilippSelenium@users.noreply.github.com>
Date:   Mon Aug 5 15:20:32 2019 +0200

    Added newsfragment

commit e7ba334
Author: PhilippSelenium <31542906+PhilippSelenium@users.noreply.github.com>
Date:   Mon Aug 5 15:12:34 2019 +0200

    Buildbot worker dockerfile from ubuntu 18.04

    1. dumb-init can be installed from apt
    2. pip can be installed from apt

commit 3b4a0cc
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Mon Aug 5 16:30:15 2019 +0530

    Create new changebuilds page

    Displays all builds for the selected change.

commit da38451
Merge: 7e3559f ca880c5
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Sat Aug 3 14:16:22 2019 +0200

    Merge pull request buildbot#4934 from kravietz/bitbucketcloud_update

    Fix buildbot#4873 per latest Bitbucket API documentation

commit ca880c5
Author: Pawel Krawczyk <616047+kravietz@users.noreply.github.com>
Date:   Sat Aug 3 12:22:25 2019 +0100

    Remove one more occurence of `username` field

commit 5bc59ac
Author: Pawel Krawczyk <616047+kravietz@users.noreply.github.com>
Date:   Sat Aug 3 10:31:54 2019 +0100

    Simplify project name retrieval

commit 3285cfa
Author: Pawel Krawczyk <616047+kravietz@users.noreply.github.com>
Date:   Fri Aug 2 10:53:50 2019 +0100

    Reflect recent changes in Bitbucket Cloud API

commit fc7d39b
Author: Pawel Krawczyk <616047+kravietz@users.noreply.github.com>
Date:   Fri Aug 2 10:50:11 2019 +0100

    Add newfragment

commit 74f1040
Author: Pawel Krawczyk <616047+kravietz@users.noreply.github.com>
Date:   Fri Aug 2 10:35:33 2019 +0100

    Fix buildbot#4873 per latest Bitbucket API documentation

    * replace the username property with nickname (per the privacy changes in API)
      https://confluence.atlassian.com/bitbucket/event-payloads-740262817.html#EventPayloads-entity_user

    * per Bitbucket documentation the project property in repository is not guaranteed to be always present which was also crashing my Buildbot
      https://confluence.atlassian.com/bitbucket/event-payloads-740262817.html#EventPayloads-entity_repositoryRepository

commit 2289275
Merge: f28087b 0b1bb83
Author: angeloudy <angeloudy@yahoo.com>
Date:   Fri Aug 2 16:09:00 2019 +1000

    Merge branch 'support-glob-recursive' of https://github.com/angeloudy/buildbot into support-glob-recursive

commit f28087b
Author: angeloudy <angeloudy@yahoo.com>
Date:   Fri Aug 2 16:08:22 2019 +1000

    Update tests

commit 0b1bb83
Merge: d4cac79 7e3559f
Author: Tao ZHOU <angeloudy@gmail.com>
Date:   Fri Aug 2 15:54:20 2019 +1000

    Merge branch 'master' into support-glob-recursive

commit d4cac79
Author: angeloudy <angeloudy@yahoo.com>
Date:   Fri Aug 2 15:53:16 2019 +1000

    Add tests

commit c8f761d
Author: angeloudy <angeloudy@yahoo.com>
Date:   Fri Aug 2 15:24:41 2019 +1000

    recursive glob is only supported in python3.5+

commit 7e3559f
Merge: 14a2337 7658c96
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 20:08:50 2019 +0200

    Merge pull request buildbot#4932 from tardyp/trigger_ui

    fix initial value of the pagination counter and polish pagination UI

commit 14a2337
Merge: 5a15c12 7b78ea5
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 20:08:22 2019 +0200

    Merge pull request buildbot#4904 from rajdeepbharati/b4c-api-bug

    Handle exception in BuildsEndpoint

commit 5a15c12
Merge: ad33ac8 1e65e53
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 17:53:40 2019 +0200

    Merge pull request buildbot#4901 from rajdeepbharati/waterfall-tags

    Allow filtering by tags in waterfall view

commit 1e65e53
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Thu Aug 1 20:11:17 2019 +0530

    Minor refactoring

commit 7658c96
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 14:16:30 2019 +0200

    polish pagination widget look

commit 2c9cf25
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 14:06:44 2019 +0200

    fix initial value of the pagination counter

commit 7b78ea5
Merge: 2a00246 ad33ac8
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 13:45:28 2019 +0200

    Merge branch 'master' into b4c-api-bug

commit ad33ac8
Merge: 76600a6 f649c0e
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 13:32:30 2019 +0200

    Merge pull request buildbot#4920 from bertranddemiddelaer/user/bertrand/fix_multiple_sql_queries

    divide number of sql queries for getPrevSuccessfulBuild() by 100

commit 76600a6
Merge: 386d05b 039d3a1
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Aug 1 13:30:11 2019 +0200

    Merge pull request buildbot#4923 from muks/mock-docs-update

    Mock docs update

commit f649c0e
Author: Bertrand Demiddelaer <bertrand.demiddelaer@scality.com>
Date:   Thu Aug 1 10:06:20 2019 +0200

    cosmetics on newsfragment

commit 01f83fa
Merge: 95da954 5217cb9
Author: angeloudy <angeloudy@yahoo.com>
Date:   Thu Aug 1 11:31:56 2019 +1000

    Merge branch 'support-glob-recursive' of https://github.com/angeloudy/buildbot into support-glob-recursive

commit 95da954
Author: angeloudy <angeloudy@yahoo.com>
Date:   Thu Aug 1 11:29:24 2019 +1000

    Add newsfragment

commit fadf22a
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Thu Aug 1 03:31:48 2019 +0530

    Rename setTagBuilders -> makeTagBuilders

commit 0245d9c
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Tue Jul 30 20:07:25 2019 +0530

    Add e2e test for waterfall tag

commit 2a00246
Merge: 003005f 386d05b
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Wed Jul 31 18:49:07 2019 +0200

    Merge branch 'master' into b4c-api-bug

commit 2af9902
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Wed Jul 31 18:48:11 2019 +0200

    Update getPrevSuccessfulBuild-number-of-sql-queries.bugfix

commit 2b6debe
Merge: 6487085 386d05b
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Wed Jul 31 18:47:48 2019 +0200

    Merge branch 'master' into user/bertrand/fix_multiple_sql_queries

commit 5217cb9
Merge: 6a2ac28 386d05b
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Wed Jul 31 18:46:36 2019 +0200

    Merge branch 'master' into support-glob-recursive

commit 039d3a1
Merge: 24cc8cf 386d05b
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Wed Jul 31 18:46:20 2019 +0200

    Merge branch 'master' into mock-docs-update

commit 386d05b
Merge: 501f1d9 056301f
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Wed Jul 31 18:44:16 2019 +0200

    Merge pull request buildbot#4929 from tardyp/trigger_ui

    Trigger ui optimization for large scale triggers

commit 6487085
Author: Bertrand Demiddelaer <bertrand.demiddelaer@scality.com>
Date:   Fri Jul 26 16:03:49 2019 +0200

    added a newsfragment

commit 056301f
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Tue Jul 30 14:23:51 2019 +0200

    add paging to the trigger step UI

    This will avoid big slow downs of the UI when managing huge numbers of triggered builds

commit fd46efa
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Tue Jul 30 13:40:53 2019 +0200

    buildsummary: optimize split between buildrequest URL other URLs

    We only compute those when the step data is changing.

commit 501f1d9
Merge: 8019cfc eb6eff5
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Tue Jul 30 18:02:08 2019 +0200

    Merge pull request buildbot#4931 from rajdeepbharati/pyparsing

    Update pyparsing from 2.4.1 to 2.4.2

commit eb6eff5
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Tue Jul 30 21:04:23 2019 +0530

    Update pyparsing from 2.4.1 to 2.4.2

commit 24cc8cf
Author: Mukund Sivaraman <muks@akira.org>
Date:   Mon Jul 29 14:07:01 2019 +0530

    Add docs on how to get the build master to monitor mock steps output

commit 28db079
Author: Mukund Sivaraman <muks@akira.org>
Date:   Mon Jul 29 14:06:31 2019 +0530

    Fix typo in doc

commit 6a2ac28
Author: angeloudy <angeloudy@yahoo.com>
Date:   Mon Jul 29 16:33:24 2019 +1000

    Support '**' in glob

commit 8019cfc
Merge: d41e911 a8b6694
Author: Povilas Kanapickas <povilas@radix.lt>
Date:   Sat Jul 27 20:41:02 2019 +0300

    Merge pull request buildbot#4917 from rajdeepbharati/favicon

    Configure webpack to load images.

commit 003005f
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Fri Jul 26 16:00:53 2019 +0200

    Update builds.py

commit 4c2ac16
Author: Bertrand Demiddelaer <bertrand.demiddelaer@scality.com>
Date:   Fri Jul 26 15:59:03 2019 +0200

    divide the number of sql queries triggered by getPrevSuccessfulBuild() by up to 100

commit d41e911
Merge: 7e50f43 9c4d25c
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Fri Jul 26 15:56:48 2019 +0200

    Merge pull request buildbot#4909 from esc/docs/fix_typo

    fix typo

commit 7e50f43
Merge: a862079 5213c39
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Fri Jul 26 15:56:18 2019 +0200

    Merge pull request buildbot#4910 from buildbot/pyup-scheduled-update-2019-07-22

    Scheduled weekly dependency update for week 29

commit a862079
Merge: ce20a96 b7c8e76
Author: Povilas Kanapickas <povilas@radix.lt>
Date:   Thu Jul 25 21:14:58 2019 +0300

    Merge pull request buildbot#4907 from tardyp/github_reporter_erroring

    github: Fix error handling in case of bad interpolaters

commit a8b6694
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Wed Jul 24 22:06:40 2019 +0530

    Configure webpack to load images.

    Import img files in app.module.js and configure webpack.

    Fixes buildbot#4880

commit 7531fbd
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Wed Jul 24 19:13:34 2019 +0530

    Allow changing tags from url without reloading

    watch for $locationChangeSuccess

commit 5213c39
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:25 2019 +0200

    Update werkzeug from 0.15.4 to 0.15.5

commit 45a2e80
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:24 2019 +0200

    Update sqlalchemy from 1.3.5 to 1.3.6

commit f4e91de
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:23 2019 +0200

    Update pyparsing from 2.4.0 to 2.4.1

commit 78505d6
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:22 2019 +0200

    Update pillow from 6.0.0 to 6.1.0

commit 83ae089
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:21 2019 +0200

    Update pbr from 5.3.1 to 5.4.1

commit a9ef7be
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:20 2019 +0200

    Update pbr from 5.3.1 to 5.4.1

commit 68c0c47
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:19 2019 +0200

    Update moto from 1.3.9 to 1.3.13

commit 2a44b5b
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:17 2019 +0200

    Update flake8 from 3.7.7 to 3.7.8

commit 955df86
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:16 2019 +0200

    Update docutils from 0.14 to 0.15

commit 1e13ef9
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:15 2019 +0200

    Update botocore from 1.12.180 to 1.12.192

commit b48d5db
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:14 2019 +0200

    Update boto3 from 1.9.180 to 1.9.192

commit d10117b
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 22 16:05:13 2019 +0200

    Update autobahn from 19.6.2 to 19.7.1

commit 6b775d7
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Sun Jul 21 18:45:24 2019 +0530

    Handle exception in BuildsEndpoint

    Check whether `filters` variable is defined.

    Fixes buildbot#4903

commit 72cc587
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Mon Jul 22 18:39:50 2019 +0530

    Restructure code related to waterfall tags

    Use better variable/function names, follow best practices.

commit 9c4d25c
Author: Valentin Haenel <vhaenel@anaconda.com>
Date:   Mon Jul 22 15:04:52 2019 +0200

    fix typo

commit b7c8e76
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Mon Jul 22 13:31:24 2019 +0200

    github: Fix error handling in case of bad interpolaters

commit 7cc1cb9
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Sat Jul 20 20:41:03 2019 +0530

    Allow filtering by tags in waterfall view

    Closes buildbot#3471

commit ce20a96
Merge: 3a084f7 3fb462e
Author: Povilas Kanapickas <povilas@radix.lt>
Date:   Sun Jul 21 12:37:37 2019 +0300

    Merge pull request buildbot#4902 from rajdeepbharati/redundant-fn

    Remove redundant function getAllTags in builders.controller.js

commit 3fb462e
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Sat Jul 20 23:09:37 2019 +0530

    Remove redundant function getAllTags in builders.controller.js

    Closes buildbot#4900

commit 3a084f7
Merge: 8846ca9 9233f4c
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Jul 18 09:58:11 2019 +0200

    Merge pull request buildbot#4891 from D4ryus/master

    GerritStatusPush: Accept wantSteps

commit 8846ca9
Merge: d6f8f29 a8ac2a8
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Jul 18 09:57:38 2019 +0200

    Merge pull request buildbot#4872 from buildbot/pyup-scheduled-update-2019-07-01

    Scheduled weekly dependency update for week 26

commit d6f8f29
Merge: 2efbcb6 d941c7d
Author: Pierre Tardy <tardyp@gmail.com>
Date:   Thu Jul 18 09:57:19 2019 +0200

    Merge pull request buildbot#4892 from rajdeepbharati/buildsforchange

    Get all builds corresponding to a change

commit d941c7d
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Wed Jul 17 20:54:30 2019 +0530

    Reuse code

commit 66f0f5e
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Wed Jul 17 16:33:33 2019 +0530

    Add buildDict to spelling_wordlist

commit ab27672
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Wed Jul 17 16:19:23 2019 +0530

    Add fakedb test for getBuildsForChange

commit 2efbcb6
Merge: 41d2feb 3cdf67c
Author: Povilas Kanapickas <povilas@radix.lt>
Date:   Tue Jul 16 23:50:36 2019 +0300

    Merge pull request buildbot#4893 from angeloudy/master

    Increase performance of deleteOldLogChunks

commit 40f9501
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Tue Jul 16 21:03:29 2019 +0530

    Add tests for getBuildsForChange

commit f517230
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Tue Jul 16 20:54:33 2019 +0530

    Add builds for change route in data API

commit 3171e87
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Tue Jul 16 20:30:56 2019 +0530

    Add docs for getBuildsForChange

commit c7829dc
Author: Rajdeep Bharati <rajdeep@macports.org>
Date:   Mon Jul 15 22:59:11 2019 +0530

    Add getBuildsForChange method

    This will enable getting all builds corresponding to a change.

commit 9233f4c
Author: d4ryus <d4ryus@teknik.io>
Date:   Fri Jul 12 13:53:24 2019 +0200

    GerritStatusPush: Accept wantSteps

    Setting wantSteps=True on GerritStatusPush has no effect since it wont
    get passed to getDetailForBuilds. Change this by passing down wantSteps
    to getDetailsForBuilds in getBuildDetails.

commit a8ac2a8
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 1 19:59:43 2019 +0200

    Update pg8000 from 1.13.1 to 1.13.2

commit 59fd4ab
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 1 19:59:42 2019 +0200

    Update snowballstemmer from 1.2.1 to 1.9.0

commit dd09f51
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 1 19:59:41 2019 +0200

    Update pathlib2 from 2.3.3 to 2.3.4

commit 2284f44
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 1 19:59:40 2019 +0200

    Update moto from 1.3.8 to 1.3.9

commit 69ac67f
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 1 19:59:39 2019 +0200

    Update isort from 4.3.20 to 4.3.21

commit ee11ab1
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 1 19:59:38 2019 +0200

    Update botocore from 1.12.174 to 1.12.180

commit c2371ed
Author: pyup-bot <github-bot@pyup.io>
Date:   Mon Jul 1 19:59:37 2019 +0200

    Update boto3 from 1.9.174 to 1.9.180
  • Loading branch information
angeloudy committed Aug 13, 2019
1 parent 3cdf67c commit 701485f
Show file tree
Hide file tree
Showing 44 changed files with 557 additions and 127 deletions.
40 changes: 24 additions & 16 deletions master/buildbot/data/builds.py
View file
Expand Up @@ -137,30 +137,37 @@ class BuildsEndpoint(Db2DataMixin, base.BuildNestingMixin, base.Endpoint):
/builders/n:builderid/builds
/builders/i:buildername/builds
/buildrequests/n:buildrequestid/builds
/changes/n:changeid/builds
/workers/n:workerid/builds
"""
rootLinkName = 'builds'

@defer.inlineCallbacks
def get(self, resultSpec, kwargs):
# following returns None if no filter
# true or false, if there is a complete filter
builderid = None
if 'builderid' in kwargs or 'buildername' in kwargs:
builderid = yield self.getBuilderId(kwargs)
if builderid is None:
return []
complete = resultSpec.popBooleanFilter("complete")
buildrequestid = resultSpec.popIntegerFilter("buildrequestid")
resultSpec.fieldMapping = self.fieldMapping
builds = yield self.master.db.builds.getBuilds(
builderid=builderid,
buildrequestid=kwargs.get('buildrequestid', buildrequestid),
workerid=kwargs.get('workerid'),
complete=complete,
resultSpec=resultSpec)
changeid = kwargs.get('changeid')
if changeid is not None:
builds = yield self.master.db.builds.getBuildsForChange(changeid)
else:
# following returns None if no filter
# true or false, if there is a complete filter
builderid = None
if 'builderid' in kwargs or 'buildername' in kwargs:
builderid = yield self.getBuilderId(kwargs)
if builderid is None:
return []
complete = resultSpec.popBooleanFilter("complete")
buildrequestid = resultSpec.popIntegerFilter("buildrequestid")
resultSpec.fieldMapping = self.fieldMapping
builds = yield self.master.db.builds.getBuilds(
builderid=builderid,
buildrequestid=kwargs.get('buildrequestid', buildrequestid),
workerid=kwargs.get('workerid'),
complete=complete,
resultSpec=resultSpec)

# returns properties' list
filters = resultSpec.popProperties()

buildscol = []
for b in builds:
data = yield self.db2data(b)
Expand All @@ -171,6 +178,7 @@ def get(self, resultSpec, kwargs):
props, filters)
if filtered_properties:
data['properties'] = filtered_properties

buildscol.append(data)
return buildscol

Expand Down
33 changes: 31 additions & 2 deletions master/buildbot/db/builds.py
View file
Expand Up @@ -72,6 +72,7 @@ def getPrevSuccessfulBuild(self, builderid, number, ssBuild):
rv = None
tbl = self.db.model.builds
offset = 0
increment = 1000
matchssBuild = {(ss['repository'],
ss['branch'],
ss['codebase']) for ss in ssBuild}
Expand All @@ -81,7 +82,7 @@ def getPrevSuccessfulBuild(self, builderid, number, ssBuild):
(tbl.c.number < number) &
(tbl.c.results == 0)),
offset=offset,
limit=10)
limit=increment)
if not prevBuilds:
break
for prevBuild in prevBuilds:
Expand All @@ -93,10 +94,38 @@ def getPrevSuccessfulBuild(self, builderid, number, ssBuild):
# repository/branch/codebase was found !
rv = prevBuild
break
offset += 10
offset += increment

return rv

def getBuildsForChange(self, changeid):
assert changeid > 0

def thd(conn):
# Get builds for the change
changes_tbl = self.db.model.changes
bsets_tbl = self.db.model.buildsets
bsss_tbl = self.db.model.buildset_sourcestamps
reqs_tbl = self.db.model.buildrequests
builds_tbl = self.db.model.builds

from_clause = changes_tbl.join(bsss_tbl,
changes_tbl.c.sourcestampid == bsss_tbl.c.sourcestampid)
from_clause = from_clause.join(bsets_tbl,
bsss_tbl.c.buildsetid == bsets_tbl.c.id)
from_clause = from_clause.join(reqs_tbl,
bsets_tbl.c.id == reqs_tbl.c.buildsetid)
from_clause = from_clause.join(builds_tbl,
reqs_tbl.c.id == builds_tbl.c.buildrequestid)

q = sa.select([builds_tbl]).select_from(
from_clause).where(changes_tbl.c.changeid == changeid)
res = conn.execute(q)
return [self._builddictFromRow(row)
for row in res.fetchall()]

return self.db.pool.do(thd)

# returns a Deferred that returns a value
def getBuilds(self, builderid=None, buildrequestid=None, workerid=None, complete=None, resultSpec=None):
def thd(conn):
Expand Down
1 change: 1 addition & 0 deletions master/buildbot/newsfragments/buildbot-worker-ubuntu-18.04.feature
View file
@@ -0,0 +1 @@
:issue:`4928`: Update buildbot worker image to ubuntu 18.04
1 change: 1 addition & 0 deletions master/buildbot/newsfragments/fix_bitbucketcloud_hook.bugfix
View file
@@ -0,0 +1 @@
:issue:`4873`: Fix Bitbucket Cloud hook crash due to changes in their API.
1 change: 1 addition & 0 deletions master/buildbot/newsfragments/getPrevSuccessfulBuild-number-of-sql-queries.bugfix
View file
@@ -0,0 +1 @@
Reduced the number of SQL queries triggered by ``getPrevSuccessfulBuild()`` by up to 100.
1 change: 1 addition & 0 deletions master/buildbot/newsfragments/github_reporter_erroring.bugfix
View file
@@ -0,0 +1 @@
Fix Github error reporting to handle exceptions that happen before the http request is sent
2 changes: 2 additions & 0 deletions master/buildbot/newsfragments/support_recursive_glob_MultipleFileUpload.feature
View file
@@ -0,0 +1,2 @@
Support recursive matching ('**') in MultipleFileUpload when `glob=True`.
This is only supported in python3.5+
1 change: 1 addition & 0 deletions master/buildbot/newsfragments/trigger_step_ui.feature
View file
@@ -0,0 +1 @@
UI now shows a paginated view for trigger step sub builds
6 changes: 5 additions & 1 deletion master/buildbot/reporters/gerrit.py
View file
Expand Up @@ -304,7 +304,11 @@ def buildComplete(self, key, build):
def getBuildDetails(self, build):
br = yield self.master.data.get(("buildrequests", build['buildrequestid']))
buildset = yield self.master.data.get(("buildsets", br['buildsetid']))
yield utils.getDetailsForBuilds(self.master, buildset, [build], wantProperties=True)
yield utils.getDetailsForBuilds(self.master,
buildset,
[build],
wantProperties=True,
wantSteps=self.wantSteps)

def isBuildReported(self, build):
return self.builders is None or build['builder']['name'] in self.builders
Expand Down
9 changes: 7 additions & 2 deletions master/buildbot/reporters/github.py
View file
Expand Up @@ -149,6 +149,7 @@ def send(self, build):

for sourcestamp in sourcestamps:
sha = sourcestamp['revision']
response = None
try:
repo_user = repoOwner
repo_name = repoName
Expand All @@ -174,15 +175,19 @@ def send(self, build):
state=state, repoOwner=repoOwner, repoName=repoName,
sha=sha, issue=issue, context=context))
except Exception as e:
content = yield response.content()
if response:
content = yield response.content()
code = response.code
else:
content = code = "n/a"
log.err(
e,
'Failed to update "{state}" for {repoOwner}/{repoName} '
'at {sha}, context "{context}", issue {issue}. '
'http {code}, {content}'.format(
state=state, repoOwner=repoOwner, repoName=repoName,
sha=sha, issue=issue, context=context,
code=response.code, content=content))
code=code, content=content))


class GitHubCommentPush(GitHubStatusPush):
Expand Down
37 changes: 37 additions & 0 deletions master/buildbot/test/fake/fakedb.py
View file
Expand Up @@ -1956,6 +1956,43 @@ def setBuildProperty(self, bid, name, value, source):
self.builds[bid]['properties'][name] = (value, source)
return defer.succeed(None)

@defer.inlineCallbacks
def getBuildsForChange(self, changeid):
change = yield self.db.changes.getChange(changeid)
bsets = yield self.db.buildsets.getBuildsets()
breqs = yield self.db.buildrequests.getBuildRequests()
builds = yield self.db.builds.getBuilds()

results = []
for bset in bsets:
for ssid in bset['sourcestamps']:
if change['sourcestampid'] == ssid:
bset['changeid'] = changeid
results.append({'buildsetid': bset['bsid']})

for breq in breqs:
for result in results:
if result['buildsetid'] == breq['buildsetid']:
result['buildrequestid'] = breq['buildrequestid']

for build in builds:
for result in results:
if result['buildrequestid'] == build['buildrequestid']:
result['id'] = build['id']
result['number'] = build['number']
result['builderid'] = build['builderid']
result['workerid'] = build['workerid']
result['masterid'] = build['masterid']
result['started_at'] = epoch2datetime(1304262222)
result['complete_at'] = build['complete_at']
result['state_string'] = build['state_string']
result['results'] = build['results']

for result in results:
del result['buildsetid']

return results


class FakeStepsComponent(FakeDBComponent):

Expand Down
48 changes: 47 additions & 1 deletion master/buildbot/test/unit/test_db_builds.py
View file
Expand Up @@ -30,6 +30,7 @@
TIME2 = 1304262223
TIME3 = 1304262224
TIME4 = 1304262235
CREATED_AT = 927845299


class Tests(interfaces.InterfaceTests):
Expand Down Expand Up @@ -185,6 +186,50 @@ def test_getBuilds_workerid(self):
self.assertEqual(sorted(bdicts, key=lambda bd: bd['id']),
[self.threeBdicts[50], self.threeBdicts[51]])

def test_signature_getBuildsForChange(self):
@self.assertArgSpecMatches(self.db.builds.getBuildsForChange)
def getBuildsForChange(self, changeid):
pass

@defer.inlineCallbacks
def do_test_getBuildsForChange(self, rows, changeid, expected):
yield self.insertTestData(rows)

builds = yield self.db.builds.getBuildsForChange(changeid)

self.assertEqual(sorted(builds), sorted(expected))

def test_getBuildsForChange_OneCodebase(self):
rows = [fakedb.Master(id=88, name="bar"),
fakedb.Worker(id=13, name='one'),
fakedb.Builder(id=77, name='A'),
fakedb.SourceStamp(id=234, created_at=CREATED_AT,
revision="aaa"),
fakedb.Change(changeid=14, codebase='A', sourcestampid=234),
fakedb.Buildset(id=30, reason='foo',
submitted_at=1300305712, results=1),
fakedb.BuildsetSourceStamp(sourcestampid=234, buildsetid=30),
fakedb.BuildRequest(id=19, buildsetid=30, builderid=77,
priority=13, submitted_at=1300305712, results=1,
complete=0, complete_at=None),
fakedb.Build(id=50, buildrequestid=19, number=5, masterid=88,
builderid=77, state_string="test", workerid=13,
started_at=1304262222, results=1), ]

expected = [{
'id': 50,
'number': 5,
'builderid': 77,
'buildrequestid': 19,
'workerid': 13,
'masterid': 88,
'started_at': epoch2datetime(1304262222),
'complete_at': None,
'state_string': 'test',
'results': 1}]

return self.do_test_getBuildsForChange(rows, 14, expected)

@defer.inlineCallbacks
def test_getBuilds_complete(self):
yield self.insertTestData(self.backgroundData + self.threeBuilds)
Expand Down Expand Up @@ -443,7 +488,8 @@ class TestRealDB(unittest.TestCase,
def setUp(self):
yield self.setUpConnectorComponent(
table_names=['builds', 'builders', 'masters', 'buildrequests',
'buildsets', 'workers', 'build_properties'])
'buildsets', 'workers', 'build_properties', 'changes',
'sourcestamps', 'buildset_sourcestamps', 'patches'])

self.db.builds = builds.BuildsConnectorComponent(self.db)

Expand Down
2 changes: 1 addition & 1 deletion master/buildbot/test/unit/test_process_buildrequestdistributor.py
View file
Expand Up @@ -396,7 +396,7 @@ def do_test_maybeStartBuildsOnBuilder(self, rows=None, exp_claims=None, exp_buil
self.assertBuildsStarted(exp_builds)

@defer.inlineCallbacks
def test_no_buildreqests(self):
def test_no_buildrequests(self):
self.addWorkers({'test-worker11': 1})
yield self.do_test_maybeStartBuildsOnBuilder(exp_claims=[], exp_builds=[])

Expand Down

0 comments on commit 701485f

Please sign in to comment.