New release deploy script #5363

Merged
merged 7 commits into from Oct 18, 2016

Projects

None yet

1 participant

@bryevdv
Member
bryevdv commented Oct 17, 2016 edited

issues: fixes #5325

This is WIP but would appreciate early feedback, in particular suggestions for how to add testing for it. Things remaining:

  • generate CHANGELOG
  • test upload to CDN?

I'm not a huge fan of the global "config" object but I wanted the test flow call sequence to be clear above all else. I also wanted this code (with the exception of issues.py which is useful by itself) to all be in "one place".

@bryevdv bryevdv added the status: WIP label Oct 17, 2016
@bryevdv bryevdv changed the title from initial commit to New release deploy script Oct 17, 2016
@bryevdv
Member
bryevdv commented Oct 18, 2016 edited

OK, here is a current list of checks:

[PASS] Running Python 3.x
[PASS] Command 'git' is available
[PASS] Git config user.email 'bryanv@continuum.io' found in MAINTAINERS file
[PASS] Executing inside the the bokeh/bokeh repository
[PASS] Working on master branch
[PASS] Local checkout is clean
[PASS] Checkout is up to date with GitHub
[PASS] New version '0.12.4' is a valid Bokeh version (FULL RELEASE)
[PASS] Detected valid last dev/rc/full version '0.12.4dev1'
[PASS] Detected valid last full release version '0.12.3'
[PASS] New version '0.12.4' is newer than last version '0.12.4dev1'
[PASS] Release branch 'release_0.12.4' does not already exist
[FAIL] Issues cannot be checked right now due to GitHub rate limiting

Among other things, the issues check and CHANGELOG update can fail due to GH rate limiting, and this is now protected against. If the local checkout is not up to date with GH it will state whether a push or pull is needed (or if they have diverged). If the local checkout is not clean, it will state which files are dirty.

I also added the notion of MAINTAINERS file which is just a list of email addresses of people who are responsible for releases. right now this is me, @birdsarah, @canavandl and @mattpap

As an aside, this script is a bit difficult to test, since it complains when it is not run on master. have to copy the files out of the branch, check out master, and copy them back in. There's a temporary exclusion in the check repo that ignores a few particular dirty files.

@bryevdv
Member
bryevdv commented Oct 18, 2016

Here's some failed output:


================================================================================
            You are starting a Bokeh release deployment for '0.12.4'
================================================================================

!!! Running pre-checks for release deploy

[PASS] Running Python 3.x
[PASS] Command 'git' is available
[PASS] Git config user.email 'bryanv@continuum.io' found in MAINTAINERS file
[PASS] Executing inside the the bokeh/bokeh repository
[PASS] Working on master branch
[FAIL] Local checkout is NOT clean

     M scripts/issues.py
     ?? MAINTAINERS
     ?? scripts/deploy.py

[PASS] Checkout is up to date with GitHub
[PASS] New version '0.12.4' is a valid Bokeh version (FULL RELEASE)
[PASS] Detected valid last dev/rc/full version '0.12.4dev1'
[PASS] Detected valid last full release version '0.12.3'
[PASS] New version '0.12.4' is newer than last version '0.12.4dev1'
[PASS] Release branch 'release_0.12.4' does not already exist
[FAIL] Issue labels are NOT BEP-1 compliant

     INFO:root:finding issues...
     WARNING:root:closed issue with no reso label: #5323 [component: bokehjs] Colormapper special colors are not respected for images
     WARNING:root:issue with no type label: #5334 Multiple callbacks to one widget
     WARNING:root:issue with no type label: #5341 How to draw a stacked bar with line in same page?
     WARNING:root:issue with no type label: #5357 Dategaptickformatter 'import' and 'export' may appear only with 'sourcetype: module'
     WARNING:root:closed issue with no reso label: #5357 Dategaptickformatter 'import' and 'export' may appear only with 'sourcetype: module'
     WARNING:root:issue with no type label: #5368 Feature request: add bokeh command support for streaming to stdout



!!! Some pre-checks have failed:

    - Local checkout is NOT clean
    - Issue labels are NOT BEP-1 compliant

!!! The deploy has been aborted.

!!! NO REMOTE ACTIONS have been taken --- local checkout may be dirty
@bryevdv
Member
bryevdv commented Oct 18, 2016

The last thing I guess I would consider is the check for a test upload to CDN. That's one of the things that failed last release. I'm not sure how to do that that, the credentials would need to be available somehow, and I can't think of a good way to make that happen.

Maybe a better idea is a test upload at the very beginning of the Travis job? Then if that fails, abort early? The situation I am trying to avoid is "half-releases" with things successfully uploaded to PyPI and npm but not to CDN, etc. It's easy enough to delete just a tag, cleaning up "half-releases" is a pain. Thoughts welcome.

@bryevdv bryevdv added status: ready and removed status: WIP labels Oct 18, 2016
@bryevdv
Member
bryevdv commented Oct 18, 2016

@bokeh/dev OK I have removed the last "testing harness" type code, and pushed. I think this is ready. I propose to merge this and test it on a dev build this afternoon. Absent any negative feedback in the next few hours, I will plan to do that.

Also, given the difficulty of testing/maintaining this on a branch, I propose a special dispensation to push changes to this one file deploy.py directly on to master in the future.

@bryevdv
Member
bryevdv commented Oct 18, 2016

re: CDN I am going to work on the "build and upload" script in a separate PR. I will make it check up front, but also perform some number of retries on failure.

@bryevdv
Member
bryevdv commented Oct 18, 2016

@bokeh/dev OK I am planning to merge this as soon as the four current CI jobs finish, then also to test out releasing a new dev build.

@bryevdv bryevdv merged commit 6c02419 into master Oct 18, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@bryevdv bryevdv deleted the bryanv/5325_deploy_script branch Oct 18, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment