Integrate the native engine release into the release script. #4035

Closed
jsirois opened this Issue Nov 6, 2016 · 7 comments

Comments

Projects
1 participant
@jsirois
Member

jsirois commented Nov 6, 2016

The script will need to build binaries for at least linux/x86_64 and OSX as well as probably linux/i386. In addition it will need to learn upload to bintray.

@jsirois jsirois self-assigned this Nov 6, 2016

@jsirois

This comment has been minimized.

Show comment
Hide comment
@jsirois

jsirois Nov 6, 2016

Member

Spoke with @stuhood about this in slack and I'll aim to finish what's needed on this on Wednesday 11/9 so that I can dogfood in my release duty Thursday 11/10.

Member

jsirois commented Nov 6, 2016

Spoke with @stuhood about this in slack and I'll aim to finish what's needed on this on Wednesday 11/9 so that I can dogfood in my release duty Thursday 11/10.

@jsirois jsirois added the in-progress label Nov 11, 2016

@jsirois

This comment has been minimized.

Show comment
Hide comment
@jsirois

jsirois Nov 11, 2016

Member

OK - targeting OSX from linux and vice-versa is not going to happen easily. I'm going to look into a TravisCI job with a linux/osx matrix that generates the binaries on every master commit, staging them somewhere for the release script to find. Green master means we have a green set of binaries in that case.

Member

jsirois commented Nov 11, 2016

OK - targeting OSX from linux and vice-versa is not going to happen easily. I'm going to look into a TravisCI job with a linux/osx matrix that generates the binaries on every master commit, staging them somewhere for the release script to find. Green master means we have a green set of binaries in that case.

@jsirois

This comment has been minimized.

Show comment
Hide comment
@jsirois

jsirois Nov 11, 2016

Member

Investigating the following:

The idea would be to burn binaries on every master or deploy branch green commit. Since the binaries are using an MD5 hash as their version, this works if the release-prep commit includes now:

  • bump VERSION in src/python/pants/version.py manually
  • bump --version in src/python/pants/engine/subsystem/native.py - can be scripted since the current value is exported by the pants script as both NATIVE_BOOTSTRAP_HASH and PANTS_NATIVE_ENGINE_VERSION, ie:
$ ./pants options --scope=native-engine --name=version --output-format=json | python -c 'import json, sys; print(json.load(sys.stdin)["native-engine.version"]["value"])'
27e0016cfd490ba89830d6e99c93764c4357c268
Member

jsirois commented Nov 11, 2016

Investigating the following:

The idea would be to burn binaries on every master or deploy branch green commit. Since the binaries are using an MD5 hash as their version, this works if the release-prep commit includes now:

  • bump VERSION in src/python/pants/version.py manually
  • bump --version in src/python/pants/engine/subsystem/native.py - can be scripted since the current value is exported by the pants script as both NATIVE_BOOTSTRAP_HASH and PANTS_NATIVE_ENGINE_VERSION, ie:
$ ./pants options --scope=native-engine --name=version --output-format=json | python -c 'import json, sys; print(json.load(sys.stdin)["native-engine.version"]["value"])'
27e0016cfd490ba89830d6e99c93764c4357c268
@jsirois

This comment has been minimized.

Show comment
Hide comment
@jsirois

jsirois Nov 12, 2016

Member

OK - Travis' matrix.include construct does in fact allow mixing in an osx shard to do our osx native engine binary builds. Now to get bintray uploading working...

Member

jsirois commented Nov 12, 2016

OK - Travis' matrix.include construct does in fact allow mixing in an osx shard to do our osx native engine binary builds. Now to get bintray uploading working...

@jsirois

This comment has been minimized.

Show comment
Hide comment
@jsirois

jsirois Nov 12, 2016

Member

OK - #4808 + @3ac4f10 mostly works. This has CI uploading native binaries for linux and OSX x86_64 on green commits landing on origin/master and origin/, ie: stable release branches. If the native engine binary has already been uploaded, the process fails gracefully. The remaining steps are:

  • have the linux shards use rustup to add the i686 (32 bit) target and generate a 32 bit binary and upload that as well
  • have the OSX shard upload its binary for all OSX versions (10.6 - or maybe 10.8+) - right now its just for 10.11 - the version of the OSX CI is running under on travis.
Member

jsirois commented Nov 12, 2016

OK - #4808 + @3ac4f10 mostly works. This has CI uploading native binaries for linux and OSX x86_64 on green commits landing on origin/master and origin/, ie: stable release branches. If the native engine binary has already been uploaded, the process fails gracefully. The remaining steps are:

  • have the linux shards use rustup to add the i686 (32 bit) target and generate a 32 bit binary and upload that as well
  • have the OSX shard upload its binary for all OSX versions (10.6 - or maybe 10.8+) - right now its just for 10.11 - the version of the OSX CI is running under on travis.
@jsirois

This comment has been minimized.

Show comment
Hide comment
@jsirois

jsirois Nov 13, 2016

Member

#4049 with that addition of fixes in @30e034ba04d0977a101089c2f5b4162c1ed91915, @4e40a16bf0b405cf54369fa338ac385130c08e12, @64c6ef5d0549ae77f3153698600530fc06843370 and @877bd0e1b10c5b84a819a88d2d2885474857833b has OSX 10.7+ binaries being automatically published to bintray from the stable branches. The only supported platform left to address is linux 32 bit.

Member

jsirois commented Nov 13, 2016

#4049 with that addition of fixes in @30e034ba04d0977a101089c2f5b4162c1ed91915, @4e40a16bf0b405cf54369fa338ac385130c08e12, @64c6ef5d0549ae77f3153698600530fc06843370 and @877bd0e1b10c5b84a819a88d2d2885474857833b has OSX 10.7+ binaries being automatically published to bintray from the stable branches. The only supported platform left to address is linux 32 bit.

@jsirois

This comment has been minimized.

Show comment
Hide comment
@jsirois

jsirois Nov 16, 2016

Member

As of 396e3e1 the native engine binaries are now available on every green master / stable branch commit and so the release process need not be changed.

Member

jsirois commented Nov 16, 2016

As of 396e3e1 the native engine binaries are now available on every green master / stable branch commit and so the release process need not be changed.

@jsirois jsirois closed this Nov 16, 2016

@jsirois jsirois removed the in-progress label Nov 16, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment