feat(build_release): Use gradle to publish builds instead of manual c… #1430

jtk54 · 2017-02-16T19:02:52Z

…opy.

ewiseblatt · 2017-02-16T22:22:07Z

dev/build_release.py

@@ -211,28 +212,62 @@ def determine_gradle_root(self, name):
      gradle_root = name if name != 'spinnaker' else self.__project_dir
      return gradle_root

-  def start_build_target(self, name, target):
-      """Start a subprocess to build the designated target.
+  def start_candidate_release(self, name):


Use terminology we established earlier.

candidate releases dont start until builds have been verified.
The build part is initiating an unverified build.

However this function is merely building a subsystem.
Where it falls into the release process is a matter the caller's context.
Dont assume that here. This is just the primitive operator that the
release process is composed with.

Ok, I can rename the function but unfortunately the gradle task that is executed has to be either candidate or final.

ewiseblatt · 2017-02-16T22:26:41Z

dev/build_release.py

+    print 'Building and publishing {name}...'.format(name=name)
+    return BackgroundProcess.spawn(
+      'Building and publishing {name}...'.format(name=name),
+      'cd "{gradle_root}"; ./gradlew {extra} candidate'.format(


I dont like "candidate" here. Or if it is we need to change our terminology. However calling these things candidates prematurely causes confusion later as we discovered earlier on defining the process.

candidate is the Nebula gradle task name. The only other choice that behaves the way we want is final at the moment, and that doesn't seem better. I'm not too sure about our ability to rename these.

ewiseblatt · 2017-02-16T22:29:09Z

dev/build_release.py

+      BackgroundProcess
+    """
+    jarRepo = self.__options.jar_repo
+    parts = self.__options.bintray_repo.split('/')


Do these come from gradle properties?
I'm ok with environment variables.
I thought the current gradle plugins used properties.

These are flags passed into this script, which can be populated by environment variables. We then use them as properties in the gradle task (which are also flags/arguments).

ewiseblatt · 2017-02-16T22:31:59Z

dev/build_release.py

@@ -529,6 +511,14 @@ def init_argument_parser(cls, parser):
        '--bintray_repo', default='',
        help='Publish to this bintray repo.\n'
             'This requires BINTRAY_USER and BINTRAY_KEY are set.')
+      parser.add_argument(


jar repo shouldnt have a default.
Maybe this should be a property?

Is this jar repo write only or is it also read?
When other people run this script, they shouldnt publish to the official repo, but should be able to read form it rather than building everything themselves. But if they do build things, they should read those from their repo. So this should be for writing, and there should be a path set somewhere that this has precedence of the existing external path for jar files in this repo but not elsewhere.

First part makes sense. I'm confused at the second.

The jar repo is read/write as long as you have a valid key with permission to that repo or it is public. I agree that everyone shouldn't be able to publish to the official repo, but I don't understand why users would want to read from the official repo in this case. This is for building from source, not copying upstream, prebuilt artifacts. As you put it, this is for building artifacts and writing to bintray_repo and jar_repo. Consuming artifacts from those repos is a separate concern.

I could be confused.

If I am building jars I put them in this repository. Fine. But I dont build every jar I depend on, only the ones defined in this [git] repository. I dont know how this works under the covers, but isnt clear how the precedence and scoping works (or how things get removed so that I can use official external jars even if I may have built those projects myself earlier so the same versions are in my repo (built were built with a different tool chain).

This jar_repo is the target repo for the microservice jars, not the dependency jars. For example, the jars produced from testing this script: https://bintray.com/spinnaker-team/jacobkiefer-jars. My understanding is all the dependency jars are downloaded to a local repository when the microservices are being built, and are pulled at whatever version is specified in build.gradle.

ewiseblatt · 2017-02-16T23:20:01Z

ok. add a comment.

…

On Thu, Feb 16, 2017 at 6:14 PM, Jacob Kiefer ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In dev/build_release.py <#1430 (comment)>: > @@ -211,28 +212,62 @@ def determine_gradle_root(self, name): gradle_root = name if name != 'spinnaker' else self.__project_dir return gradle_root - def start_build_target(self, name, target): - """Start a subprocess to build the designated target. + def start_candidate_release(self, name): Ok, I can rename the function but unfortunately the gradle task that is executed has to be either candidate or final. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#1430 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEKkGL5iLFgEkn3zVLuVtY-MHUGbwX2Rks5rdNhOgaJpZM4MDafG> .

ewiseblatt · 2017-02-16T23:53:23Z

So what happens if I want to build my own kork?

…

On Thu, Feb 16, 2017 at 6:46 PM, Jacob Kiefer ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In dev/build_release.py <#1430 (comment)>: > @@ -529,6 +511,14 @@ def init_argument_parser(cls, parser): '--bintray_repo', default='', help='Publish to this bintray repo.\n' 'This requires BINTRAY_USER and BINTRAY_KEY are set.') + parser.add_argument( This jar_repo is the target repo for the microservice jars, not the dependency jars. For example, the jars produced from testing this script: https://bintray.com/spinnaker-team/jacobkiefer-jars. My understanding is all the dependency jars are downloaded to a local repository when the microservices are being built, and are pulled at whatever version is specified in build.gradle. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#1430 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEKkGLM9Y7f3YGKR6i28Ro3Du6P7HR_7ks5rdN_bgaJpZM4MDafG> .

jtk54 · 2017-02-23T17:14:37Z

Kork would need to be built and published and the up-to-date version specified in spinnaker dependencies. If you wanted to include your own version of kork in the microservices, then you'd need to modify each build.gradle to point to a different target maven repository.

ewiseblatt · 2017-02-23T17:24:57Z

dev/build_release.py

+        help='Publish produced jars to this repo.\n'
+             'This requires BINTRAY_USER and BINTRAY_KEY are set.')
+      parser.add_argument(
+        '--build_number', default='',


Is this required? What's the default or is it left out if not specified?

ewiseblatt · 2017-02-23T17:27:03Z

dev/build_release.py

@@ -185,6 +185,7 @@ def __init__(self, options):
      self.__background_processes = []

      os.environ['NODE_ENV'] = os.environ.get('NODE_ENV', 'dev')
+      self.__build_number = options.build_number or os.environ.get('BUILD_NUMBER', '')


set the argparser default to os.environ.get('BUILD_NUMBER', '')

ewiseblatt · 2017-02-23T17:27:32Z

dev/build_release.py

+    extra_args = [
+      '--stacktrace',
+      '-Prelease.useLastTag=true',
+      '-PbintrayPackageBuildNumber={number}'.format(number=self.__build_number),


build_number could be empty. is that ok?

The gradle task interprets an empty string as 'no build number' and names the package accordingly.

ewiseblatt

LGTM

…opy.

jtk54 requested a review from ewiseblatt February 16, 2017 19:03

ewiseblatt reviewed Feb 16, 2017

View reviewed changes

jtk54 force-pushed the gradle-publish branch 3 times, most recently from 22f3b65 to cccec5b Compare February 16, 2017 23:37

ewiseblatt reviewed Feb 23, 2017

View reviewed changes

feat(build_release): Use gradle to publish builds instead of manual c…

afd3fde

…opy.

jtk54 force-pushed the gradle-publish branch from cccec5b to afd3fde Compare February 23, 2017 18:18

jtk54 merged commit 66d0952 into spinnaker:master Feb 23, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(build_release): Use gradle to publish builds instead of manual c… #1430

feat(build_release): Use gradle to publish builds instead of manual c… #1430

jtk54 commented Feb 16, 2017

ewiseblatt Feb 16, 2017

jtk54 Feb 16, 2017

ewiseblatt Feb 16, 2017

jtk54 Feb 16, 2017

ewiseblatt Feb 16, 2017

jtk54 Feb 16, 2017 •

edited

ewiseblatt Feb 16, 2017

jtk54 Feb 16, 2017

ewiseblatt Feb 16, 2017

jtk54 Feb 16, 2017

ewiseblatt commented Feb 16, 2017 via email

ewiseblatt commented Feb 16, 2017 via email

jtk54 commented Feb 23, 2017 •

edited

ewiseblatt Feb 23, 2017

ewiseblatt Feb 23, 2017

ewiseblatt Feb 23, 2017

jtk54 Feb 23, 2017

ewiseblatt left a comment

feat(build_release): Use gradle to publish builds instead of manual c… #1430

feat(build_release): Use gradle to publish builds instead of manual c… #1430

Conversation

jtk54 commented Feb 16, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtk54 Feb 16, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ewiseblatt commented Feb 16, 2017 via email

ewiseblatt commented Feb 16, 2017 via email

jtk54 commented Feb 23, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ewiseblatt left a comment

Choose a reason for hiding this comment

jtk54 Feb 16, 2017 •

edited

jtk54 commented Feb 23, 2017 •

edited