Skip to content

Publish

Jump to bottom
Nick Miyake edited this page Apr 2, 2021 · 17 revisions

Summary

./godelw publish publishes distributions for the products in the project based on the dist configuration.

Tutorial start state

  • ${GOPATH}/src/${PROJECT_PATH} exists, is the working directory and is initialized as a Git repository and Go module
  • Project contains godel and godelw
  • Project contains main.go
  • Project contains .gitignore that ignores GoLand files
  • Project contains echo/echo.go, echo/echo_test.go and echo/echoer.go
  • godel/config/dist-plugin.yml is configured to build echgo2
  • Project is tagged as 0.0.1
  • godel/config/dist-plugin.yml is configured to create distributions for echgo

Publish

Our project is now configured to build binaries and distributions. The next step is to have a way to publish these distributions so that they can be downloaded and consumed by users.

The publish task can be used to publish the distributions created by the dist task. The publish task has built-in support for publishing to the following common distribution platforms, as well as to the local file system:

  • GitHub releases
  • Bintray
  • Artifactory

It is also possible to add custom publishers using assets.

The publish task is most commonly run in a CI environment, but it is possible to run locally as well. We will walk through an example of publishing this product using GitHub releases. The "More" section also provides documentation on performing a local publish operation (which does not require setting up a GitHub repository and token).

In order to publish this product on GitHub, we will need a GitHub repository and a GitHub API token. This tutorial uses github.com/nmiyake/echgo2 as the GitHub repository. To follow along with this tutorial, create a repository in your own GitHub space called echgo2. GitHub provides instructions for this if you have not done so before.

Because we already have a repository locally, you can follow the instructions provided by GitHub for pushing a local Git repository to the newly created repository. The following is an example of pushing the local repository to the nmiyake/echgo2 GitHub repository set up using SSH:

➜ git remote add origin git@github.com:nmiyake/echgo2.git
➜ git push -u origin master
Counting objects: 48, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (39/39), done.
Writing objects: 100% (48/48), 7.04 KiB | 0 bytes/s, done.
Total 48 (delta 15), reused 0 (delta 0)
remote: Resolving deltas: 100% (15/15), done.
To git@github.com:nmiyake/echgo2.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin by rebasing.

We will perform a release for a 0.0.2 of our product. Create a 0.0.2 tag and push the tags to the repository:

➜ git tag 0.0.2
➜ ./godelw dist
Building echgo2 for darwin-amd64 at out/build/echgo2/0.0.2/darwin-amd64/echgo2
Building echgo2 for linux-amd64 at out/build/echgo2/0.0.2/linux-amd64/echgo2
Finished building echgo2 for linux-amd64 (0.226s)
Finished building echgo2 for darwin-amd64 (0.229s)
Creating distribution for echgo2 at out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz, out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz
Finished creating os-arch-bin distribution for echgo2

➜ git push origin --tags
Total 0 (delta 0), reused 0 (delta 0)
To git@github.com:nmiyake/echgo2.git
 * [new tag]         0.0.1 -> 0.0.1
 * [new tag]         0.0.2 -> 0.0.2

Publishing to GitHub requires an access token. Follow the GitHub instructions for creating a personal access token. For this example, the token requires the public_repo scope.

Once you have created the token, run the following command, replacing nmiyake with your own username and <token> with your GitHub token:

➜ ./godelw publish github --api-url https://api.github.com/ --user nmiyake --token <token> --owner nmiyake --repository echgo2
Building echgo2 for darwin-amd64 at /Volumes/git/go2/src/github.com/nmiyake/echgo2/out/build/echgo2/0.0.2/darwin-amd64/echgo2
Building echgo2 for linux-amd64 at /Volumes/git/go2/src/github.com/nmiyake/echgo2/out/build/echgo2/0.0.2/linux-amd64/echgo2
Finished building echgo2 for linux-amd64 (0.308s)
Finished building echgo2 for darwin-amd64 (0.308s)
Creating distribution for echgo2 at /Volumes/git/go2/src/github.com/nmiyake/echgo2/out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz, /Volumes/git/go2/src/github.com/nmiyake/echgo2/out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz
Finished creating os-arch-bin distribution for echgo2
Creating GitHub release 0.0.2 for nmiyake/echgo2...done
Uploading /Volumes/git/go2/src/github.com/nmiyake/echgo2/out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz to https://uploads.github.com/repos/nmiyake/echgo2/releases/10284636/assets?name=echgo2-0.0.2-darwin-amd64.tgz
 800.00 KiB / 800.00 KiB [==================================================================================] 100.00% 0s
Uploading /Volumes/git/go2/src/github.com/nmiyake/echgo2/out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz to https://uploads.github.com/repos/nmiyake/echgo2/releases/10284636/assets?name=echgo2-0.0.2-linux-amd64.tgz
 766.73 KiB / 766.73 KiB [==================================================================================] 100.00% 0s

As described by the output, the publish task has created a GitHub release called 0.0.2 (the name of the tag) and has uploaded the distribution artifacts to the release.

Navigate to the "releases" section of the repository to verify that the distribution artifacts were uploaded:

github release

The publish task can be configured to run in a CI environment on release tags.

Tutorial end state

  • ${GOPATH}/src/${PROJECT_PATH} exists, is the working directory and is initialized as a Git repository and Go module
  • Project contains godel and godelw
  • Project contains main.go
  • Project contains .gitignore that ignores GoLand files
  • Project contains echo/echo.go, echo/echo_test.go and echo/echoer.go
  • godel/config/dist-plugin.yml is configured to build echgo2
  • Project is tagged as 0.0.1
  • godel/config/dist-plugin.yml is configured to create distributions for echgo
  • Project is tagged as 0.0.2

Tutorial next step

Build and push Docker images

More

Specifying a Maven group ID

Some publish operations use a Maven group ID to determine the publish location. The "local" publisher is one such publisher. Attempting to run a local publish will fail:

➜ ./godelw publish maven-local --dry-run
Error: failed to publish echgo2 using maven-local publisher: group-id was not specified -- it must be specified in configuration or using a flag

As indicated by the error message, the group-id can be specified using a flag:

➜ ./godelw publish maven-local --dry-run --group-id com.palantir.echgo2
[DRY RUN] Writing POM to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2.pom
[DRY RUN] Copying artifact from out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-darwin-amd64.tgz
[DRY RUN] Copying artifact from out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-linux-amd64.tgz

Although this approach works for one-off publishing operations, if the group ID is conceptually part of the product, it should be specified as part of configuration so that all publish operations that require a maven group ID can use it.

Run the following to add a publish block with a group-id field:

➜ echo 'products:
  echgo2:
    build:
      main-pkg: .
      version-var: main.version
      os-archs:
        - os: darwin
          arch: amd64
        - os: linux
          arch: amd64
    dist:
      disters:
        type: os-arch-bin
        config:
          os-archs:
            - os: darwin
              arch: amd64
            - os: linux
              arch: amd64
    publish:
      group-id: com.palantir.echgo2' > godel/config/dist-plugin.yml

Now that the group ID is specified as part of configuration, any publish operation that requires a Maven group ID can use it. First, run the ./godelw dist task to ensure that the dist outputs exist:

➜ ./godelw dist
Building echgo2 for darwin-amd64 at out/build/echgo2/0.0.2.dirty/darwin-amd64/echgo2
Building echgo2 for linux-amd64 at out/build/echgo2/0.0.2.dirty/linux-amd64/echgo2
Finished building echgo2 for linux-amd64 (0.206s)
Finished building echgo2 for darwin-amd64 (0.218s)
Creating distribution for echgo2 at out/dist/echgo2/0.0.2.dirty/os-arch-bin/echgo2-0.0.2.dirty-darwin-amd64.tgz, out/dist/echgo2/0.0.2.dirty/os-arch-bin/echgo2-0.0.2.dirty-linux-amd64.tgz
Finished creating os-arch-bin distribution for echgo2

Now, dry run the publish again without the --group-id flag and verify that it succeeds:

➜ ./godelw publish maven-local --dry-run
[DRY RUN] Creating distribution for echgo2 at out/dist/echgo2/0.0.2.dirty/os-arch-bin/echgo2-0.0.2.dirty-darwin-amd64.tgz, out/dist/echgo2/0.0.2.dirty/os-arch-bin/echgo2-0.0.2.dirty-linux-amd64.tgz
[DRY RUN] Finished creating os-arch-bin distribution for echgo2
[DRY RUN] Writing POM to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2.dirty/echgo2-0.0.2.dirty.pom
[DRY RUN] Copying artifact from out/dist/echgo2/0.0.2.dirty/os-arch-bin/echgo2-0.0.2.dirty-darwin-amd64.tgz to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2.dirty/echgo2-0.0.2.dirty-darwin-amd64.tgz
[DRY RUN] Copying artifact from out/dist/echgo2/0.0.2.dirty/os-arch-bin/echgo2-0.0.2.dirty-linux-amd64.tgz to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2.dirty/echgo2-0.0.2.dirty-linux-amd64.tgz

Revert these changes by running the following:

➜ git checkout -- godel/config/dist-plugin.yml

Maven local publishing

The maven-local publisher type copies the artifacts and a generated POM to a local directory:

➜ ./godelw publish maven-local --dry-run --group-id com.palantir.echgo2
[DRY RUN] Creating distribution for echgo2 at out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz, out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz
[DRY RUN] Finished creating os-arch-bin distribution for echgo2
[DRY RUN] Writing POM to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2.pom
[DRY RUN] Copying artifact from out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-darwin-amd64.tgz
[DRY RUN] Copying artifact from out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz to /root/.m2/repository/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-linux-amd64.tgz

Artifactory publishing

The artifactory publisher type uploads the artifacts and a generated POM to an Artifactory instance:

➜ ./godelw publish artifactory --dry-run --group-id com.palantir.echgo2 --url https://artifactory.domain.com --repository dist-repo --username username --password password
[DRY RUN] Creating distribution for echgo2 at out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz, out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz
[DRY RUN] Finished creating os-arch-bin distribution for echgo2
[DRY RUN] Uploading out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz to https://artifactory.domain.com/artifactory/dist-repo/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-darwin-amd64.tgz
[DRY RUN] Uploading out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz to https://artifactory.domain.com/artifactory/dist-repo/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-linux-amd64.tgz
[DRY RUN] Uploading to https://artifactory.domain.com/artifactory/dist-repo/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2.pom

The Artifactory publish task uses the Artifactory API to compute the SHA-256 checksums for the uploaded artifacts.

Bintray publishing

The bintray publisher type uploads the artifacts and a generated POM to a Bintray instance:

➜ ./godelw publish bintray --dry-run --group-id com.palantir.echgo2 --url https://api.bintray.com --subject org --repository releases --username username --password password --publish --downloads-list
[DRY RUN] Creating distribution for echgo2 at out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz, out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz
[DRY RUN] Finished creating os-arch-bin distribution for echgo2
[DRY RUN] Uploading out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-darwin-amd64.tgz to https://api.bintray.com/content/org/releases/echgo2/0.0.2/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-darwin-amd64.tgz
[DRY RUN] Uploading out/dist/echgo2/0.0.2/os-arch-bin/echgo2-0.0.2-linux-amd64.tgz to https://api.bintray.com/content/org/releases/echgo2/0.0.2/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2-linux-amd64.tgz
[DRY RUN] Uploading to https://api.bintray.com/content/org/releases/echgo2/0.0.2/com/palantir/echgo2/echgo2/0.0.2/echgo2-0.0.2.pom
[DRY RUN] Running Bintray publish for uploaded artifacts...done
[DRY RUN] Adding artifact to Bintray downloads list for package...done
[DRY RUN] Adding artifact to Bintray downloads list for package...done

The --publish flag triggers a Bintray "release" action and the --downloads-list flag adds the artifact to the downloads list of the Bintray page.

Add publishers

It is possible to define and add custom publishers using assets.

For example, consider a fictional publisher asset that publishes to Dropbox with the locator "com.palantir.godel-distgo-asset-publish-dropbox:publish-dropbox-asset:1.0.0". The following configuration in godel/config/godel.yml would add this publisher:

default-tasks:
  resolvers:
    - https://palantir.bintray.com/releases/{{GroupPath}}/{{Product}}/{{Version}}/{{Product}}-{{Version}}-{{OS}}-{{Arch}}.tgz
  tasks:
    com.palantir.distgo:dist-plugin:
      assets:
        - locator:
            id: "com.palantir.godel-distgo-asset-publish-dropbox:publish-dropbox-asset:1.0.0"
Clone this wiki locally